All of lore.kernel.org
 help / color / mirror / Atom feed
From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: bitbake-devel@lists.openembedded.org
Subject: [PATCH] main/process: Add extra sockname debugging
Date: Tue, 13 Dec 2022 23:25:02 +0000	[thread overview]
Message-ID: <20221213232502.114660-1-richard.purdie@linuxfoundation.org> (raw)

We're struggling to understand how bitbake.sock can sometimes disappear
in live builds when we can't see where it could have been deleted.
This causes connection failures to the server and failed builds.

Add some extra debugging around the server log and client retry
log messages to give more information for the next time this issue
occurs.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 lib/bb/main.py           | 3 ++-
 lib/bb/server/process.py | 7 ++++---
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/lib/bb/main.py b/lib/bb/main.py
index ee12256bc8..ed3e1ede61 100755
--- a/lib/bb/main.py
+++ b/lib/bb/main.py
@@ -446,7 +446,8 @@ def setup_bitbake(configParams, extrafeatures=None):
                         logger.info("Previous bitbake instance shutting down?, waiting to retry... (%s)" % timestamp())
                         procs = bb.server.process.get_lockfile_process_msg(lockfile)
                         if procs:
-                            logger.info("Processes holding bitbake.lock:\n%s" % procs)
+                            logger.info("Processes holding bitbake.lock (missing socket %s):\n%s" % (sockname, procs))
+                        logger.info("Directory listing: %s" % (str(os.listdir(topdir))))
                         i = 0
                         lock = None
                         # Wait for 5s or until we can get the lock
diff --git a/lib/bb/server/process.py b/lib/bb/server/process.py
index f4ab80ba67..44c65451fc 100644
--- a/lib/bb/server/process.py
+++ b/lib/bb/server/process.py
@@ -154,9 +154,10 @@ class ProcessServer():
             fds.append(self.xmlrpc)
         seendata = False
         serverlog("Entering server connection loop")
+        serverlog("Lockfile is: %s\nSocket is %s (%s)" % (self.bitbake_lock_name, self.sockname, os.path.exists(self.sockname)))
 
         def disconnect_client(self, fds):
-            serverlog("Disconnecting Client")
+            serverlog("Disconnecting Client (socket: %s)" % os.path.exists(self.sockname))
             if self.controllersock:
                 fds.remove(self.controllersock)
                 self.controllersock.close()
@@ -246,7 +247,7 @@ class ProcessServer():
                 try:
                     serverlog("Running command %s" % command)
                     self.command_channel_reply.send(self.cooker.command.runCommand(command))
-                    serverlog("Command Completed")
+                    serverlog("Command Completed (socket: %s)" % os.path.exists(self.sockname))
                 except Exception as e:
                    stack = traceback.format_exc()
                    serverlog('Exception in server main event loop running command %s (%s)' % (command, stack))
@@ -273,7 +274,7 @@ class ProcessServer():
 
             ready = self.idle_commands(.1, fds)
 
-        serverlog("Exiting")
+        serverlog("Exiting (socket: %s)" % os.path.exists(self.sockname))
         # Remove the socket file so we don't get any more connections to avoid races
         try:
             os.unlink(self.sockname)
-- 
2.37.2



                 reply	other threads:[~2022-12-13 23:25 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20221213232502.114660-1-richard.purdie@linuxfoundation.org \
    --to=richard.purdie@linuxfoundation.org \
    --cc=bitbake-devel@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.