[PATCH] toaster: log ProcessLookupError in test_runbuilds

[PATCH] toaster: log ProcessLookupError in test_runbuilds

[Tim Orling]

This repeatably fails:

Exception in thread Thread-2:
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File ".../poky/bitbake/lib/toaster/tests/commands/test_runbuilds.py", line 39, in run
    os.kill(int(pid), signal.SIGTERM)
ProcessLookupError: [Errno 3] No such process

Rather than have a hard error, add logging and output as a warning.

Signed-off-by: Tim Orling <tim.orling@konsulko.com>
---
 lib/toaster/tests/commands/test_runbuilds.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/lib/toaster/tests/commands/test_runbuilds.py b/lib/toaster/tests/commands/test_runbuilds.py
index c77d6cf4..14b35764 100644
--- a/lib/toaster/tests/commands/test_runbuilds.py
+++ b/lib/toaster/tests/commands/test_runbuilds.py
@@ -19,6 +19,10 @@ import time
 import subprocess
 import signal
 
+import logging
+
+logger = logging.getLogger("toaster")
+
 
 class KillRunbuilds(threading.Thread):
     """ Kill the runbuilds process after an amount of time """
@@ -36,8 +40,11 @@ class KillRunbuilds(threading.Thread):
 
         with open(pidfile_path) as pidfile:
             pid = pidfile.read()
-            os.kill(int(pid), signal.SIGTERM)
-
+            try:
+                os.kill(int(pid), signal.SIGTERM)
+            except ProcessLookupError as err:
+                logger.warning("Failed to kill pid. %s" % err)
+                pass
 
 class TestCommands(TestCase):
     """ Sanity test that runbuilds executes OK """
-- 
2.34.1

Re: [Toaster] [PATCH] toaster: log ProcessLookupError in test_runbuilds

[Richard Purdie]

On Fri, 2023-11-10 at 07:30 -0800, Tim Orling wrote:
> This repeatably fails:
> 
> Exception in thread Thread-2:
> Traceback (most recent call last):
>   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
>     self.run()
>   File ".../poky/bitbake/lib/toaster/tests/commands/test_runbuilds.py", line 39, in run
>     os.kill(int(pid), signal.SIGTERM)
> ProcessLookupError: [Errno 3] No such process
> 
> Rather than have a hard error, add logging and output as a warning.
> 
> Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> ---
>  lib/toaster/tests/commands/test_runbuilds.py | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/toaster/tests/commands/test_runbuilds.py b/lib/toaster/tests/commands/test_runbuilds.py
> index c77d6cf4..14b35764 100644
> --- a/lib/toaster/tests/commands/test_runbuilds.py
> +++ b/lib/toaster/tests/commands/test_runbuilds.py
> @@ -19,6 +19,10 @@ import time
>  import subprocess
>  import signal
>  
> +import logging
> +
> +logger = logging.getLogger("toaster")
> +
>  
>  class KillRunbuilds(threading.Thread):
>      """ Kill the runbuilds process after an amount of time """
> @@ -36,8 +40,11 @@ class KillRunbuilds(threading.Thread):
>  
>          with open(pidfile_path) as pidfile:
>              pid = pidfile.read()
> -            os.kill(int(pid), signal.SIGTERM)
> -
> +            try:
> +                os.kill(int(pid), signal.SIGTERM)
> +            except ProcessLookupError as err:
> +                logger.warning("Failed to kill pid. %s" % err)
> +                pass
>  

Is this a case of "just ensure bitbake has really exited"? If so we
perhaps don't need the warning as it is quite likely it has exited?

Cheers,

Richard

Re: [Toaster] [PATCH] toaster: log ProcessLookupError in test_runbuilds

[Tim Orling]

[-- Attachment #1: Type: text/plain, Size: 2372 bytes --]

Ignore this patch. More analysis is required.

On Fri, Nov 10, 2023 at 10:01 AM Richard Purdie <
richard.purdie@linuxfoundation.org> wrote:

> On Fri, 2023-11-10 at 07:30 -0800, Tim Orling wrote:
> > This repeatably fails:
> >
> > Exception in thread Thread-2:
> > Traceback (most recent call last):
> >   File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
> >     self.run()
> >   File ".../poky/bitbake/lib/toaster/tests/commands/test_runbuilds.py",
> line 39, in run
> >     os.kill(int(pid), signal.SIGTERM)
> > ProcessLookupError: [Errno 3] No such process
> >
> > Rather than have a hard error, add logging and output as a warning.
> >
> > Signed-off-by: Tim Orling <tim.orling@konsulko.com>
> > ---
> >  lib/toaster/tests/commands/test_runbuilds.py | 11 +++++++++--
> >  1 file changed, 9 insertions(+), 2 deletions(-)
> >
> > diff --git a/lib/toaster/tests/commands/test_runbuilds.py
> b/lib/toaster/tests/commands/test_runbuilds.py
> > index c77d6cf4..14b35764 100644
> > --- a/lib/toaster/tests/commands/test_runbuilds.py
> > +++ b/lib/toaster/tests/commands/test_runbuilds.py
> > @@ -19,6 +19,10 @@ import time
> >  import subprocess
> >  import signal
> >
> > +import logging
> > +
> > +logger = logging.getLogger("toaster")
> > +
> >
> >  class KillRunbuilds(threading.Thread):
> >      """ Kill the runbuilds process after an amount of time """
> > @@ -36,8 +40,11 @@ class KillRunbuilds(threading.Thread):
> >
> >          with open(pidfile_path) as pidfile:
> >              pid = pidfile.read()
> > -            os.kill(int(pid), signal.SIGTERM)
> > -
> > +            try:
> > +                os.kill(int(pid), signal.SIGTERM)
> > +            except ProcessLookupError as err:
> > +                logger.warning("Failed to kill pid. %s" % err)
> > +                pass
> >
>
> Is this a case of "just ensure bitbake has really exited"? If so we
> perhaps don't need the warning as it is quite likely it has exited?
>
>
When Toaster starts, the .runbuild.pid file is created in the $BUILDDIR.
Manually running 'manage.py runbuilds' should leave a process running.
My suspicion is the path to .runbuilds.pid is somehow getting mangled or
a stale, old .runbuilds.pid is sticking around. Something is wonky in the
test environment.



> Cheers,
>
> Richard
>
>

[-- Attachment #2: Type: text/html, Size: 3378 bytes --]