[PATCH 0/5] rteval: Fixes and speedups

[PATCH 0/5] rteval: Fixes and speedups

[Crystal Wood]

Apart from the first patch, these are mainly motivated by a desire to
make development on rteval more pleasant.  With these patches, a one
second testing run takes around 14 seconds on my laptop with standard
loads, or around 5 seconds with no loads configured (except stressng
without the enabling options, to prevent the default loads from running).

Crystal Wood (5):
  rteval: default_config_search: Return None on failure
  rteval: kcompile: Fix path lookups in _remove_build_dirs
  rteval: kcompile: Skip mrproper, and re-extract if clean fails
  rteval: Break out of main loop faster on interrupt/stoptime
  rteval: Remove 30 second "settling" period

 rteval/__init__.py               | 15 +++++------
 rteval/modules/loads/kcompile.py | 45 ++++++++++++++------------------
 rteval/rtevalConfig.py           |  2 +-
 3 files changed, 27 insertions(+), 35 deletions(-)

-- 
2.43.0

[PATCH 1/5] rteval: default_config_search: Return None on failure

[Crystal Wood]

If False is used without checking, it will be interpreted as stdin,
hanging rteval waiting for input.  OTOH, None will cause os.path.exists()
to throw an exception, so we need to check both the name and the existence
separately anyway.  However, this is a better failure mode than hanging
on stdin if the user of the filename fails to check both.

Signed-off-by: Crystal Wood <crwood@redhat.com>
---
I found this by trying to use "make install", which fails to install the
XSL files... apparently data_files is deprecated.
---
 rteval/__init__.py     | 2 +-
 rteval/rtevalConfig.py | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/rteval/__init__.py b/rteval/__init__.py
index ca018f6fd8b8..5d43615af5bb 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -76,7 +76,7 @@ class RtEval(rtevalReport):
         else:
             self.__mailer = None
 
-        if not os.path.exists(self.__rtevcfg.xslt_report):
+        if not self.__rtevcfg.xslt_report or not os.path.exists(self.__rtevcfg.xslt_report):
             raise RuntimeError(f"can't find XSL template ({self.__rtevcfg.xslt_report})!")
 
         # Add rteval directory into module search path
diff --git a/rteval/rtevalConfig.py b/rteval/rtevalConfig.py
index 030d4205efab..e62da25119a6 100644
--- a/rteval/rtevalConfig.py
+++ b/rteval/rtevalConfig.py
@@ -46,7 +46,7 @@ def default_config_search(relative_path, verifdef=os.path.isdir):
             if verifdef(os.path.join(path, *relative_path)):
                 return os.path.join(path, *relative_path)
 
-    return False
+    return None
 
 
 # HACK: A temporary hack to try to figure out where the install dir is.
-- 
2.43.0

Re: [PATCH 1/5] rteval: default_config_search: Return None on failure

[John Kacur]

On Mon, 4 Mar 2024, Crystal Wood wrote:

> If False is used without checking, it will be interpreted as stdin,
> hanging rteval waiting for input.  OTOH, None will cause os.path.exists()
> to throw an exception, so we need to check both the name and the existence
> separately anyway.  However, this is a better failure mode than hanging
> on stdin if the user of the filename fails to check both.
> 
> Signed-off-by: Crystal Wood <crwood@redhat.com>
> ---
> I found this by trying to use "make install", which fails to install the
> XSL files... apparently data_files is deprecated.
> ---
>  rteval/__init__.py     | 2 +-
>  rteval/rtevalConfig.py | 2 +-
>  2 files changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/rteval/__init__.py b/rteval/__init__.py
> index ca018f6fd8b8..5d43615af5bb 100644
> --- a/rteval/__init__.py
> +++ b/rteval/__init__.py
> @@ -76,7 +76,7 @@ class RtEval(rtevalReport):
>          else:
>              self.__mailer = None
>  
> -        if not os.path.exists(self.__rtevcfg.xslt_report):
> +        if not self.__rtevcfg.xslt_report or not os.path.exists(self.__rtevcfg.xslt_report):
>              raise RuntimeError(f"can't find XSL template ({self.__rtevcfg.xslt_report})!")
>  
>          # Add rteval directory into module search path
> diff --git a/rteval/rtevalConfig.py b/rteval/rtevalConfig.py
> index 030d4205efab..e62da25119a6 100644
> --- a/rteval/rtevalConfig.py
> +++ b/rteval/rtevalConfig.py
> @@ -46,7 +46,7 @@ def default_config_search(relative_path, verifdef=os.path.isdir):
>              if verifdef(os.path.join(path, *relative_path)):
>                  return os.path.join(path, *relative_path)
>  
> -    return False
> +    return None
>  
>  
>  # HACK: A temporary hack to try to figure out where the install dir is.
> -- 
> 2.43.0
> 
> 
> 
Signed-off-by: John Kacur <jkacur@redhat.com>

[PATCH 2/5] rteval: kcompile: Fix path lookups in _remove_build_dirs

[Crystal Wood]

- Use the actual source directory rather than the incorrect kernel* to
  match what the tarball generates
- Use glob to expand the node* wildcard
- Fix the incorrect variable name in the error message

Signed-off-by: Crystal Wood <crwood@redhat.com>
---
 rteval/modules/loads/kcompile.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index b606f7aad202..de1539986723 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -145,12 +145,11 @@ class Kcompile(CommandLineLoad):
             return
         self._log(Log.DEBUG, f"removing kcompile directories in {self.builddir}")
         null = os.open("/dev/null", os.O_RDWR)
-        cmd = ["rm", "-rf", os.path.join(self.builddir, "kernel*"),
-               os.path.join(self.builddir, "node*")]
+        cmd = ["rm", "-rf", self.mydir, *glob.glob(os.path.join(self.builddir, 'node*'))]
         ret = subprocess.call(cmd, stdin=null, stdout=null, stderr=null)
         if ret:
             raise rtevalRuntimeError(self, \
-                f"error removing builddir ({self.buildir}) (ret={ret})")
+                f"error removing builddir ({self.builddir}) (ret={ret})")
 
     def _find_tarball(self):
        # If the user specifies the full kernel name, check if available
-- 
2.43.0

Re: [PATCH 2/5] rteval: kcompile: Fix path lookups in _remove_build_dirs

[John Kacur]

On Mon, 4 Mar 2024, Crystal Wood wrote:

> - Use the actual source directory rather than the incorrect kernel* to
>   match what the tarball generates
> - Use glob to expand the node* wildcard
> - Fix the incorrect variable name in the error message
> 
> Signed-off-by: Crystal Wood <crwood@redhat.com>
> ---
>  rteval/modules/loads/kcompile.py | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)
> 
> diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
> index b606f7aad202..de1539986723 100644
> --- a/rteval/modules/loads/kcompile.py
> +++ b/rteval/modules/loads/kcompile.py
> @@ -145,12 +145,11 @@ class Kcompile(CommandLineLoad):
>              return
>          self._log(Log.DEBUG, f"removing kcompile directories in {self.builddir}")
>          null = os.open("/dev/null", os.O_RDWR)
> -        cmd = ["rm", "-rf", os.path.join(self.builddir, "kernel*"),
> -               os.path.join(self.builddir, "node*")]
> +        cmd = ["rm", "-rf", self.mydir, *glob.glob(os.path.join(self.builddir, 'node*'))]
>          ret = subprocess.call(cmd, stdin=null, stdout=null, stderr=null)
>          if ret:
>              raise rtevalRuntimeError(self, \
> -                f"error removing builddir ({self.buildir}) (ret={ret})")
> +                f"error removing builddir ({self.builddir}) (ret={ret})")
>  
>      def _find_tarball(self):
>         # If the user specifies the full kernel name, check if available
> -- 
> 2.43.0

Signed-off-by: John Kacur <jkacur@redhat.com>

[PATCH 3/5] rteval: kcompile: Skip mrproper, and re-extract if clean fails

[Crystal Wood]

We only ever do out-of-tree builds, so the kernel directory should not be
getting damaged except by incomplete extraction (which mrproper would
probably not fix) or external meddling.  So, skip the mrproper and
instead re-extract if a "make clean" fails.

Also, add -j to cleancmd to further speed things up.  Startup speed may
not seem all that important given how long rteval is typically run for,
but this helps make quick tests (e.g. while debugging things, or when
hunting a latency that shows up very quickly) less painful.

On my 12-cpu laptop, this patch saves about 15 seconds of startup time.

Signed-off-by: Crystal Wood <crwood@redhat.com>
---
 rteval/modules/loads/kcompile.py | 40 ++++++++++++++------------------
 1 file changed, 18 insertions(+), 22 deletions(-)

diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index de1539986723..db1941074157 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -54,7 +54,7 @@ class KBuildJob:
             self.jobs = self.calc_jobs_per_cpu() * cpus_available
 
         self.runcmd = f"make O={self.objdir} -C {self.kdir} -j{self.jobs}"
-        self.cleancmd = f"make O={self.objdir} -C {self.kdir} clean allmodconfig"
+        self.cleancmd = f"make O={self.objdir} -C {self.kdir} clean allmodconfig -j${self.jobs}"
         self.cleancmd += f"&& pushd {self.objdir} && {self.kdir}/scripts/config -d CONFIG_MODULE_SIG_SHA1 -e CONFIG_MODULE_SIG_SHA512 && popd && make O={self.objdir} -C {self.kdir} olddefconfig"
         if self.binder:
             self.runcmd = self.binder + " " + self.runcmd
@@ -90,8 +90,8 @@ class KBuildJob:
     def clean(self, sin=None, sout=None, serr=None):
         """ Runs command to clean any previous builds and configure kernel """
         self.log(Log.DEBUG, f"cleaning objdir {self.objdir}")
-        subprocess.call(self.cleancmd, shell=True,
-                        stdin=sin, stdout=sout, stderr=serr)
+        return subprocess.call(self.cleancmd, shell=True,
+                               stdin=sin, stdout=sout, stderr=serr)
 
     def run(self, sin=None, sout=None, serr=None):
         """ Use Popen to launch a kcompile job """
@@ -234,6 +234,10 @@ class Kcompile(CommandLineLoad):
                 self.logger, self.cpus[n] if self.cpulist else None)
             self.args.append(str(self.buildjobs[n])+";")
 
+    def _repair_tarball(self):
+        self._log(Log.DEBUG, "Invalid state in kernel build tree, reloading")
+        self._remove_build_dirs()
+        self._extract_tarball()
 
     def _WorkloadBuild(self):
         if self._donotrun:
@@ -246,30 +250,22 @@ class Kcompile(CommandLineLoad):
         else:
             out = err = null
 
-        # clean up any damage from previous runs
-        try:
-            cmd = ["make", "-C", self.mydir, "mrproper"]
-            ret = subprocess.call(cmd, stdin=null, stdout=out, stderr=err)
+        # clean up object dirs and make sure each has a config file
+        for n in self.nodes:
+            ret = self.buildjobs[n].clean(sin=null, sout=out, serr=err)
             if ret:
-                # if the above make failed, remove and reinstall the source tree
-                self._log(Log.DEBUG, "Invalid state in kernel build tree, reloading")
-                self._remove_build_dirs()
-                self._extract_tarball()
-                ret = subprocess.call(cmd, stdin=null, stdout=out, stderr=err)
-                if ret:
-                    # give up
-                    raise rtevalRuntimeError(self, f"kcompile setup failed: {ret}")
-        except KeyboardInterrupt as m:
-            self._log(Log.DEBUG, "keyboard interrupt, aborting")
-            return
-        self._log(Log.DEBUG, "ready to run")
+                self._repair_tarball()
+                ret = self.buildjobs[n].clean(sin=null, sout=out, serr=err)
+            if ret:
+                raise rtevalRuntimeError(self, f"kcompile setup failed: {ret}")
+
         if self._logging:
             os.close(out)
             os.close(err)
-        # clean up object dirs and make sure each has a config file
-        for n in self.nodes:
-            self.buildjobs[n].clean(sin=null, sout=null, serr=null)
+
         os.close(null)
+
+        self._log(Log.DEBUG, "ready to run")
         self._setReady()
 
     def _WorkloadPrepare(self):
-- 
2.43.0

Re: [PATCH 3/5] rteval: kcompile: Skip mrproper, and re-extract if clean fails

[John Kacur]

On Mon, 4 Mar 2024, Crystal Wood wrote:

> We only ever do out-of-tree builds, so the kernel directory should not be
> getting damaged except by incomplete extraction (which mrproper would
> probably not fix) or external meddling.  So, skip the mrproper and
> instead re-extract if a "make clean" fails.
> 
> Also, add -j to cleancmd to further speed things up.  Startup speed may
> not seem all that important given how long rteval is typically run for,
> but this helps make quick tests (e.g. while debugging things, or when
> hunting a latency that shows up very quickly) less painful.
> 
> On my 12-cpu laptop, this patch saves about 15 seconds of startup time.
> 
> Signed-off-by: Crystal Wood <crwood@redhat.com>
> ---
>  rteval/modules/loads/kcompile.py | 40 ++++++++++++++------------------
>  1 file changed, 18 insertions(+), 22 deletions(-)
> 
> diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
> index de1539986723..db1941074157 100644
> --- a/rteval/modules/loads/kcompile.py
> +++ b/rteval/modules/loads/kcompile.py
> @@ -54,7 +54,7 @@ class KBuildJob:
>              self.jobs = self.calc_jobs_per_cpu() * cpus_available
>  
>          self.runcmd = f"make O={self.objdir} -C {self.kdir} -j{self.jobs}"
> -        self.cleancmd = f"make O={self.objdir} -C {self.kdir} clean allmodconfig"
> +        self.cleancmd = f"make O={self.objdir} -C {self.kdir} clean allmodconfig -j${self.jobs}"
>          self.cleancmd += f"&& pushd {self.objdir} && {self.kdir}/scripts/config -d CONFIG_MODULE_SIG_SHA1 -e CONFIG_MODULE_SIG_SHA512 && popd && make O={self.objdir} -C {self.kdir} olddefconfig"
>          if self.binder:
>              self.runcmd = self.binder + " " + self.runcmd
> @@ -90,8 +90,8 @@ class KBuildJob:
>      def clean(self, sin=None, sout=None, serr=None):
>          """ Runs command to clean any previous builds and configure kernel """
>          self.log(Log.DEBUG, f"cleaning objdir {self.objdir}")
> -        subprocess.call(self.cleancmd, shell=True,
> -                        stdin=sin, stdout=sout, stderr=serr)
> +        return subprocess.call(self.cleancmd, shell=True,
> +                               stdin=sin, stdout=sout, stderr=serr)
>  
>      def run(self, sin=None, sout=None, serr=None):
>          """ Use Popen to launch a kcompile job """
> @@ -234,6 +234,10 @@ class Kcompile(CommandLineLoad):
>                  self.logger, self.cpus[n] if self.cpulist else None)
>              self.args.append(str(self.buildjobs[n])+";")
>  
> +    def _repair_tarball(self):
> +        self._log(Log.DEBUG, "Invalid state in kernel build tree, reloading")
> +        self._remove_build_dirs()
> +        self._extract_tarball()
>  
>      def _WorkloadBuild(self):
>          if self._donotrun:
> @@ -246,30 +250,22 @@ class Kcompile(CommandLineLoad):
>          else:
>              out = err = null
>  
> -        # clean up any damage from previous runs
> -        try:
> -            cmd = ["make", "-C", self.mydir, "mrproper"]
> -            ret = subprocess.call(cmd, stdin=null, stdout=out, stderr=err)
> +        # clean up object dirs and make sure each has a config file
> +        for n in self.nodes:
> +            ret = self.buildjobs[n].clean(sin=null, sout=out, serr=err)
>              if ret:
> -                # if the above make failed, remove and reinstall the source tree
> -                self._log(Log.DEBUG, "Invalid state in kernel build tree, reloading")
> -                self._remove_build_dirs()
> -                self._extract_tarball()
> -                ret = subprocess.call(cmd, stdin=null, stdout=out, stderr=err)
> -                if ret:
> -                    # give up
> -                    raise rtevalRuntimeError(self, f"kcompile setup failed: {ret}")
> -        except KeyboardInterrupt as m:
> -            self._log(Log.DEBUG, "keyboard interrupt, aborting")
> -            return
> -        self._log(Log.DEBUG, "ready to run")
> +                self._repair_tarball()
> +                ret = self.buildjobs[n].clean(sin=null, sout=out, serr=err)
> +            if ret:
> +                raise rtevalRuntimeError(self, f"kcompile setup failed: {ret}")
> +
>          if self._logging:
>              os.close(out)
>              os.close(err)
> -        # clean up object dirs and make sure each has a config file
> -        for n in self.nodes:
> -            self.buildjobs[n].clean(sin=null, sout=null, serr=null)
> +
>          os.close(null)
> +
> +        self._log(Log.DEBUG, "ready to run")
>          self._setReady()
>  
>      def _WorkloadPrepare(self):
> -- 
> 2.43.0

You would probably achieve close to the same or better by just using '-j' 
with make mrproper

[jkacur@fionn tmp]$ time tar xJf ../linux-6.6.1.tar.xz 

real	0m7.224s
user	0m7.233s
sys	0m2.204s
[jkacur@fionn tmp]$ cd linux-6.6.1/
[jkacur@fionn linux-6.6.1]$ time make mrproper

real	0m14.451s
user	0m7.998s
sys	0m6.307s
[jkacur@fionn linux-6.6.1]$ time make mrproper -j

real	0m1.336s
user	0m3.473s
sys	0m2.147s

John

Re: [PATCH 3/5] rteval: kcompile: Skip mrproper, and re-extract if clean fails

[Crystal Wood]

On Tue, 2024-04-02 at 15:07 -0400, John Kacur wrote:
> 
> 
> On Mon, 4 Mar 2024, Crystal Wood wrote:
> 
> > We only ever do out-of-tree builds, so the kernel directory should not
> > be
> > getting damaged except by incomplete extraction (which mrproper would
> > probably not fix) or external meddling.  So, skip the mrproper and
> > instead re-extract if a "make clean" fails.
> > 
> > Also, add -j to cleancmd to further speed things up.  Startup speed may
> > not seem all that important given how long rteval is typically run for,
> > but this helps make quick tests (e.g. while debugging things, or when
> > hunting a latency that shows up very quickly) less painful.
> > 
> > On my 12-cpu laptop, this patch saves about 15 seconds of startup time.
> > 
> > Signed-off-by: Crystal Wood <crwood@redhat.com>

> You would probably achieve close to the same or better by just using '-j' 
> with make mrproper

Perhaps it would be almost as fast, depending on the system you're running
it on (and how many non-isolated cores it has).  But it really doesn't seem
necessary.

As for better, how?  We're going from mrproper *and* clean to just clean. 
The re-extraction only happens if clean fails.


-Crystal

Re: [PATCH 3/5] rteval: kcompile: Skip mrproper, and re-extract if clean fails

[John Kacur]

[-- Attachment #1: Type: text/plain, Size: 1846 bytes --]



On Wed, 3 Apr 2024, Crystal Wood wrote:

> On Tue, 2024-04-02 at 15:07 -0400, John Kacur wrote:
> > 
> > 
> > On Mon, 4 Mar 2024, Crystal Wood wrote:
> > 
> > > We only ever do out-of-tree builds, so the kernel directory should not
> > > be
> > > getting damaged except by incomplete extraction (which mrproper would
> > > probably not fix) or external meddling.  So, skip the mrproper and
> > > instead re-extract if a "make clean" fails.
> > > 
> > > Also, add -j to cleancmd to further speed things up.  Startup speed may
> > > not seem all that important given how long rteval is typically run for,
> > > but this helps make quick tests (e.g. while debugging things, or when
> > > hunting a latency that shows up very quickly) less painful.
> > > 
> > > On my 12-cpu laptop, this patch saves about 15 seconds of startup time.
> > > 
> > > Signed-off-by: Crystal Wood <crwood@redhat.com>
> 
> > You would probably achieve close to the same or better by just using '-j' 
> > with make mrproper
> 
> Perhaps it would be almost as fast, depending on the system you're running
> it on (and how many non-isolated cores it has).  But it really doesn't seem
> necessary.
> 
> As for better, how?  We're going from mrproper *and* clean to just clean. 
> The re-extraction only happens if clean fails.
> 
We already reextract if we're in a state that is broken. Perhaps you're 
right and the mrproper isn't necessary, but perhaps I have a long memory 
of when we just did this by default and a lot of problems went away. 
Maybe I'm just stubborn. It seems like an "if it ain't broke don't fix it" 
problem to me, especially since your other patches already shortened the 
time for very short development runs. If you'd like to send a patch to add 
'-j' to this I would except that, otherwise, let's move on to some more 
interesting problems.

John

[PATCH 4/5] rteval: Break out of main loop faster on interrupt/stoptime

[Crystal Wood]

Waiting up to a full minute for rteval to stop on ctrl-c can be
frustrating.  Likewise, if a very short run is requested
(e.g. for testing rteval itself) rounding it up to a minute is
not polite.

Signed-off-by: Crystal Wood <crwood@redhat.com>
---
 rteval/__init__.py | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/rteval/__init__.py b/rteval/__init__.py
index 5d43615af5bb..72e3412a860e 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -32,12 +32,11 @@ RTEVAL_VERSION = version.RTEVAL_VERSION
 
 earlystop = False
 
-stopsig_received = False
+stopsig = threading.Event()
 def sig_handler(signum, frame):
     """ Handle SIGINT (CTRL + C) or SIGTERM (Termination signal) """
     if signum in (signal.SIGINT, signal.SIGTERM):
-        global stopsig_received
-        stopsig_received = True
+        stopsig.set()
         print("*** stop signal received - stopping rteval run ***")
     else:
         raise RuntimeError(f"SIGNAL received! ({signum})")
@@ -208,8 +207,8 @@ class RtEval(rtevalReport):
             currtime = time.time()
             rpttime = currtime + report_interval
             load_avg_checked = 5
-            while (currtime <= stoptime) and not stopsig_received:
-                time.sleep(60.0)
+            while (currtime <= stoptime) and not stopsig.is_set():
+                stopsig.wait(min(stoptime - currtime, 60.0))
                 if not measure_profile.isAlive():
                     stoptime = currtime
                     earlystop = True
@@ -238,7 +237,7 @@ class RtEval(rtevalReport):
             signal.signal(signal.SIGTERM, signal.SIG_DFL)
 
         except RuntimeError as err:
-            if not stopsig_received:
+            if not stopsig.is_set():
                 raise RuntimeError(f"appeared during measurement: {err}")
 
         finally:
-- 
2.43.0

Re: [PATCH 4/5] rteval: Break out of main loop faster on interrupt/stoptime

[John Kacur]

On Mon, 4 Mar 2024, Crystal Wood wrote:

> Waiting up to a full minute for rteval to stop on ctrl-c can be
> frustrating.  Likewise, if a very short run is requested
> (e.g. for testing rteval itself) rounding it up to a minute is
> not polite.
> 
> Signed-off-by: Crystal Wood <crwood@redhat.com>
> ---
>  rteval/__init__.py | 11 +++++------
>  1 file changed, 5 insertions(+), 6 deletions(-)
> 
> diff --git a/rteval/__init__.py b/rteval/__init__.py
> index 5d43615af5bb..72e3412a860e 100644
> --- a/rteval/__init__.py
> +++ b/rteval/__init__.py
> @@ -32,12 +32,11 @@ RTEVAL_VERSION = version.RTEVAL_VERSION
>  
>  earlystop = False
>  
> -stopsig_received = False
> +stopsig = threading.Event()
>  def sig_handler(signum, frame):
>      """ Handle SIGINT (CTRL + C) or SIGTERM (Termination signal) """
>      if signum in (signal.SIGINT, signal.SIGTERM):
> -        global stopsig_received
> -        stopsig_received = True
> +        stopsig.set()
>          print("*** stop signal received - stopping rteval run ***")
>      else:
>          raise RuntimeError(f"SIGNAL received! ({signum})")
> @@ -208,8 +207,8 @@ class RtEval(rtevalReport):
>              currtime = time.time()
>              rpttime = currtime + report_interval
>              load_avg_checked = 5
> -            while (currtime <= stoptime) and not stopsig_received:
> -                time.sleep(60.0)
> +            while (currtime <= stoptime) and not stopsig.is_set():
> +                stopsig.wait(min(stoptime - currtime, 60.0))
>                  if not measure_profile.isAlive():
>                      stoptime = currtime
>                      earlystop = True
> @@ -238,7 +237,7 @@ class RtEval(rtevalReport):
>              signal.signal(signal.SIGTERM, signal.SIG_DFL)
>  
>          except RuntimeError as err:
> -            if not stopsig_received:
> +            if not stopsig.is_set():
>                  raise RuntimeError(f"appeared during measurement: {err}")
>  
>          finally:
> -- 

Signed-off-by: John Kacur <jkacur@redhat.com>

[PATCH 5/5] rteval: Remove 30 second "settling" period

[Crystal Wood]

Waiting for 30 seconds for loads to "settle down" adds unneccesary delay
to very short runs, besides being a giant hack.  Load modules already
have the opportunity to do setup beforehand.

Even if there are a few seconds before the loads get to their expected
"heaviness", that shouldn't meaningfully affect the output except for:
 - very short runs where you either
   - don't care about the latency because you're debugging rteval itself, or
   - are debugging a latency that reliably shows quickly, in which case
     consistent behavior is enough
 - latency spikes that only show up during load startup activity, in
   which case measuring it is a good thing
 - minimum latency values, which are not really the point of rteval.

...and the 30 second delay is *especially* useless if loads are disabled.
Currently there's no official way to do that (as far as I can find), but
it can be done by disabling all but stressng in the conf file, and not
supplying stressng options on the command line.

Signed-off-by: Crystal Wood <crwood@redhat.com>
---
 rteval/__init__.py | 2 --
 1 file changed, 2 deletions(-)

diff --git a/rteval/__init__.py b/rteval/__init__.py
index 72e3412a860e..2c1dc1a7a22c 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -194,8 +194,6 @@ class RtEval(rtevalReport):
                 nthreads = threading.active_count()
             else:
                 nthreads = None
-            self.__logger.log(Log.INFO, "Waiting 30 seconds to let load modules settle down")
-            time.sleep(30)
             measure_profile.Unleash()
             measure_start = datetime.now()
 
-- 
2.43.0

Re: [PATCH 5/5] rteval: Remove 30 second "settling" period

[John Kacur]

On Mon, 4 Mar 2024, Crystal Wood wrote:

> Waiting for 30 seconds for loads to "settle down" adds unneccesary delay
> to very short runs, besides being a giant hack.  Load modules already
> have the opportunity to do setup beforehand.
> 
> Even if there are a few seconds before the loads get to their expected
> "heaviness", that shouldn't meaningfully affect the output except for:
>  - very short runs where you either
>    - don't care about the latency because you're debugging rteval itself, or
>    - are debugging a latency that reliably shows quickly, in which case
>      consistent behavior is enough
>  - latency spikes that only show up during load startup activity, in
>    which case measuring it is a good thing
>  - minimum latency values, which are not really the point of rteval.
> 
> ...and the 30 second delay is *especially* useless if loads are disabled.
> Currently there's no official way to do that (as far as I can find), but
> it can be done by disabling all but stressng in the conf file, and not
> supplying stressng options on the command line.
> 
> Signed-off-by: Crystal Wood <crwood@redhat.com>
> ---
>  rteval/__init__.py | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/rteval/__init__.py b/rteval/__init__.py
> index 72e3412a860e..2c1dc1a7a22c 100644
> --- a/rteval/__init__.py
> +++ b/rteval/__init__.py
> @@ -194,8 +194,6 @@ class RtEval(rtevalReport):
>                  nthreads = threading.active_count()
>              else:
>                  nthreads = None
> -            self.__logger.log(Log.INFO, "Waiting 30 seconds to let load modules settle down")
> -            time.sleep(30)
>              measure_profile.Unleash()
>              measure_start = datetime.now()
>  
> -- 

kludge begone!
Signed-off-by: John Kacur <jkacur@redhat.com>