[PATCH 0/2] rteval: Create exclusive load modules

[PATCH 0/2] rteval: Create exclusive load modules

[John Kacur]

The ability to run stress-ng as a load to rteval was already added.

The original intention was to run stress-ng as the only load for
diagnostic purposes.

The following patches create the notion of a module with an exclusive
field meaning it should run exclusively, and makes the stress-ng module
one of those.

The first patch does some clean-ups and makes sure that the current
donotrun field works as advertised.

The second patch creates the ability to run a load exclusively and sets
stress-ng as an exclusive module by default

John Kacur (2):
  rteval: Make donotrun work correctly in load modules
  rteval: Add idea of exclusive load module and make stress-ng one

 rteval/__init__.py                |  4 ++--
 rteval/modules/__init__.py        | 21 +++++++++++++++++++++
 rteval/modules/loads/__init__.py  |  2 +-
 rteval/modules/loads/hackbench.py |  3 +++
 rteval/modules/loads/kcompile.py  |  9 +++++++++
 rteval/modules/loads/stressng.py  |  2 ++
 6 files changed, 38 insertions(+), 3 deletions(-)

-- 
2.31.1

[PATCH 1/2] rteval: Make donotrun work correctly in load modules

[John Kacur]

hackbench and kcompile don't correctly adhere to the donotrun variable
which is inherited class rtevalModulePrototype. If the variable is set
after the modules are loaded, although the modules do not run, some
threading accounting still occurs which causes problems on shutdown of
rteval.

Fix in the methods in hackbench and kcompile where this is relevant, by
checking the variable before the method runs.

While we are at it fix a few other things
- change len(threading.enumerate()) to threading.active_count()
- Fix a spelling typo

This patch is in preparation for a patch to allow stress-ng to run as a
load module without running other modules.

Signed-off-by: John Kacur <jkacur@redhat.com>
---
 rteval/__init__.py                | 4 ++--
 rteval/modules/loads/__init__.py  | 2 +-
 rteval/modules/loads/hackbench.py | 3 +++
 rteval/modules/loads/kcompile.py  | 9 +++++++++
 4 files changed, 15 insertions(+), 3 deletions(-)

diff --git a/rteval/__init__.py b/rteval/__init__.py
index 5faed23bc927..22af8e85d5aa 100644
--- a/rteval/__init__.py
+++ b/rteval/__init__.py
@@ -208,7 +208,7 @@ class RtEval(rtevalReport):
             report_interval = int(self.__rtevcfg.report_interval)
             if with_loads:
                 self._loadmods.Unleash()
-                nthreads = len(threading.enumerate())
+                nthreads = threading.active_count()
             else:
                 nthreads = None
             self.__logger.log(Log.INFO, "Waiting 30 seconds to let load modules settle down")
@@ -233,7 +233,7 @@ class RtEval(rtevalReport):
                                       "Measurement threads did not use the full time slot. Doing a controlled stop.")
 
                 if with_loads:
-                    if len(threading.enumerate()) < nthreads:
+                    if threading.active_count() < nthreads:
                         raise RuntimeError("load thread died!")
 
                 if not load_avg_checked:
diff --git a/rteval/modules/loads/__init__.py b/rteval/modules/loads/__init__.py
index 9a942f454536..2c2105efa964 100644
--- a/rteval/modules/loads/__init__.py
+++ b/rteval/modules/loads/__init__.py
@@ -106,7 +106,7 @@ class LoadModules(RtEvalModules):
         "Loads and imports all the configured modules"
 
         for m in modcfg:
-            # hope to eventually have different kinds but module is only on
+            # hope to eventually have different kinds but module is only one
             # for now (jcw)
             if m[1].lower() == 'module':
                 self._LoadModule(m[0])
diff --git a/rteval/modules/loads/hackbench.py b/rteval/modules/loads/hackbench.py
index 2fb90c1946a5..21a803b8a3cd 100644
--- a/rteval/modules/loads/hackbench.py
+++ b/rteval/modules/loads/hackbench.py
@@ -42,6 +42,9 @@ class Hackbench(CommandLineLoad):
         CommandLineLoad.__init__(self, "hackbench", config, logger)
 
     def _WorkloadSetup(self):
+        if self._donotrun:
+            return
+
         'calculate arguments based on input parameters'
         (mem, units) = self.memsize
         if units == 'KB':
diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index 8d08a3d44302..ec85b75f38b5 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -158,6 +158,9 @@ class Kcompile(CommandLineLoad):
                 "error removing builddir (%s) (ret=%d)" % (self.builddir, ret))
 
     def _WorkloadSetup(self):
+        if self._donotrun:
+            return
+
         # find our source tarball
         if 'tarball' in self._cfg:
             tarfile = os.path.join(self.srcdir, self._cfg.tarfile)
@@ -219,6 +222,9 @@ class Kcompile(CommandLineLoad):
 
 
     def _WorkloadBuild(self):
+        if self._donotrun:
+            return
+
         null = os.open("/dev/null", os.O_RDWR)
         if self._logging:
             out = self.open_logfile("kcompile-build.stdout")
@@ -293,6 +299,9 @@ class Kcompile(CommandLineLoad):
 
 
     def _WorkloadCleanup(self):
+        if self._donotrun:
+            return
+
         self._log(Log.DEBUG, "out of stopevent loop")
         for n in self.buildjobs:
             if self.buildjobs[n].jobid.poll() is None:
-- 
2.31.1

[PATCH 2/2] rteval: Add idea of exclusive load module and make stress-ng one

[John Kacur]

When running stress-ng as a load module in rteval, we don't want to run
kcompile or hackbench, so create the notion of an exclusive load module
and make stress-ng one.

Signed-off-by: John Kacur <jkacur@redhat.com>
---
 rteval/modules/__init__.py       | 21 +++++++++++++++++++++
 rteval/modules/loads/stressng.py |  2 ++
 2 files changed, 23 insertions(+)

diff --git a/rteval/modules/__init__.py b/rteval/modules/__init__.py
index 6ff82d10bf8c..d52dd597186a 100644
--- a/rteval/modules/__init__.py
+++ b/rteval/modules/__init__.py
@@ -58,6 +58,7 @@ class rtevalModulePrototype(threading.Thread):
                          "stop": threading.Event(),
                          "finished": threading.Event()}
         self._donotrun = False
+        self._exclusive = False
         self.__timestamps = {}
         self.__sleeptime = 2.0
 
@@ -75,6 +76,16 @@ class rtevalModulePrototype(threading.Thread):
         return self.__ready
 
 
+    def is_exclusive(self):
+        """ Returns true if this workload should run alone """
+        return self._exclusive
+
+
+    def set_donotrun(self):
+        """ set a module's donotrun field to True """
+        self._donotrun = True
+
+
     def _setReady(self, state=True):
         """ Sets the ready flag for the module """
         self.__ready = state
@@ -459,7 +470,17 @@ class RtEvalModules:
             raise rtevalRuntimeError("No %s modules configured" % self._module_type)
 
         self._logger.log(Log.INFO, "Preparing %s modules" % self._module_type)
+        exclusive = 0
+        for (modname, mod) in self.__modules:
+            if mod.is_exclusive() and mod.WorkloadWillRun():
+                exclusive += 1
         for (modname, mod) in self.__modules:
+            if exclusive >= 1:
+                if exclusive != 1:
+                    msg = f"More than one exclusive load: {exclusive}"
+                    raise RuntimeError(msg)
+                if not mod.is_exclusive() and mod.WorkloadWillRun():
+                    mod.set_donotrun()
             mod.start()
             if mod.WorkloadWillRun():
                 self._logger.log(Log.DEBUG, "\t - Started %s preparations" % modname)
diff --git a/rteval/modules/loads/stressng.py b/rteval/modules/loads/stressng.py
index 926de38e3116..d084814142fb 100644
--- a/rteval/modules/loads/stressng.py
+++ b/rteval/modules/loads/stressng.py
@@ -27,6 +27,8 @@ class Stressng(CommandLineLoad):
             self._donotrun = False
         else:
             self._donotrun = True
+        """ When this module runs, other load modules should not """
+        self._exclusive = True
 
     def _WorkloadSetup(self):
         " Since there is nothing to build, we don't need to do anything here "
-- 
2.31.1