* [PATCH] rteval: kcompile: Tweak default source tarball selection
@ 2022-04-19 16:21 Valentin Schneider
2022-05-02 15:17 ` John Kacur
0 siblings, 1 reply; 3+ messages in thread
From: Valentin Schneider @ 2022-04-19 16:21 UTC (permalink / raw)
To: linux-rt-users; +Cc: John Kacur, Clark Williams
Kcompile._WorkloadSetup() looks for a "tarball" and "tarfile" entry in
the CfgSection, but I couldn't find a single setter for thoses. The
only way for a user to specify a file is via --kcompile-source, which
doesn't seem to be actually used by the module.
As it stands, this method will look for a tarball matching the
hardcoded kernel_prefix.
Make Kcompile
- actually use --kcompile-source
- use the latest tarball in the loadsource if no source is specified.
Signed-off-by: Valentin Schneider <vschneid@redhat.com>
---
I'm not entirely sure about that one, I tried to dig up the git
history for uses of those "tarball" and "tarfile" options but didn't
really get anywhere, so maybe I'm completely wrong...
---
rteval/modules/loads/kcompile.py | 22 ++++++++++++++++------
1 file changed, 16 insertions(+), 6 deletions(-)
diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
index 367f8dc..2747913 100644
--- a/rteval/modules/loads/kcompile.py
+++ b/rteval/modules/loads/kcompile.py
@@ -28,6 +28,7 @@ import sys
import os
import os.path
import glob
+import re
import subprocess
from rteval.modules import rtevalRuntimeError
from rteval.modules.loads import CommandLineLoad
@@ -35,7 +36,7 @@ from rteval.Log import Log
from rteval.misc import expand_cpulist, compress_cpulist
from rteval.systopology import SysTopology
-kernel_prefix = "linux-5.13"
+RE_KERNEL_TAR = re.compile(r"linux-(?P<maj>\d+)\.(?P<min>\d+).*\.tar.*")
class KBuildJob:
'''Class to manage a build job bound to a particular node'''
@@ -163,18 +164,27 @@ class Kcompile(CommandLineLoad):
return
# find our source tarball
- if 'tarball' in self._cfg:
- tarfile = os.path.join(self.srcdir, self._cfg.tarfile)
+ if self._cfg.source:
+ tarfile = os.path.join(self.srcdir, self._cfg.source)
if not os.path.exists(tarfile):
raise rtevalRuntimeError(self, " tarfile %s does not exist!" % tarfile)
self.source = tarfile
else:
- tarfiles = glob.glob(os.path.join(self.srcdir, "%s*" % kernel_prefix))
+ tarfiles = [f for f in os.listdir(self.srcdir)
+ if RE_KERNEL_TAR.match(f)]
if tarfiles:
- self.source = tarfiles[0]
+ # Use (one of) the most recent kernels
+ def verkey(f):
+ match = RE_KERNEL_TAR.match(f)
+ return (match.group("maj"), match.group("min"))
+
+ tarfiles.sort(key=verkey)
+ self.source = tarfiles[-1]
else:
raise rtevalRuntimeError(self, " no kernel tarballs found in %s" % self.srcdir)
+ kernel_prefix = re.search(r"linux-\d\.\d", self.source).group(0)
+
# check for existing directory
kdir = None
names = os.listdir(self.builddir)
@@ -322,7 +332,7 @@ class Kcompile(CommandLineLoad):
def ModuleParameters():
return {"source": {"descr": "Source tar ball",
- "default": "linux-5.13.2.tar.xz",
+ "default": "",
"metavar": "TARBALL"},
"jobspercore": {"descr": "Number of working threads per core",
"default": 2,
--
2.27.0
^ permalink raw reply related [flat|nested] 3+ messages in thread
* Re: [PATCH] rteval: kcompile: Tweak default source tarball selection
2022-04-19 16:21 [PATCH] rteval: kcompile: Tweak default source tarball selection Valentin Schneider
@ 2022-05-02 15:17 ` John Kacur
2022-05-03 10:27 ` Valentin Schneider
0 siblings, 1 reply; 3+ messages in thread
From: John Kacur @ 2022-05-02 15:17 UTC (permalink / raw)
To: Valentin Schneider; +Cc: linux-rt-users, Clark Williams
On Tue, 19 Apr 2022, Valentin Schneider wrote:
> Kcompile._WorkloadSetup() looks for a "tarball" and "tarfile" entry in
> the CfgSection, but I couldn't find a single setter for thoses. The
> only way for a user to specify a file is via --kcompile-source, which
> doesn't seem to be actually used by the module.
>
> As it stands, this method will look for a tarball matching the
> hardcoded kernel_prefix.
>
> Make Kcompile
> - actually use --kcompile-source
> - use the latest tarball in the loadsource if no source is specified.
>
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> ---
> I'm not entirely sure about that one, I tried to dig up the git
> history for uses of those "tarball" and "tarfile" options but didn't
> really get anywhere, so maybe I'm completely wrong...
> ---
> rteval/modules/loads/kcompile.py | 22 ++++++++++++++++------
> 1 file changed, 16 insertions(+), 6 deletions(-)
>
> diff --git a/rteval/modules/loads/kcompile.py b/rteval/modules/loads/kcompile.py
> index 367f8dc..2747913 100644
> --- a/rteval/modules/loads/kcompile.py
> +++ b/rteval/modules/loads/kcompile.py
> @@ -28,6 +28,7 @@ import sys
> import os
> import os.path
> import glob
> +import re
> import subprocess
> from rteval.modules import rtevalRuntimeError
> from rteval.modules.loads import CommandLineLoad
> @@ -35,7 +36,7 @@ from rteval.Log import Log
> from rteval.misc import expand_cpulist, compress_cpulist
> from rteval.systopology import SysTopology
>
> -kernel_prefix = "linux-5.13"
> +RE_KERNEL_TAR = re.compile(r"linux-(?P<maj>\d+)\.(?P<min>\d+).*\.tar.*")
>
> class KBuildJob:
> '''Class to manage a build job bound to a particular node'''
> @@ -163,18 +164,27 @@ class Kcompile(CommandLineLoad):
> return
>
> # find our source tarball
> - if 'tarball' in self._cfg:
> - tarfile = os.path.join(self.srcdir, self._cfg.tarfile)
> + if self._cfg.source:
> + tarfile = os.path.join(self.srcdir, self._cfg.source)
> if not os.path.exists(tarfile):
> raise rtevalRuntimeError(self, " tarfile %s does not exist!" % tarfile)
> self.source = tarfile
> else:
> - tarfiles = glob.glob(os.path.join(self.srcdir, "%s*" % kernel_prefix))
> + tarfiles = [f for f in os.listdir(self.srcdir)
> + if RE_KERNEL_TAR.match(f)]
> if tarfiles:
> - self.source = tarfiles[0]
> + # Use (one of) the most recent kernels
> + def verkey(f):
> + match = RE_KERNEL_TAR.match(f)
> + return (match.group("maj"), match.group("min"))
> +
> + tarfiles.sort(key=verkey)
> + self.source = tarfiles[-1]
> else:
> raise rtevalRuntimeError(self, " no kernel tarballs found in %s" % self.srcdir)
>
> + kernel_prefix = re.search(r"linux-\d\.\d", self.source).group(0)
> +
> # check for existing directory
> kdir = None
> names = os.listdir(self.builddir)
> @@ -322,7 +332,7 @@ class Kcompile(CommandLineLoad):
>
> def ModuleParameters():
> return {"source": {"descr": "Source tar ball",
> - "default": "linux-5.13.2.tar.xz",
> + "default": "",
> "metavar": "TARBALL"},
> "jobspercore": {"descr": "Number of working threads per core",
> "default": 2,
> --
> 2.27.0
I think we wanted to make sure we were doing an apples to apples
comparison, so we restricted the default kernel to the the one we supply
in rteval-loads. I am thinking of changing this in the future to using the
one supplied by the distribution, but still mulling this over.
However, --kcompile-source should work as advertised. Do you think you
could resend a patch with just the part to make --kcompile-source work
correctly?
Thanks
John
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: [PATCH] rteval: kcompile: Tweak default source tarball selection
2022-05-02 15:17 ` John Kacur
@ 2022-05-03 10:27 ` Valentin Schneider
0 siblings, 0 replies; 3+ messages in thread
From: Valentin Schneider @ 2022-05-03 10:27 UTC (permalink / raw)
To: John Kacur; +Cc: linux-rt-users, Clark Williams
On 02/05/22 11:17, John Kacur wrote:
> On Tue, 19 Apr 2022, Valentin Schneider wrote:
>
>> Kcompile._WorkloadSetup() looks for a "tarball" and "tarfile" entry in
>> the CfgSection, but I couldn't find a single setter for thoses. The
>> only way for a user to specify a file is via --kcompile-source, which
>> doesn't seem to be actually used by the module.
>>
>> As it stands, this method will look for a tarball matching the
>> hardcoded kernel_prefix.
>>
>> Make Kcompile
>> - actually use --kcompile-source
>> - use the latest tarball in the loadsource if no source is specified.
>>
>> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
>> ---
>> I'm not entirely sure about that one, I tried to dig up the git
>> history for uses of those "tarball" and "tarfile" options but didn't
>> really get anywhere, so maybe I'm completely wrong...
>> ---
>> rteval/modules/loads/kcompile.py | 22 ++++++++++++++++------
>> 1 file changed, 16 insertions(+), 6 deletions(-)
>>
> I think we wanted to make sure we were doing an apples to apples
> comparison, so we restricted the default kernel to the the one we supply
> in rteval-loads. I am thinking of changing this in the future to using the
> one supplied by the distribution, but still mulling this over.
>
> However, --kcompile-source should work as advertised. Do you think you
> could resend a patch with just the part to make --kcompile-source work
> correctly?
>
Sure thing, will do.
> Thanks
>
> John
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2022-05-03 10:27 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-04-19 16:21 [PATCH] rteval: kcompile: Tweak default source tarball selection Valentin Schneider
2022-05-02 15:17 ` John Kacur
2022-05-03 10:27 ` Valentin Schneider
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.