* [PATCH 0/3] kunit: add support in kunit.py for --qemu_args @ 2022-05-18 17:01 Daniel Latypov 2022-05-18 17:01 ` [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args Daniel Latypov ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: Daniel Latypov @ 2022-05-18 17:01 UTC (permalink / raw) To: brendanhiggins, davidgow Cc: elver, linux-kernel, kasan-dev, kunit-dev, linux-kselftest, skhan, Daniel Latypov Note: this series applies on top of https://lore.kernel.org/linux-kselftest/20220516194730.1546328-2-dlatypov@google.com/. That patch greatly simplified the process of adding new flags. This flag would let users pass additional arguments to QEMU when using a non-UML arch to run their tests. E.g. for kcsan's tests, they require SMP and with this patch, you can do $ ./tools/testing/kunit/kunit.py run --kconfig_add=CONFIG_SMP --qemu_args='-smp 8' This is proposed as an alternative to users manually creating new qemu_config python files and also to [1], where we discussed checking in a new x86_64 variant w/ `-smp 8` hard-coded into it. This patch also contains a fix to the example `run_kunit` bash function since it didn't quote properly and would parse the example above as --qemu_args='-smp' '8' no matter how you tried to quote your arguments. [1] https://lore.kernel.org/linux-kselftest/20220518073232.526443-1-davidgow@google.com/ Daniel Latypov (3): Documentation: kunit: fix example run_kunit func to allow spaces in args kunit: tool: simplify creating LinuxSourceTreeOperations kunit: tool: introduce --qemu_args .../dev-tools/kunit/running_tips.rst | 2 +- tools/testing/kunit/kunit.py | 14 +++++++++- tools/testing/kunit/kunit_kernel.py | 26 +++++++++++-------- tools/testing/kunit/kunit_tool_test.py | 20 +++++++++++--- 4 files changed, 46 insertions(+), 16 deletions(-) -- 2.36.1.124.g0e6072fb45-goog ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args 2022-05-18 17:01 [PATCH 0/3] kunit: add support in kunit.py for --qemu_args Daniel Latypov @ 2022-05-18 17:01 ` Daniel Latypov 2022-05-19 13:20 ` David Gow 2022-07-06 18:34 ` Brendan Higgins 2022-05-18 17:01 ` [PATCH 2/3] kunit: tool: simplify creating LinuxSourceTreeOperations Daniel Latypov 2022-05-18 17:01 ` [PATCH 3/3] kunit: tool: introduce --qemu_args Daniel Latypov 2 siblings, 2 replies; 10+ messages in thread From: Daniel Latypov @ 2022-05-18 17:01 UTC (permalink / raw) To: brendanhiggins, davidgow Cc: elver, linux-kernel, kasan-dev, kunit-dev, linux-kselftest, skhan, Daniel Latypov Without the quoting, the example will mess up invocations like $ run_kunit "Something with spaces" Note: this example isn't valid, but if ever a usecase arises where a flag argument might have spaces in it, it'll break. Signed-off-by: Daniel Latypov <dlatypov@google.com> --- Documentation/dev-tools/kunit/running_tips.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst index c36f6760087d..da8677c32aee 100644 --- a/Documentation/dev-tools/kunit/running_tips.rst +++ b/Documentation/dev-tools/kunit/running_tips.rst @@ -15,7 +15,7 @@ It can be handy to create a bash function like: .. code-block:: bash function run_kunit() { - ( cd "$(git rev-parse --show-toplevel)" && ./tools/testing/kunit/kunit.py run $@ ) + ( cd "$(git rev-parse --show-toplevel)" && ./tools/testing/kunit/kunit.py run "$@" ) } .. note:: -- 2.36.1.124.g0e6072fb45-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args 2022-05-18 17:01 ` [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args Daniel Latypov @ 2022-05-19 13:20 ` David Gow 2022-07-06 18:34 ` Brendan Higgins 1 sibling, 0 replies; 10+ messages in thread From: David Gow @ 2022-05-19 13:20 UTC (permalink / raw) To: Daniel Latypov Cc: Brendan Higgins, Marco Elver, Linux Kernel Mailing List, kasan-dev, KUnit Development, open list:KERNEL SELFTEST FRAMEWORK, Shuah Khan [-- Attachment #1: Type: text/plain, Size: 1236 bytes --] On Thu, May 19, 2022 at 1:01 AM Daniel Latypov <dlatypov@google.com> wrote: > > Without the quoting, the example will mess up invocations like > $ run_kunit "Something with spaces" > > Note: this example isn't valid, but if ever a usecase arises where a > flag argument might have spaces in it, it'll break. > > Signed-off-by: Daniel Latypov <dlatypov@google.com> > --- Looks correct to me, though I'm not a bash _expert_. Reviewed-by: David Gow <davidgow@google.com> -- David > Documentation/dev-tools/kunit/running_tips.rst | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/Documentation/dev-tools/kunit/running_tips.rst b/Documentation/dev-tools/kunit/running_tips.rst > index c36f6760087d..da8677c32aee 100644 > --- a/Documentation/dev-tools/kunit/running_tips.rst > +++ b/Documentation/dev-tools/kunit/running_tips.rst > @@ -15,7 +15,7 @@ It can be handy to create a bash function like: > .. code-block:: bash > > function run_kunit() { > - ( cd "$(git rev-parse --show-toplevel)" && ./tools/testing/kunit/kunit.py run $@ ) > + ( cd "$(git rev-parse --show-toplevel)" && ./tools/testing/kunit/kunit.py run "$@" ) > } > > .. note:: > -- > 2.36.1.124.g0e6072fb45-goog > [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4003 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args 2022-05-18 17:01 ` [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args Daniel Latypov 2022-05-19 13:20 ` David Gow @ 2022-07-06 18:34 ` Brendan Higgins 1 sibling, 0 replies; 10+ messages in thread From: Brendan Higgins @ 2022-07-06 18:34 UTC (permalink / raw) To: Daniel Latypov Cc: davidgow, elver, linux-kernel, kasan-dev, kunit-dev, linux-kselftest, skhan On Wed, May 18, 2022 at 1:01 PM Daniel Latypov <dlatypov@google.com> wrote: > > Without the quoting, the example will mess up invocations like > $ run_kunit "Something with spaces" > > Note: this example isn't valid, but if ever a usecase arises where a > flag argument might have spaces in it, it'll break. > > Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 2/3] kunit: tool: simplify creating LinuxSourceTreeOperations 2022-05-18 17:01 [PATCH 0/3] kunit: add support in kunit.py for --qemu_args Daniel Latypov 2022-05-18 17:01 ` [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args Daniel Latypov @ 2022-05-18 17:01 ` Daniel Latypov 2022-05-19 13:20 ` David Gow 2022-07-06 20:06 ` Brendan Higgins 2022-05-18 17:01 ` [PATCH 3/3] kunit: tool: introduce --qemu_args Daniel Latypov 2 siblings, 2 replies; 10+ messages in thread From: Daniel Latypov @ 2022-05-18 17:01 UTC (permalink / raw) To: brendanhiggins, davidgow Cc: elver, linux-kernel, kasan-dev, kunit-dev, linux-kselftest, skhan, Daniel Latypov Drop get_source_tree_ops() and just call what used to be get_source_tree_ops_from_qemu_config() in both cases. Also rename the functions to have shorter names and add a "_" prefix to note they're not meant to be used outside this function. Signed-off-by: Daniel Latypov <dlatypov@google.com> --- tools/testing/kunit/kunit_kernel.py | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index 8bc8305ba817..e93f07ac0af1 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -178,19 +178,16 @@ def get_old_kunitconfig_path(build_dir: str) -> str: def get_outfile_path(build_dir: str) -> str: return os.path.join(build_dir, OUTFILE_PATH) -def get_source_tree_ops(arch: str, cross_compile: Optional[str]) -> LinuxSourceTreeOperations: +def _default_qemu_config_path(arch: str) -> str: config_path = os.path.join(QEMU_CONFIGS_DIR, arch + '.py') - if arch == 'um': - return LinuxSourceTreeOperationsUml(cross_compile=cross_compile) if os.path.isfile(config_path): - return get_source_tree_ops_from_qemu_config(config_path, cross_compile)[1] + return config_path options = [f[:-3] for f in os.listdir(QEMU_CONFIGS_DIR) if f.endswith('.py')] raise ConfigError(arch + ' is not a valid arch, options are ' + str(sorted(options))) -def get_source_tree_ops_from_qemu_config(config_path: str, - cross_compile: Optional[str]) -> Tuple[ - str, LinuxSourceTreeOperations]: +def _get_qemu_ops(config_path: str, + cross_compile: Optional[str]) -> Tuple[str, LinuxSourceTreeOperations]: # The module name/path has very little to do with where the actual file # exists (I learned this through experimentation and could not find it # anywhere in the Python documentation). @@ -226,11 +223,14 @@ class LinuxSourceTree: qemu_config_path=None) -> None: signal.signal(signal.SIGINT, self.signal_handler) if qemu_config_path: - self._arch, self._ops = get_source_tree_ops_from_qemu_config( - qemu_config_path, cross_compile) + self._arch, self._ops = _get_qemu_ops(qemu_config_path, cross_compile) else: self._arch = 'um' if arch is None else arch - self._ops = get_source_tree_ops(self._arch, cross_compile) + if self._arch == 'um': + self._ops = LinuxSourceTreeOperationsUml(cross_compile=cross_compile) + else: + qemu_config_path = _default_qemu_config_path(self._arch) + _, self._ops = _get_qemu_ops(qemu_config_path, cross_compile) if kunitconfig_path: if os.path.isdir(kunitconfig_path): -- 2.36.1.124.g0e6072fb45-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 2/3] kunit: tool: simplify creating LinuxSourceTreeOperations 2022-05-18 17:01 ` [PATCH 2/3] kunit: tool: simplify creating LinuxSourceTreeOperations Daniel Latypov @ 2022-05-19 13:20 ` David Gow 2022-07-06 20:06 ` Brendan Higgins 1 sibling, 0 replies; 10+ messages in thread From: David Gow @ 2022-05-19 13:20 UTC (permalink / raw) To: Daniel Latypov Cc: Brendan Higgins, Marco Elver, Linux Kernel Mailing List, kasan-dev, KUnit Development, open list:KERNEL SELFTEST FRAMEWORK, Shuah Khan [-- Attachment #1: Type: text/plain, Size: 3393 bytes --] On Thu, May 19, 2022 at 1:01 AM Daniel Latypov <dlatypov@google.com> wrote: > > Drop get_source_tree_ops() and just call what used to be > get_source_tree_ops_from_qemu_config() in both cases. > > Also rename the functions to have shorter names and add a "_" prefix to > note they're not meant to be used outside this function. > > Signed-off-by: Daniel Latypov <dlatypov@google.com> > --- Reviewed-by: David Gow <davidgow@google.com> -- David > tools/testing/kunit/kunit_kernel.py | 20 ++++++++++---------- > 1 file changed, 10 insertions(+), 10 deletions(-) > > diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py > index 8bc8305ba817..e93f07ac0af1 100644 > --- a/tools/testing/kunit/kunit_kernel.py > +++ b/tools/testing/kunit/kunit_kernel.py > @@ -178,19 +178,16 @@ def get_old_kunitconfig_path(build_dir: str) -> str: > def get_outfile_path(build_dir: str) -> str: > return os.path.join(build_dir, OUTFILE_PATH) > > -def get_source_tree_ops(arch: str, cross_compile: Optional[str]) -> LinuxSourceTreeOperations: > +def _default_qemu_config_path(arch: str) -> str: > config_path = os.path.join(QEMU_CONFIGS_DIR, arch + '.py') > - if arch == 'um': > - return LinuxSourceTreeOperationsUml(cross_compile=cross_compile) > if os.path.isfile(config_path): > - return get_source_tree_ops_from_qemu_config(config_path, cross_compile)[1] > + return config_path > > options = [f[:-3] for f in os.listdir(QEMU_CONFIGS_DIR) if f.endswith('.py')] > raise ConfigError(arch + ' is not a valid arch, options are ' + str(sorted(options))) > > -def get_source_tree_ops_from_qemu_config(config_path: str, > - cross_compile: Optional[str]) -> Tuple[ > - str, LinuxSourceTreeOperations]: > +def _get_qemu_ops(config_path: str, > + cross_compile: Optional[str]) -> Tuple[str, LinuxSourceTreeOperations]: > # The module name/path has very little to do with where the actual file > # exists (I learned this through experimentation and could not find it > # anywhere in the Python documentation). > @@ -226,11 +223,14 @@ class LinuxSourceTree: > qemu_config_path=None) -> None: > signal.signal(signal.SIGINT, self.signal_handler) > if qemu_config_path: > - self._arch, self._ops = get_source_tree_ops_from_qemu_config( > - qemu_config_path, cross_compile) > + self._arch, self._ops = _get_qemu_ops(qemu_config_path, cross_compile) > else: > self._arch = 'um' if arch is None else arch > - self._ops = get_source_tree_ops(self._arch, cross_compile) > + if self._arch == 'um': > + self._ops = LinuxSourceTreeOperationsUml(cross_compile=cross_compile) > + else: > + qemu_config_path = _default_qemu_config_path(self._arch) > + _, self._ops = _get_qemu_ops(qemu_config_path, cross_compile) > > if kunitconfig_path: > if os.path.isdir(kunitconfig_path): > -- > 2.36.1.124.g0e6072fb45-goog > [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4003 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 2/3] kunit: tool: simplify creating LinuxSourceTreeOperations 2022-05-18 17:01 ` [PATCH 2/3] kunit: tool: simplify creating LinuxSourceTreeOperations Daniel Latypov 2022-05-19 13:20 ` David Gow @ 2022-07-06 20:06 ` Brendan Higgins 1 sibling, 0 replies; 10+ messages in thread From: Brendan Higgins @ 2022-07-06 20:06 UTC (permalink / raw) To: Daniel Latypov Cc: davidgow, elver, linux-kernel, kasan-dev, kunit-dev, linux-kselftest, skhan On Wed, May 18, 2022 at 1:01 PM Daniel Latypov <dlatypov@google.com> wrote: > > Drop get_source_tree_ops() and just call what used to be > get_source_tree_ops_from_qemu_config() in both cases. > > Also rename the functions to have shorter names and add a "_" prefix to > note they're not meant to be used outside this function. > > Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> ^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 3/3] kunit: tool: introduce --qemu_args 2022-05-18 17:01 [PATCH 0/3] kunit: add support in kunit.py for --qemu_args Daniel Latypov 2022-05-18 17:01 ` [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args Daniel Latypov 2022-05-18 17:01 ` [PATCH 2/3] kunit: tool: simplify creating LinuxSourceTreeOperations Daniel Latypov @ 2022-05-18 17:01 ` Daniel Latypov 2022-05-19 13:20 ` David Gow 2022-07-06 20:11 ` Brendan Higgins 2 siblings, 2 replies; 10+ messages in thread From: Daniel Latypov @ 2022-05-18 17:01 UTC (permalink / raw) To: brendanhiggins, davidgow Cc: elver, linux-kernel, kasan-dev, kunit-dev, linux-kselftest, skhan, Daniel Latypov Example usage: $ ./tools/testing/kunit/kunit.py run --arch=x86_64 \ --kconfig_add=CONFIG_SMP=y --qemu_args='-smp 8' Looking in the test.log, one can see > smp: Bringing up secondary CPUs ... > .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 > smp: Brought up 1 node, 8 CPUs This flag would allow people to make tweaks like this without having to create custom qemu_config files. For consistency with --kernel_args, we allow users to repeat this argument, e.g. you can tack on a --qemu_args='-m 2048', or you could just append it to the first string ('-smp 8 -m 2048'). Signed-off-by: Daniel Latypov <dlatypov@google.com> --- tools/testing/kunit/kunit.py | 14 +++++++++++++- tools/testing/kunit/kunit_kernel.py | 10 +++++++--- tools/testing/kunit/kunit_tool_test.py | 20 +++++++++++++++++--- 3 files changed, 37 insertions(+), 7 deletions(-) diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py index 8a90d80ee66e..e01c7964f744 100755 --- a/tools/testing/kunit/kunit.py +++ b/tools/testing/kunit/kunit.py @@ -10,6 +10,7 @@ import argparse import os import re +import shlex import sys import time @@ -323,6 +324,10 @@ def add_common_opts(parser) -> None: 'a QemuArchParams object.'), type=str, metavar='FILE') + parser.add_argument('--qemu_args', + help='Additional QEMU arguments, e.g. "-smp 8"', + action='append', metavar='') + def add_build_opts(parser) -> None: parser.add_argument('--jobs', help='As in the make command, "Specifies the number of ' @@ -368,12 +373,19 @@ def add_parse_opts(parser) -> None: def tree_from_args(cli_args: argparse.Namespace) -> kunit_kernel.LinuxSourceTree: """Returns a LinuxSourceTree based on the user's arguments.""" + # Allow users to specify multiple arguments in one string, e.g. '-smp 8' + qemu_args: List[str] = [] + if cli_args.qemu_args: + for arg in cli_args.qemu_args: + qemu_args.extend(shlex.split(arg)) + return kunit_kernel.LinuxSourceTree(cli_args.build_dir, kunitconfig_path=cli_args.kunitconfig, kconfig_add=cli_args.kconfig_add, arch=cli_args.arch, cross_compile=cli_args.cross_compile, - qemu_config_path=cli_args.qemu_config) + qemu_config_path=cli_args.qemu_config, + extra_qemu_args=qemu_args) def main(argv): diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py index e93f07ac0af1..a791073d25f9 100644 --- a/tools/testing/kunit/kunit_kernel.py +++ b/tools/testing/kunit/kunit_kernel.py @@ -187,6 +187,7 @@ def _default_qemu_config_path(arch: str) -> str: raise ConfigError(arch + ' is not a valid arch, options are ' + str(sorted(options))) def _get_qemu_ops(config_path: str, + extra_qemu_args: Optional[List[str]], cross_compile: Optional[str]) -> Tuple[str, LinuxSourceTreeOperations]: # The module name/path has very little to do with where the actual file # exists (I learned this through experimentation and could not find it @@ -207,6 +208,8 @@ def _get_qemu_ops(config_path: str, if not hasattr(config, 'QEMU_ARCH'): raise ValueError('qemu_config module missing "QEMU_ARCH": ' + config_path) params: qemu_config.QemuArchParams = config.QEMU_ARCH # type: ignore + if extra_qemu_args: + params.extra_qemu_params.extend(extra_qemu_args) return params.linux_arch, LinuxSourceTreeOperationsQemu( params, cross_compile=cross_compile) @@ -220,17 +223,18 @@ class LinuxSourceTree: kconfig_add: Optional[List[str]]=None, arch=None, cross_compile=None, - qemu_config_path=None) -> None: + qemu_config_path=None, + extra_qemu_args=None) -> None: signal.signal(signal.SIGINT, self.signal_handler) if qemu_config_path: - self._arch, self._ops = _get_qemu_ops(qemu_config_path, cross_compile) + self._arch, self._ops = _get_qemu_ops(qemu_config_path, extra_qemu_args, cross_compile) else: self._arch = 'um' if arch is None else arch if self._arch == 'um': self._ops = LinuxSourceTreeOperationsUml(cross_compile=cross_compile) else: qemu_config_path = _default_qemu_config_path(self._arch) - _, self._ops = _get_qemu_ops(qemu_config_path, cross_compile) + _, self._ops = _get_qemu_ops(qemu_config_path, extra_qemu_args, cross_compile) if kunitconfig_path: if os.path.isdir(kunitconfig_path): diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py index baee11d96474..7fe5c8b0fb57 100755 --- a/tools/testing/kunit/kunit_tool_test.py +++ b/tools/testing/kunit/kunit_tool_test.py @@ -649,7 +649,8 @@ class KUnitMainTest(unittest.TestCase): kconfig_add=None, arch='um', cross_compile=None, - qemu_config_path=None) + qemu_config_path=None, + extra_qemu_args=[]) def test_config_kunitconfig(self): kunit.main(['config', '--kunitconfig=mykunitconfig']) @@ -659,7 +660,8 @@ class KUnitMainTest(unittest.TestCase): kconfig_add=None, arch='um', cross_compile=None, - qemu_config_path=None) + qemu_config_path=None, + extra_qemu_args=[]) def test_run_kconfig_add(self): kunit.main(['run', '--kconfig_add=CONFIG_KASAN=y', '--kconfig_add=CONFIG_KCSAN=y']) @@ -669,7 +671,19 @@ class KUnitMainTest(unittest.TestCase): kconfig_add=['CONFIG_KASAN=y', 'CONFIG_KCSAN=y'], arch='um', cross_compile=None, - qemu_config_path=None) + qemu_config_path=None, + extra_qemu_args=[]) + + def test_run_qemu_args(self): + kunit.main(['run', '--arch=x86_64', '--qemu_args', '-m 2048']) + # Just verify that we parsed and initialized it correctly here. + self.mock_linux_init.assert_called_once_with('.kunit', + kunitconfig_path=None, + kconfig_add=None, + arch='x86_64', + cross_compile=None, + qemu_config_path=None, + extra_qemu_args=['-m', '2048']) def test_run_kernel_args(self): kunit.main(['run', '--kernel_args=a=1', '--kernel_args=b=2']) -- 2.36.1.124.g0e6072fb45-goog ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [PATCH 3/3] kunit: tool: introduce --qemu_args 2022-05-18 17:01 ` [PATCH 3/3] kunit: tool: introduce --qemu_args Daniel Latypov @ 2022-05-19 13:20 ` David Gow 2022-07-06 20:11 ` Brendan Higgins 1 sibling, 0 replies; 10+ messages in thread From: David Gow @ 2022-05-19 13:20 UTC (permalink / raw) To: Daniel Latypov Cc: Brendan Higgins, Marco Elver, Linux Kernel Mailing List, kasan-dev, KUnit Development, open list:KERNEL SELFTEST FRAMEWORK, Shuah Khan [-- Attachment #1: Type: text/plain, Size: 8602 bytes --] On Thu, May 19, 2022 at 1:01 AM Daniel Latypov <dlatypov@google.com> wrote: > > Example usage: > $ ./tools/testing/kunit/kunit.py run --arch=x86_64 \ > --kconfig_add=CONFIG_SMP=y --qemu_args='-smp 8' > > Looking in the test.log, one can see > > smp: Bringing up secondary CPUs ... > > .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 > > smp: Brought up 1 node, 8 CPUs > > This flag would allow people to make tweaks like this without having to > create custom qemu_config files. > > For consistency with --kernel_args, we allow users to repeat this > argument, e.g. you can tack on a --qemu_args='-m 2048', or you could > just append it to the first string ('-smp 8 -m 2048'). > > Signed-off-by: Daniel Latypov <dlatypov@google.com> > --- I like this -- it's definitely something I've wanted to have in the past. I was a bit worried about how we'd handle longer strings of arguments, but the combination of the shlex-based splitting and repeated arguments seems to work for all the cases I can think of. (And it's much nicer than, e.g, passing linker flags with -Wl,a,b,c in my opinion!) Reviewed-by: David Gow <davidgow@google.com> -- David > tools/testing/kunit/kunit.py | 14 +++++++++++++- > tools/testing/kunit/kunit_kernel.py | 10 +++++++--- > tools/testing/kunit/kunit_tool_test.py | 20 +++++++++++++++++--- > 3 files changed, 37 insertions(+), 7 deletions(-) > > diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py > index 8a90d80ee66e..e01c7964f744 100755 > --- a/tools/testing/kunit/kunit.py > +++ b/tools/testing/kunit/kunit.py > @@ -10,6 +10,7 @@ > import argparse > import os > import re > +import shlex > import sys > import time > > @@ -323,6 +324,10 @@ def add_common_opts(parser) -> None: > 'a QemuArchParams object.'), > type=str, metavar='FILE') > > + parser.add_argument('--qemu_args', > + help='Additional QEMU arguments, e.g. "-smp 8"', > + action='append', metavar='') > + > def add_build_opts(parser) -> None: > parser.add_argument('--jobs', > help='As in the make command, "Specifies the number of ' > @@ -368,12 +373,19 @@ def add_parse_opts(parser) -> None: > > def tree_from_args(cli_args: argparse.Namespace) -> kunit_kernel.LinuxSourceTree: > """Returns a LinuxSourceTree based on the user's arguments.""" > + # Allow users to specify multiple arguments in one string, e.g. '-smp 8' > + qemu_args: List[str] = [] > + if cli_args.qemu_args: > + for arg in cli_args.qemu_args: > + qemu_args.extend(shlex.split(arg)) > + > return kunit_kernel.LinuxSourceTree(cli_args.build_dir, > kunitconfig_path=cli_args.kunitconfig, > kconfig_add=cli_args.kconfig_add, > arch=cli_args.arch, > cross_compile=cli_args.cross_compile, > - qemu_config_path=cli_args.qemu_config) > + qemu_config_path=cli_args.qemu_config, > + extra_qemu_args=qemu_args) > > > def main(argv): > diff --git a/tools/testing/kunit/kunit_kernel.py b/tools/testing/kunit/kunit_kernel.py > index e93f07ac0af1..a791073d25f9 100644 > --- a/tools/testing/kunit/kunit_kernel.py > +++ b/tools/testing/kunit/kunit_kernel.py > @@ -187,6 +187,7 @@ def _default_qemu_config_path(arch: str) -> str: > raise ConfigError(arch + ' is not a valid arch, options are ' + str(sorted(options))) > > def _get_qemu_ops(config_path: str, > + extra_qemu_args: Optional[List[str]], > cross_compile: Optional[str]) -> Tuple[str, LinuxSourceTreeOperations]: > # The module name/path has very little to do with where the actual file > # exists (I learned this through experimentation and could not find it > @@ -207,6 +208,8 @@ def _get_qemu_ops(config_path: str, > if not hasattr(config, 'QEMU_ARCH'): > raise ValueError('qemu_config module missing "QEMU_ARCH": ' + config_path) > params: qemu_config.QemuArchParams = config.QEMU_ARCH # type: ignore > + if extra_qemu_args: > + params.extra_qemu_params.extend(extra_qemu_args) > return params.linux_arch, LinuxSourceTreeOperationsQemu( > params, cross_compile=cross_compile) > > @@ -220,17 +223,18 @@ class LinuxSourceTree: > kconfig_add: Optional[List[str]]=None, > arch=None, > cross_compile=None, > - qemu_config_path=None) -> None: > + qemu_config_path=None, > + extra_qemu_args=None) -> None: > signal.signal(signal.SIGINT, self.signal_handler) > if qemu_config_path: > - self._arch, self._ops = _get_qemu_ops(qemu_config_path, cross_compile) > + self._arch, self._ops = _get_qemu_ops(qemu_config_path, extra_qemu_args, cross_compile) > else: > self._arch = 'um' if arch is None else arch > if self._arch == 'um': > self._ops = LinuxSourceTreeOperationsUml(cross_compile=cross_compile) > else: > qemu_config_path = _default_qemu_config_path(self._arch) > - _, self._ops = _get_qemu_ops(qemu_config_path, cross_compile) > + _, self._ops = _get_qemu_ops(qemu_config_path, extra_qemu_args, cross_compile) > > if kunitconfig_path: > if os.path.isdir(kunitconfig_path): > diff --git a/tools/testing/kunit/kunit_tool_test.py b/tools/testing/kunit/kunit_tool_test.py > index baee11d96474..7fe5c8b0fb57 100755 > --- a/tools/testing/kunit/kunit_tool_test.py > +++ b/tools/testing/kunit/kunit_tool_test.py > @@ -649,7 +649,8 @@ class KUnitMainTest(unittest.TestCase): > kconfig_add=None, > arch='um', > cross_compile=None, > - qemu_config_path=None) > + qemu_config_path=None, > + extra_qemu_args=[]) > > def test_config_kunitconfig(self): > kunit.main(['config', '--kunitconfig=mykunitconfig']) > @@ -659,7 +660,8 @@ class KUnitMainTest(unittest.TestCase): > kconfig_add=None, > arch='um', > cross_compile=None, > - qemu_config_path=None) > + qemu_config_path=None, > + extra_qemu_args=[]) > > def test_run_kconfig_add(self): > kunit.main(['run', '--kconfig_add=CONFIG_KASAN=y', '--kconfig_add=CONFIG_KCSAN=y']) > @@ -669,7 +671,19 @@ class KUnitMainTest(unittest.TestCase): > kconfig_add=['CONFIG_KASAN=y', 'CONFIG_KCSAN=y'], > arch='um', > cross_compile=None, > - qemu_config_path=None) > + qemu_config_path=None, > + extra_qemu_args=[]) > + > + def test_run_qemu_args(self): > + kunit.main(['run', '--arch=x86_64', '--qemu_args', '-m 2048']) > + # Just verify that we parsed and initialized it correctly here. > + self.mock_linux_init.assert_called_once_with('.kunit', > + kunitconfig_path=None, > + kconfig_add=None, > + arch='x86_64', > + cross_compile=None, > + qemu_config_path=None, > + extra_qemu_args=['-m', '2048']) > > def test_run_kernel_args(self): > kunit.main(['run', '--kernel_args=a=1', '--kernel_args=b=2']) > -- > 2.36.1.124.g0e6072fb45-goog > [-- Attachment #2: S/MIME Cryptographic Signature --] [-- Type: application/pkcs7-signature, Size: 4003 bytes --] ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [PATCH 3/3] kunit: tool: introduce --qemu_args 2022-05-18 17:01 ` [PATCH 3/3] kunit: tool: introduce --qemu_args Daniel Latypov 2022-05-19 13:20 ` David Gow @ 2022-07-06 20:11 ` Brendan Higgins 1 sibling, 0 replies; 10+ messages in thread From: Brendan Higgins @ 2022-07-06 20:11 UTC (permalink / raw) To: Daniel Latypov Cc: davidgow, elver, linux-kernel, kasan-dev, kunit-dev, linux-kselftest, skhan On Wed, May 18, 2022 at 1:01 PM Daniel Latypov <dlatypov@google.com> wrote: > > Example usage: > $ ./tools/testing/kunit/kunit.py run --arch=x86_64 \ > --kconfig_add=CONFIG_SMP=y --qemu_args='-smp 8' > > Looking in the test.log, one can see > > smp: Bringing up secondary CPUs ... > > .... node #0, CPUs: #1 #2 #3 #4 #5 #6 #7 > > smp: Brought up 1 node, 8 CPUs > > This flag would allow people to make tweaks like this without having to > create custom qemu_config files. > > For consistency with --kernel_args, we allow users to repeat this > argument, e.g. you can tack on a --qemu_args='-m 2048', or you could > just append it to the first string ('-smp 8 -m 2048'). > > Signed-off-by: Daniel Latypov <dlatypov@google.com> Reviewed-by: Brendan Higgins <brendanhiggins@google.com> ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-07-06 20:12 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2022-05-18 17:01 [PATCH 0/3] kunit: add support in kunit.py for --qemu_args Daniel Latypov 2022-05-18 17:01 ` [PATCH 1/3] Documentation: kunit: fix example run_kunit func to allow spaces in args Daniel Latypov 2022-05-19 13:20 ` David Gow 2022-07-06 18:34 ` Brendan Higgins 2022-05-18 17:01 ` [PATCH 2/3] kunit: tool: simplify creating LinuxSourceTreeOperations Daniel Latypov 2022-05-19 13:20 ` David Gow 2022-07-06 20:06 ` Brendan Higgins 2022-05-18 17:01 ` [PATCH 3/3] kunit: tool: introduce --qemu_args Daniel Latypov 2022-05-19 13:20 ` David Gow 2022-07-06 20:11 ` Brendan Higgins
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.