[RFC net-next 1/1] tdc.py: Introduce required plugins

* [RFC net-next 1/1] tdc.py: Introduce required plugins
@ 2019-04-09 21:44 Lucas Bates
  2019-04-10 14:33 ` Nicolas Dichtel
  0 siblings, 1 reply; 9+ messages in thread
From: Lucas Bates @ 2019-04-09 21:44 UTC (permalink / raw)
  To: netdev
  Cc: davem, jhs, xiyou.wangcong, jiri, mleitner, vladbu, dcaratti,
	nicolas.dichtel, kernel, Lucas Bates

Some of the testcases (for example, all of the fw tests) in tdc
require activating the nsplugin. This RFC introduces a feature which
tags one such test with the keyword "requires". Anyone running a test
that requires nsplugin will now get a warning if they are missing
the plugin.

After compiling the list of test cases to execute, tdc will
gather all of the required plugins for that run, and validate
that they have been enabled with a symlink in the plugins/
directory. If required plugins are missing, tdc will create the
symlink for the user and then terminate. (This is because plugin-
specific options may exist and need to be parsed at startup)

Please provide feedback.  If this is amenable to all, I will proceed
to submit for the rest.

Signed-off-by: Lucas Bates <lucasb@mojatatu.com>
---
 tools/testing/selftests/tc-testing/README          | 28 ++++++++++-----
 .../selftests/tc-testing/tc-tests/filters/fw.json  |  1 +
 tools/testing/selftests/tc-testing/tdc.py          | 41 ++++++++++++++++++++++
 3 files changed, 62 insertions(+), 8 deletions(-)

diff --git a/tools/testing/selftests/tc-testing/README b/tools/testing/selftests/tc-testing/README
index f9281e8..b91ce8b 100644
--- a/tools/testing/selftests/tc-testing/README
+++ b/tools/testing/selftests/tc-testing/README
@@ -36,14 +36,20 @@ REQUIREMENTS
 BEFORE YOU RUN
 --------------

-The path to the tc executable that will be most commonly tested can be defined
-in the tdc_config.py file. Find the 'TC' entry in the NAMES dictionary and
-define the path.

-If you need to test a different tc executable on the fly, you can do so by
-using the -p option when running tdc:
+*  The path to the tc executable that will be most commonly tested can be
+   defined in the tdc_config.py file. Find the 'TC' entry in the NAMES
+   dictionary and define the path.
+
+   If you need to test a different tc executable on the fly, you can do so by
+   using the -p option when running tdc:
 	./tdc.py -p /path/to/tc

+*  Tests may be executed in a network namespace with a port that is created
+   automatically. To use this feature, create a symlink in the plugins/
+   directory to plugin-lib/nsPlugin.py and use the '-n' option when
+   executing tdc. If this is not done, you MUST have a port with the same
+   name as the value of DEV1 in tdc_config.py before running tdc.

 RUNNING TDC
 -----------
@@ -53,9 +59,6 @@ commands being tested must be run as root.  The code that enforces
 execution by root uid has been moved into a plugin (see PLUGIN
 ARCHITECTURE, below).

-If nsPlugin is linked, all tests are executed inside a network
-namespace to prevent conflicts within the host.
-
 Running tdc without any arguments will run all tests. Refer to the section
 on command line arguments for more information, or run:
 	./tdc.py -h
@@ -235,6 +238,15 @@ directory:
   - buildebpfPlugin.py:
       builds all programs in $EBPFDIR.

+If a plugin is required for a test case to succeed, it can be defined in
+the test case definition as follows:
+	"requires": ["nsPlugin", "fooPlugin"]
+
+This will cause tdc to verify that these plugins are present before
+starting test execution. If they aren't, the symlinks will be created
+for you and tdc will terminate to be restarted. This is done as some
+plugins may have command line options that are needed for a successful run.
+

 ACKNOWLEDGEMENTS
 ----------------
diff --git a/tools/testing/selftests/tc-testing/tc-tests/filters/fw.json b/tools/testing/selftests/tc-testing/tc-tests/filters/fw.json
index 3b97cfd..dfba083 100644
--- a/tools/testing/selftests/tc-testing/tc-tests/filters/fw.json
+++ b/tools/testing/selftests/tc-testing/tc-tests/filters/fw.json
@@ -6,6 +6,7 @@
             "filter",
             "fw"
         ],
+	"requires": ["nsPlugin"],
         "setup": [
             "$TC qdisc add dev $DEV1 ingress"
         ],
diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py
index 5cee156..aa9a804 100755
--- a/tools/testing/selftests/tc-testing/tdc.py
+++ b/tools/testing/selftests/tc-testing/tdc.py
@@ -550,6 +550,46 @@ def filter_tests_by_category(args, testlist):

     return answer

+def check_required_plugins(pm, testlist):
+    '''
+    Get all required plugins from the list of test cases and ensure
+    they're enabled.
+    '''
+    reqs = []
+    pnf = False
+    plinked = False
+    for t in testlist:
+        try:
+            reqs.extend(t['requires'])
+        except KeyError:
+            continue
+    reqs = get_unique_item(reqs)
+
+    for r in reqs:
+        if r not in pm.plugins:
+            import glob
+            fname = '{}.py'.format(r)
+            source_path = []
+            source_path.extend(glob.glob('plugin-lib/{}'.format(fname)))
+            source_path.extend(glob.glob('plugin-lib-custom/{}'.format(fname)))
+            if len(source_path) == 0:
+                print('ERROR: unable to find required plugin {}'.format(r))
+                pnf = True
+                continue
+            elif len(source_path) > 1:
+                print('WARNING: multiple copies of plugin {} found, using version found')
+                print('at {}'.format(source_path[0]))
+            os.symlink('../{}'.format(source_path[0]), 'plugins/{}'.format(fname))
+            os.chown('plugins/{}'.format(fname), uid=int(os.getenv('SUDO_UID')),
+                        gid=int(os.getenv('SUDO_GID')))
+            plinked = True
+    if pnf:
+        print('Plugins that are required could not be found. Please refer to the README.')
+    if plinked:
+        print('Required plugins have been linked in. Please re-run tdc to enable their options.')
+    if pnf or plinked:
+        exit(1)
+
 def get_test_cases(args):
     """
     If a test case file is specified, retrieve tests from that file.
@@ -649,6 +689,7 @@ def set_operation_mode(pm, args):
             exit(0)

     if len(alltests):
+        check_required_plugins(pm, alltests)
         catresults = test_runner(pm, args, alltests)
         if args.format == 'none':
             print('Test results output suppression requested\n')
--
2.7.4


^ permalink raw reply related	[flat|nested] 9+ messages in thread