[nft PATCH 0/4] Extend testsuites to run against installed binaries

[nft PATCH 0/4] Extend testsuites to run against installed binaries

[Phil Sutter]

Help with CI integration by allowing to run the testsuite on installed
binaries instead of the local ones in the built source tree.

This series contains an unrelated Python3 fix for json_echo test tool in
patch 1, the remaining three extend json_echo, monitor and py testsuites
as described. Of the remaining testsuites, shell already accepts NFT env
variable and build is bound to source tree anyway.

Phil Sutter (4):
  tests: json_echo: Fix for Python3
  tests: json_echo: Support testing host binaries
  tests: monitor: Support testing host's nft binary
  tests: py: Support testing host binaries

 tests/json_echo/run-test.py | 25 ++++++++++++++++++++-----
 tests/monitor/run-tests.sh  |  4 ++++
 tests/py/nft-test.py        | 22 ++++++++++++++++++----
 3 files changed, 42 insertions(+), 9 deletions(-)

-- 
2.24.1

[nft PATCH 1/4] tests: json_echo: Fix for Python3

[Phil Sutter]

The keys() method returns an object which does not support indexing, so
convert it to a list prior to doing so.

Fixes: a35e3a0cdc63a ("tests: json_echo: convert to py3")
Signed-off-by: Phil Sutter <phil@nwl.cc>
---
 tests/json_echo/run-test.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tests/json_echo/run-test.py b/tests/json_echo/run-test.py
index a636d5f247702..fa7d69ab75645 100755
--- a/tests/json_echo/run-test.py
+++ b/tests/json_echo/run-test.py
@@ -119,7 +119,7 @@ def get_handle(output, search):
             else:
                 data = item
 
-            k = search.keys()[0]
+            k = list(search.keys())[0]
 
             if not k in data:
                 continue
-- 
2.24.1

[nft PATCH 2/4] tests: json_echo: Support testing host binaries

[Phil Sutter]

Support -H/--host option to use host's libnftables.so.1. Alternatively
users may specify a custom library path via -l/--library option.

Signed-off-by: Phil Sutter <phil@nwl.cc>
---
 tests/json_echo/run-test.py | 23 +++++++++++++++++++----
 1 file changed, 19 insertions(+), 4 deletions(-)

diff --git a/tests/json_echo/run-test.py b/tests/json_echo/run-test.py
index fa7d69ab75645..36a377ac95eec 100755
--- a/tests/json_echo/run-test.py
+++ b/tests/json_echo/run-test.py
@@ -4,6 +4,7 @@ from __future__ import print_function
 import sys
 import os
 import json
+import argparse
 
 TESTS_PATH = os.path.dirname(os.path.abspath(__file__))
 sys.path.insert(0, os.path.join(TESTS_PATH, '../../py/'))
@@ -13,12 +14,26 @@ from nftables import Nftables
 # Change working directory to repository root
 os.chdir(TESTS_PATH + "/../..")
 
-if not os.path.exists('src/.libs/libnftables.so'):
-    print("The nftables library does not exist. "
-          "You need to build the project.")
+parser = argparse.ArgumentParser(description='Run JSON echo tests')
+parser.add_argument('-H', '--host', action='store_true',
+                    help='Run tests against installed libnftables.so.1')
+parser.add_argument('-l', '--library', default=None,
+                    help='Path to libntables.so, overrides --host')
+args = parser.parse_args()
+
+check_lib_path = True
+if args.library is None:
+    if args.host:
+        args.library = 'libnftables.so.1'
+        check_lib_path = False
+    else:
+        args.library = 'src/.libs/libnftables.so.1'
+
+if check_lib_path and not os.path.exists(args.library):
+    print("Library not found at '%s'." % args.library)
     sys.exit(1)
 
-nftables = Nftables(sofile = 'src/.libs/libnftables.so')
+nftables = Nftables(sofile = args.library)
 nftables.set_echo_output(True)
 
 # various commands to work with
-- 
2.24.1

[nft PATCH 3/4] tests: monitor: Support testing host's nft binary

[Phil Sutter]

Add support for -H/--host flag to use 'nft' tool from $PATH instead of
the local one.

Signed-off-by: Phil Sutter <phil@nwl.cc>
---
 tests/monitor/run-tests.sh | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/monitor/run-tests.sh b/tests/monitor/run-tests.sh
index efacdaaab952b..ffb833a7f86f0 100755
--- a/tests/monitor/run-tests.sh
+++ b/tests/monitor/run-tests.sh
@@ -119,6 +119,10 @@ while [ -n "$1" ]; do
 		test_json=true
 		shift
 		;;
+	-H|--host)
+		nft=nft
+		shift
+		;;
 	testcases/*.t)
 		testcases+=" $1"
 		shift
-- 
2.24.1

[nft PATCH 4/4] tests: py: Support testing host binaries

[Phil Sutter]

Support -H/--host option to use host's libnftables.so.1. Alternatively
users may specify a custom library path via -l/--library option.

Signed-off-by: Phil Sutter <phil@nwl.cc>
---
 tests/py/nft-test.py | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/tests/py/nft-test.py b/tests/py/nft-test.py
index 6edca3c6a5a2f..01ee6c980ad4a 100755
--- a/tests/py/nft-test.py
+++ b/tests/py/nft-test.py
@@ -1357,10 +1357,16 @@ def main():
                         dest='force_all_family',
                         help='keep testing all families on error')
 
+    parser.add_argument('-H', '--host', action='store_true',
+                        help='run tests against installed libnftables.so.1')
+
     parser.add_argument('-j', '--enable-json', action='store_true',
                         dest='enable_json',
                         help='test JSON functionality as well')
 
+    parser.add_argument('-l', '--library', default=None,
+                        help='path to libntables.so.1, overrides --host')
+
     parser.add_argument('-s', '--schema', action='store_true',
                         dest='enable_schema',
                         help='verify json input/output against schema')
@@ -1388,9 +1394,17 @@ def main():
     # Change working directory to repository root
     os.chdir(TESTS_PATH + "/../..")
 
-    if not os.path.exists('src/.libs/libnftables.so'):
-        print("The nftables library does not exist. "
-              "You need to build the project.")
+    check_lib_path = True
+    if args.library is None:
+        if args.host:
+            args.library = 'libnftables.so.1'
+            check_lib_path = False
+        else:
+            args.library = 'src/.libs/libnftables.so.1'
+
+    if check_lib_path and not os.path.exists(args.library):
+        print("The nftables library at '%s' does not exist. "
+              "You need to build the project." % args.library)
         return
 
     if args.enable_schema and not args.enable_json:
@@ -1398,7 +1412,7 @@ def main():
         return
 
     global nftables
-    nftables = Nftables(sofile = 'src/.libs/libnftables.so')
+    nftables = Nftables(sofile = args.library)
 
     test_files = files_ok = run_total = 0
     tests = passed = warnings = errors = 0
-- 
2.24.1

Re: [nft PATCH 0/4] Extend testsuites to run against installed binaries

[Pablo Neira Ayuso]

On Thu, Feb 06, 2020 at 01:58:47AM +0100, Phil Sutter wrote:
> Help with CI integration by allowing to run the testsuite on installed
> binaries instead of the local ones in the built source tree.
> 
> This series contains an unrelated Python3 fix for json_echo test tool in
> patch 1, the remaining three extend json_echo, monitor and py testsuites
> as described. Of the remaining testsuites, shell already accepts NFT env
> variable and build is bound to source tree anyway.

LGTM.