* [PATCH v5 02/26] auto-t: introduce pure python test-runner re-write
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 03/26] auto-t: hostapd.py: update to work with test-runner rewrite James Prestwood
` (24 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 40448 bytes --]
This patch completely re-writes test-runner in Python. This was done
because the existing C test-runner had some clunky work arounds and
maintaining or adding new features was starting to become a huge pain.
There were a few aspects of test-runner which continually had to
be dealt with when adding any new functionality:
* Argument parsing: Adding new arguments to test-runner wasn't so
bad, but if you wanted those arguments passed into the VM it
became a huge pain. Arguments needed to be parsed, then re-formatted
into the qemu command line, then re-parsed in a special order
(backwards) once in the VM. The burden for adding new arguments was
quite high so it was avoided (at least by me) at all costs.
* The separation between C and Python: The tests are all written in
python, but the executables, radios, and interfaces were all created
from C. The way we solved this was by encoding the require info as
environment variables, then parsing those from Python. It worked,
but it was, again, a huge pain.
* Process management: It started with all processes being launched
from C, but eventually tests required the ability to start IWD, or
kill hostapd ungracefully in order to test certain functionality.
Since the processes were tracked in C, Python had no way of
signalling that it killed a process and when it started one C had
no idea. This was mitigated (basically by killall), but it was
no where close to an elegant solution.
Re-writing test-runner in python solves all these problems and will
be much easier to maintain.
* Argument parsing: Now all arguments are forwarded automatically
to the VM. The ArgParse library takes care of parsing and each
argument is stored in a dictionary.
* Separation between C and Python: No more C, so no more separation.
* Process management: Python will now manage all processes. This
allows a test to kill, restart, or start a new process and not
have to remember the PID or to kill it after the test.
There are a few more important aspects of the python implementation
that should now be considered when writing new tests:
* The IWD constructor now has different default arugments. IWD
will always be started unless specified and the configuration
directory will always be /tmp
* Any non *.py file in the test directory will be copied to /tmp.
This avoids the need for 'tmpfs_extra_stuff' completely.
* ctrl_interface will automatically be appended to every hostapd
config. There is no need to include this in a config file from
now on.
* Test cleanup is extremely important. All tests get run in the
same interpreter now and the tests themselves are actually loaded
as python modules. This means e.g. if you somehow kept a reference
to IWD() any subsequent tests would not start since IWD is still
running.
* For debugging, the test context can be printed which shows running
processes, radios, and interfaces.
Three non-native python modules were used: PrettyTable, colored, and
pyroute2
$ pip3 install prettytable
$ pip3 install termcolor
$ pip3 install pyroute2
---
autotests/util/config.py | 6 +
tools/py_runner.py | 1270 ++++++++++++++++++++++++++++++++++++++
2 files changed, 1276 insertions(+)
create mode 100644 autotests/util/config.py
create mode 100755 tools/py_runner.py
diff --git a/autotests/util/config.py b/autotests/util/config.py
new file mode 100644
index 00000000..be8edbfc
--- /dev/null
+++ b/autotests/util/config.py
@@ -0,0 +1,6 @@
+#
+# Acts as a global store for test information. 'ctx' is set by the creator of
+# of the TestContext (test-runner).
+#
+
+ctx = {}
diff --git a/tools/py_runner.py b/tools/py_runner.py
new file mode 100755
index 00000000..d09f08d2
--- /dev/null
+++ b/tools/py_runner.py
@@ -0,0 +1,1270 @@
+#!/usr/bin/python3
+
+import argparse
+import os
+import shutil
+import ctypes
+import fcntl
+import shlex
+import sys
+import subprocess
+import atexit
+import time
+import unittest
+import importlib
+import signal
+import pyroute2
+import multiprocessing
+import re
+
+from configparser import ConfigParser
+from prettytable import PrettyTable
+from termcolor import colored
+from glob import glob
+from collections import namedtuple
+from time import sleep
+import dbus.mainloop.glib
+from gi.repository import GLib
+
+libc = ctypes.cdll['libc.so.6']
+libc.mount.argtypes = (ctypes.c_char_p, ctypes.c_char_p, ctypes.c_char_p, \
+ ctypes.c_ulong, ctypes.c_char_p)
+
+# Using ctypes to load the libc library is somewhat low level. Because of this
+# we need to define our own flags/options for use with mounting.
+MS_NOSUID = 2
+MS_NODEV = 4
+MS_NOEXEC = 8
+MS_STRICTATIME = 1 << 24
+STDIN_FILENO = 0
+TIOCSTTY = 0x540E
+
+config = None
+intf_id = 0
+rad_id = 0
+
+TEST_MAX_TIMEOUT = 45
+
+dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
+
+def dbg(*s):
+ '''
+ Allows prints if stdout has been re-directed
+ '''
+ print(*s, file=sys.__stdout__)
+
+def exit_vm():
+ if config:
+ for p in config.ctx.processes:
+ print("Process %s still running!" % p.name)
+
+ os.sync()
+
+ RB_AUTOBOOT = 0x01234567
+ #
+ # Calling 'reboot' or 'shutdown' from a shell (e.g. os.system('reboot'))
+ # is not the same the POSIX reboot() and will cause a kernel panic since
+ # we are the init process. The libc.reboot() allows the VM to exit
+ # gracefully.
+ #
+ libc.reboot(RB_AUTOBOOT)
+
+def path_exists(path):
+ '''
+ Searches PATH as well as absolute paths.
+ '''
+ if shutil.which(path):
+ return True
+ try:
+ os.stat(path)
+ except:
+ return False
+ return True
+
+def find_binary(list):
+ '''
+ Returns a binary from 'list' if its found in PATH or on a
+ valid absolute path.
+ '''
+ for path in list:
+ if path_exists(path):
+ return path
+ return None
+
+def mount(source, target, fs, flags, options=''):
+ '''
+ Python wrapper for libc mount()
+ '''
+ ret = libc.mount(source.encode(), target.encode(), fs.encode(), flags,
+ options.encode())
+ if ret < 0:
+ errno = ctypes.get_errno()
+ raise Exception("Could not mount %s (%d)" % (target, errno))
+
+MountInfo = namedtuple('MountInfo', 'fstype target options flags')
+
+mount_table = [
+ MountInfo('sysfs', '/sys', '', MS_NOSUID|MS_NOEXEC|MS_NODEV),
+ MountInfo('proc', '/proc', '', MS_NOSUID|MS_NOEXEC|MS_NODEV),
+ MountInfo('devpts', '/dev/pts', 'mode=0620', MS_NOSUID|MS_NOEXEC),
+ MountInfo('tmpfs', '/dev/shm', 'mode=1777', MS_NOSUID|MS_NODEV|MS_STRICTATIME),
+ MountInfo('tmpfs', '/run', 'mode=0755', MS_NOSUID|MS_NODEV|MS_STRICTATIME),
+ MountInfo('tmpfs', '/var/lib/iwd', 'mode=0755', 0),
+ MountInfo('tmpfs', '/tmp', '', 0),
+ MountInfo('tmpfs', '/usr/share/dbus-1', 'mode=0755', MS_NOSUID|MS_NOEXEC|MS_NODEV|MS_STRICTATIME),
+ MountInfo('debugfs', '/sys/kernel/debug', '', 0)
+]
+
+DevInfo = namedtuple('DevInfo', 'target linkpath')
+
+dev_table = [
+ DevInfo('/proc/self/fd', '/dev/fd'),
+ DevInfo('/proc/self/fd/0', '/dev/stdin'),
+ DevInfo('/proc/self/fd/1', '/dev/stdout'),
+ DevInfo('/proc/self/fd/2', '/dev/stderr')
+]
+
+dbus_config = '''
+<!DOCTYPE busconfig PUBLIC \
+"-//freedesktop//DTD D-Bus Bus Configuration 1.0//EN" \
+"http://www.freedesktop.org/standards/dbus/1.0/\
+busconfig.dtd\">
+<busconfig>
+<type>system</type>
+<listen>unix:path=/run/dbus/system_bus_socket</listen>
+<limit name=\"reply_timeout\">2147483647</limit>
+<auth>ANONYMOUS</auth>
+<allow_anonymous/>
+<policy context=\"default\">
+<allow user=\"*\"/>
+<allow own=\"*\"/>
+<allow send_type=\"method_call\"/>
+<allow send_type=\"signal\"/>
+<allow send_type=\"method_return\"/>
+<allow send_type=\"error\"/>
+<allow receive_type=\"method_call\"/>
+<allow receive_type=\"signal\"/>
+<allow receive_type=\"method_return\"/>
+<allow receive_type=\"error\"/>
+</policy>
+</busconfig>
+'''
+class Process:
+ '''
+ Start a process. If 'wait' is True the constructor will start
+ the process and wait for it to exit. No PID is tracked in this
+ case. If 'multi_test' is True this indicates the process is
+ run over the entire test run and will not be killed after each
+ test exits.
+ '''
+ def __init__(self, args, wait=False, multi_test=False, env=None, ctx=None, check=False):
+ self.args = args
+ self.wait = wait
+ self.name = args[0]
+ self.multi_test = multi_test
+ self.stdout = subprocess.PIPE
+ self.stderr = subprocess.PIPE
+ self.ret = None
+ self.ctx = ctx
+
+ if ctx:
+ set_stdout = False
+
+ if ctx.is_verbose(args[0]):
+ dbg("Verbose on for %s" % args[0])
+ set_stdout = True
+
+ if os.path.basename(args[0]) == ctx.args.gdb:
+ self.args = ['gdb', '--args']
+ self.args.extend(args)
+ set_stdout = True
+
+ # Anything labeled as multi_test isn't important to
+ # log. These are processes such as dbus-daemon and
+ # haveged.
+ if set_stdout:
+ if ctx.args.log:
+ test = os.path.basename(os.getcwd())
+ test_dir = '%s/%s' % (ctx.args.log, test)
+
+ if not path_exists(test_dir):
+ os.mkdir(test_dir)
+ os.chown(test_dir, int(ctx.args.log_uid), \
+ int(ctx.args.log_gid))
+
+ self.stdout = open('%s/%s' % (test_dir, args[0]), 'w')
+ self.stderr = open('%s/%s' % (test_dir, args[0]), 'w')
+ else:
+ self.stdout = sys.__stdout__
+ self.stderr = sys.__stderr__
+
+ if not wait and not check:
+ self.pid = subprocess.Popen(self.args, stdout=self.stdout, \
+ stderr=self.stderr, env=env, \
+ cwd=os.getcwd())
+ print("Starting process {}".format(self.pid.args))
+ else:
+ self.ret = subprocess.call(self.args, stdout=self.stdout, \
+ stderr=self.stderr)
+ print("%s returned %d" % (args[0], self.ret))
+ if check and self.ret != 0:
+ raise subprocess.CalledProcessError(returncode=self.ret, cmd=self.args)
+
+ def __del__(self):
+ print("Del process %s" % self.args)
+ if self.ctx and self.ctx.args.log:
+ self.stdout.close()
+ self.stderr.close()
+
+ def kill(self, force=False):
+ print("Killing process %s" % self.args)
+
+ if force:
+ os.kill(self.pid.pid, signal.SIGKILL)
+ else:
+ self.pid.kill()
+
+ self.pid.wait(timeout=15)
+
+ def wait_for_socket(self, socket, wait):
+ waited = 0
+ while not os.path.exists(socket):
+ sleep(0.5)
+ waited += 0.5
+ if waited > wait:
+ raise Exception("Timed out waiting for socket")
+
+class Interface:
+ def __init__(self, name, config):
+ self.name = name
+ self.ctrl_interface = '/var/run/hostapd/' + name
+ self.config = config
+
+ def __del__(self):
+ Process(['iw', 'dev', self.name, 'del'], True)
+
+ def set_interface_state(self, state):
+ Process(['ifconfig', self.name, state], True)
+
+class Radio:
+ def __init__(self, name):
+ self.name = name
+ # hostapd will reset this if this radio is used by it
+ self.use = 'iwd'
+ self.interface = None
+
+ def __del__(self):
+ print("Removing radio %s" % self.name)
+ self.interface = None
+
+ def create_interface(self, hapd):
+ global intf_id
+
+ ifname = 'wln%s' % intf_id
+
+ intf_id += 1
+
+ self.interface = Interface(ifname, hapd.config)
+ # IWD does not use interfaces in test-runner so any created
+ # interface is assumed to be used by hostapd.
+ self.use = 'hostapd'
+
+ Process(['iw', 'phy', self.name, 'interface', 'add', ifname,
+ 'type', 'managed'], True)
+
+ return self.interface
+
+ def __str__(self):
+ ret = self.name + ':\n'
+ ret += '\tUsed By: %s ' % self.use
+ if self.interface:
+ ret += '(%s)' % self.interface.name
+
+ ret += '\n'
+
+ return ret
+
+class VirtualRadio(Radio):
+ '''
+ A subclass of 'Radio' specific to mac80211_hwsim radios.
+
+ TODO: Using D-Bus to create and destroy radios is more desireable
+ than the command line.
+ '''
+ def __init__(self, name, config=None):
+ global rad_id
+
+ super().__init__(name)
+
+ self.disable_cipher = None
+ self.disable_iftype = None
+
+ args = ['hwsim', '--create', '--name', self.name, '--nointerface']
+
+ if config:
+ self.disable_iftype = config.get('iftype_disable', False)
+ if self.disable_iftype:
+ args.append('--iftype-disable')
+ args.append(self.disable_iftype)
+
+ self.disable_cipher = config.get('cipher_disable', False)
+ if self.disable_cipher:
+ args.append('--cipher-disable')
+ args.append(self.disable_cipher)
+
+ Process(args, wait=True)
+
+ self.id = rad_id
+ rad_id += 1
+
+ def __del__(self):
+ super().__del__()
+
+ Process(['hwsim', '--destroy=%s' % self.id])
+
+ def __str__(self):
+ ret = super().__str__()
+
+ if self.disable_iftype:
+ ret += '\tDisabled interface types: %s\n' % self.disable_iftype
+
+ if self.disable_cipher:
+ ret += '\tDisabled ciphers: %s\n' % self.disable_cipher
+
+ ret += '\n'
+
+ return ret
+
+class HostapdInstance:
+ '''
+ A single instance of hostapd. In reality all hostapd instances
+ are started as a single process. This class just makes things
+ convenient for communicating with one of the hostapd APs.
+ '''
+ def __init__(self, config, radio):
+ self.radio = radio
+ self.config = config
+
+ self.intf = radio.create_interface(self)
+ self.intf.set_interface_state('up')
+
+ def __del__(self):
+ print("Removing HostapdInstance %s" % self.config)
+ self.intf.set_interface_state('down')
+ self.radio = None
+ self.intf = None
+
+ def __str__(self):
+ ret = 'Hostapd (%s)\n' % self.intf.name
+ ret += '\tConfig: %s\n' % self.config
+
+ return ret
+
+class Hostapd:
+ '''
+ A set of running hostapd instances. This is really just a single
+ process since hostapd can be started with multiple config files.
+ '''
+ def __init__(self, ctx, radios, configs, radius):
+ if len(configs) != len(radios):
+ raise Exception("Config (%d) and radio (%d) list length not equal" % \
+ (len(configs), len(radios)))
+
+ print("Initializing hostapd instances")
+
+ self.global_ctrl_iface = '/var/run/hostapd/ctrl'
+
+ self.instances = [HostapdInstance(c, r) for c, r in zip(configs, radios)]
+
+ ifaces = [rad.interface.name for rad in radios]
+ ifaces = ','.join(ifaces)
+
+ args = ['hostapd', '-i', ifaces, '-g', self.global_ctrl_iface]
+
+ #
+ # Config files should already be present in /tmp. This appends
+ # ctrl_interface and does any variable replacement. Currently
+ # this is just any $ifaceN occurrences.
+ #
+ for c in configs:
+ full_path = '/tmp/%s' % c
+ args.append(full_path)
+
+ self._rewrite_config(full_path)
+
+ if radius:
+ args.append(radius)
+
+ if ctx.is_verbose('hostapd'):
+ args.append('-d')
+
+ self.process = ctx.start_process(args)
+
+ self.process.wait_for_socket(self.global_ctrl_iface, 30)
+
+ def _rewrite_config(self, config):
+ '''
+ Replaces any $ifaceN values with the correct interface
+ names as well as appends the ctrl_interface path to
+ the config file.
+ '''
+ with open(config, 'r+') as f:
+ data = f.read()
+ to_replace = []
+ for match in re.finditer(r'\$iface[0-9]+', data):
+ tag = data[match.start():match.end()]
+ idx = tag.split('iface')[1]
+
+ to_replace.append((tag, self.instances[int(idx)].intf.name))
+
+ for r in to_replace:
+ data = data.replace(r[0], r[1], 1)
+
+ data += '\nctrl_interface=/var/run/hostapd\n'
+
+ f.write(data)
+
+ def __getitem__(self, config):
+ if not config:
+ return self.instances[0]
+
+ for hapd in self.instances:
+ if hapd.config == config:
+ return hapd
+
+ return None
+
+ def __del__(self):
+ print("Removing Hostapd")
+ try:
+ os.remove(self.global_ctrl_iface)
+ except:
+ dbg("Failed to remove %s" % self.global_ctrl_iface)
+
+ self.instances = None
+ self.process.kill()
+
+class TestContext:
+ '''
+ Contains all information for a given set of tests being run
+ such as processes, radios, interfaces and test results.
+ '''
+ def __init__(self, args):
+ self.processes = []
+ self.args = args
+ self.hw_config = None
+ self.hostapd = None
+ self.cur_radio_id = 0
+ self.cur_iface_id = 0
+ self.radios = []
+ self.loopback_started = False
+ self.iwd_extra_options = None
+ self.results = {}
+ self.mainloop = GLib.MainLoop()
+
+ def start_process(self, args, wait=False, multi_test=False, env=None, check=False):
+ p = Process(args, wait, multi_test, env, ctx=self, check=check)
+
+ if not wait:
+ self.processes.append(p)
+
+ return p
+
+ def start_dbus(self):
+ with open('/usr/share/dbus-1/system.conf', 'w+') as f:
+ f.write(dbus_config)
+
+ os.mkdir('/run/dbus', 755)
+
+ self.start_process(['dbus-daemon', '--system', '--nosyslog'], multi_test=True)
+
+ def start_dbus_monitor(self):
+ if not self.is_verbose('dbus-monitor'):
+ return
+
+ self.start_process(['dbus-monitor', '--system'])
+
+ def start_haveged(self):
+ self.start_process(['haveged'], multi_test=True)
+
+ def create_radios(self):
+ setup = self.hw_config['SETUP']
+ nradios = int(setup['num_radios'])
+
+ for i in range(nradios):
+ name = 'rad%u' % i
+
+ # Get any [radX] sections. These are for configuring
+ # any special radios. This no longer requires a
+ # radio_conf list, we just assume radios start rad0
+ # and increment.
+ rad_config = None
+ if self.hw_config.has_section(name):
+ rad_config = self.hw_config[name]
+
+ self.radios.append(VirtualRadio(name, rad_config))
+ self.cur_radio_id += 1
+
+ # register hwsim as medium
+ self.start_process(['hwsim'])
+
+ def discover_radios(self):
+ phys = []
+ iw = pyroute2.iwutil.IW()
+
+ attrs = [phy['attrs'] for phy in iw.list_wiphy()]
+
+ for attr in attrs:
+ for key, value in attr:
+ if key == 'NL80211_ATTR_WIPHY_NAME':
+ if value not in phys:
+ phys.append(value)
+ break
+
+ print('Discovered radios: %s' % str(phys))
+ self.radios = [Radio(name) for name in phys]
+
+ def start_radios(self):
+ reg_domain = self.hw_config['SETUP'].get('reg_domain', None)
+ if reg_domain:
+ Process(['iw', 'reg', 'set', reg_domain], True)
+
+ if self.args.hw:
+ self.discover_radios()
+ else:
+ self.create_radios()
+
+ def start_iwd(self, config_dir = '/tmp'):
+ args = []
+ iwd_radios = ','.join([r.name for r in self.radios if r.use == 'iwd'])
+
+ if self.args.valgrind:
+ args.extend(['valgrind', '--leak-check=full', '--log-file=%s' % \
+ '/tmp/valgrind.log'])
+
+ args.extend(['iwd', '-p', iwd_radios])
+
+ if self.is_verbose(args[0]):
+ args.append('-d')
+
+ if self.iwd_extra_options:
+ args.append(self.iwd_extra_options)
+
+ env = os.environ.copy()
+ env['CONFIGURATION_DIRECTORY'] = config_dir
+ env['STATE_DIRECTORY'] = '/var/lib/iwd'
+
+ pid = self.start_process(args, env=env)
+ return pid
+
+ def start_hostapd(self):
+ if not 'HOSTAPD' in self.hw_config:
+ return
+
+ settings = self.hw_config['HOSTAPD']
+
+ if self.args.hw:
+ # Just grab the first N radios. It gets rather
+ # complicated trying to map radX radios specified in
+ # hw.conf so any passed through physical adapters are
+ # just given to hostapd/IWD as they appear during
+ # discovery.
+ #
+ # TODO: It may be desireable to map PCI/USB adapters to
+ # specific radX radios specified in the config but
+ # there are really 2 separate use cases here.
+ # 1. You want to test a *specific* radio with IWD
+ # or hostapd. For this you would want radX
+ # to map to a specific radio
+ # 2. You have many adapters in use to run multiple
+ # tests. In this case you would not care what
+ # was using each radio, just that there was
+ # enough to run all tests.
+ nradios = 0
+ for k, _ in settings.items():
+ if k == 'radius_server':
+ continue
+ nradios += 1
+
+ hapd_radios = self.radios[:nradios]
+
+ else:
+ hapd_radios = [rad for rad in self.radios if rad.name in settings]
+
+ hapd_configs = [conf for rad, conf in settings.items() if rad != 'radius_server']
+
+ radius_config = settings.get('radius_server', None)
+
+ self.hostapd = Hostapd(self, hapd_radios, hapd_configs, radius_config)
+
+ def start_ofono(self):
+ sim_keys = self.hw_config['SETUP'].get('sim_keys', None)
+ if not sim_keys:
+ print("Ofono not requred")
+ return
+ elif sim_keys != 'ofono':
+ os.environ['IWD_SIM_KEYS'] = sim_keys
+ self.iwd_extra_options = '--plugin=sim_hardcoded'
+ return
+
+ if not find_binary(['ofonod']) or not find_binary(['phonesim']):
+ print("Ofono or Phonesim not found, skipping test")
+ return
+
+ Process(['ifconfig', 'lo', 'up'], wait=True)
+
+ self.iwd_extra_options = '--plugin=ofono'
+
+ os.environ['OFONO_PHONESIM_CONFIG'] = '/tmp/phonesim.conf'
+
+ phonesim_args = ['phonesim', '-p', '12345', '/usr/share/phonesim/default.xml']
+
+ self.start_process(phonesim_args)
+
+ #
+ # TODO:
+ # Is there something to wait for? Without this phonesim rejects
+ # connections on all but the fist test.
+ #
+ time.sleep(3)
+
+ ofono_args = ['ofonod', '-n', '--plugin=atmodem,phonesim']
+ if self.is_verbose('ofonod'):
+ ofono_args.append('-d')
+
+ self.start_process(ofono_args)
+
+ print("Ofono started")
+
+ def is_verbose(self, process):
+ process = os.path.basename(process)
+
+ if self.args is None:
+ return False
+
+ # every process is verbose when logging is enabled
+ if self.args.log:
+ return True
+
+ if process in self.args.verbose:
+ return True
+
+ # Special case here to enable verbose output with valgrind running
+ if process == 'valgrind' and 'iwd' in self.args.verbose:
+ return True
+
+ # Handle any glob matches
+ for item in self.args.verbose:
+ if process in glob(item):
+ return True
+
+ return False
+
+ def stop_process(self, p, force=False):
+ p.kill(force)
+ self.processes.remove(p)
+
+ def stop_test_processes(self):
+ self.radios = []
+ self.hostapd = None
+ self.iwd_extra_options = None
+
+ for p in [p for p in self.processes if p.multi_test is False]:
+ print("Killing process %s" % p.name)
+ self.stop_process(p)
+
+ def is_process_running(self, process):
+ for p in self.processes:
+ if p.name == process:
+ return True
+ return False
+
+ def __str__(self):
+ ret = 'Arguments:\n'
+ for arg in vars(self.args):
+ ret += '\t --%s %s\n' % (arg, str(getattr(self.args, arg)))
+
+ ret += 'Processes:\n'
+ for p in self.processes:
+ ret += '\t%s\n' % str(p.args)
+
+ ret += 'Radios:\n'
+ if len(self.radios) > 0:
+ for r in self.radios:
+ ret += '\t%s\n' % str(r)
+ else:
+ ret += '\tNo Radios\n'
+
+ ret += 'Hostapd:\n'
+ if self.hostapd:
+ for h in self.hostapd.instances:
+ ret += '\t%s\n' % str(h)
+ else:
+ ret += '\tNo Hostapd instances\n'
+
+ return ret
+
+def prepare_sandbox():
+ print('Preparing sandbox')
+
+ for entry in mount_table:
+ try:
+ os.lstat(entry.target)
+ except:
+ os.mkdir(entry.target, 755)
+
+ mount(entry.fstype, entry.target, entry.fstype, entry.flags,
+ entry.options)
+
+ for entry in dev_table:
+ os.symlink(entry.target, entry.linkpath)
+
+ os.setsid()
+
+ fcntl.ioctl(STDIN_FILENO, TIOCSTTY, 1)
+
+def build_unit_list(args):
+ '''
+ Build list of unit tests based on passed arguments. This first
+ checks for literal names provided in the arguments, then if
+ no matches were found, checks for a glob match.
+ '''
+ tests = []
+ test_root = args.testhome + '/unit'
+
+ for unit in args.unit_tests.split(','):
+ path = '%s/%s' % (test_root, unit)
+ if os.access(unit, os.X_OK):
+ tests.append(unit)
+ elif os.access(path, os.X_OK):
+ tests.append(path)
+ else:
+ # Full list or glob, first build up valid list of tests
+ matches = glob(path)
+ if matches == []:
+ raise Exception("Could not find test %s" % unit)
+
+ matches = [exe for exe in matches if os.access(exe, os.X_OK)]
+
+ tests.extend(matches)
+
+ return sorted(tests)
+
+def build_test_list(args):
+ '''
+ Build list of auto test directories based on passed arguments.
+ First check for absolute paths, then look in <iwd>/autotests,
+ then glob match.
+ '''
+ tests = []
+ test_root = args.testhome + '/autotests'
+
+ # Run all tests
+ if not args.auto_tests:
+ # --shell with no tests implies 'shell' test
+ if args.shell:
+ return [test_root + '/shell']
+
+ tests = os.listdir(test_root)
+ # Pair down any non-tests and append full path
+ tests = [test_root + '/' + t for t in tests if t.startswith('test')]
+ else:
+ print("Generating partial test list")
+ for t in args.auto_tests.split(','):
+ path = '%s/%s' % (test_root, t)
+ # Full test path specified
+ if os.path.exists(t):
+ tests.append(t)
+ elif os.path.exists(path):
+ tests.append(path)
+ else:
+ matches = glob(path)
+ if matches == []:
+ raise Exception("Could not find test %s" % t)
+
+ tests.extend(matches)
+
+ return sorted(tests)
+
+SimpleResult = namedtuple('SimpleResult', 'run failures errors skipped time')
+
+def start_test(ctx, subtests, rqueue):
+ '''
+ Run an individual test. 'subtests' are parsed prior to calling
+ but these effectively make up a single test. 'rqueue' is the
+ results queue which is required since this is using
+ multiprocessing.
+ '''
+ suite = unittest.TestSuite()
+
+ #
+ # Iterate through each individual python test.
+ #
+ for s in subtests:
+ loader = unittest.TestLoader()
+ subtest = importlib.import_module(os.path.splitext(s)[0])
+ suite.addTests(loader.loadTestsFromModule(subtest))
+
+ # Prevents future test modules with the same name (e.g.
+ # connection_test.py) from being loaded from the cache
+ sys.modules.pop(subtest.__name__)
+
+ start = time.time()
+ runner = unittest.TextTestRunner()
+ result = runner.run(suite)
+ #
+ # The multiprocessing queue is picky with what objects it will serialize
+ # and send between processes. Because of this we put the important bits
+ # of the result into our own 'SimpleResult' tuple.
+ #
+ sresult = SimpleResult(run=result.testsRun, failures=len(result.failures),
+ errors=len(result.errors), skipped=len(result.skipped),
+ time=time.time() - start)
+ rqueue.put(sresult)
+
+ # This may not be required since we are manually popping sys.modules
+ importlib.invalidate_caches()
+
+def pre_test(ctx, test):
+ '''
+ Copy test files, start processes, and any other pre test work.
+ '''
+ os.chdir(test)
+
+ dbg("Starting test %s" % test)
+ if not os.path.exists(test + '/hw.conf'):
+ print("No hw.conf found for %s" % test)
+ exit()
+
+ ctx.hw_config = ConfigParser()
+ ctx.hw_config.read(test + '/hw.conf')
+ #
+ # We have two types of test files: tests and everything else. Rather
+ # than require each test to specify the files needing to be copied to
+ # /tmp (previously 'tmpfs_extra_stuff'), we just copy everything which
+ # isn't a test. There is really no reason not to do this as any file
+ # present in a test directory should be needed by the test.
+ #
+ # All files
+ files = os.listdir(test)
+ # Tests (starts or ends with 'test')
+ subtests = [f for f in files if f.startswith('test') or \
+ os.path.splitext(f)[0].endswith('test')]
+ # Everything else (except .py files)
+ to_copy = [f for f in list(set(files) - set(subtests)) if not f.endswith('.py')]
+ for f in to_copy:
+ if os.path.isdir(f):
+ shutil.copytree(f, '/tmp/' + f)
+ else:
+ shutil.copy(f, '/tmp')
+
+ ctx.start_dbus_monitor()
+ ctx.start_radios()
+ ctx.start_hostapd()
+ ctx.start_ofono()
+
+ if ctx.hw_config.has_option('SETUP', 'start_iwd'):
+ start = ctx.hw_config.getboolean('SETUP', 'start_iwd')
+ else:
+ start = True
+
+ if start:
+ ctx.start_iwd()
+ else:
+ print("Not starting IWD from test-runner")
+
+ print(ctx)
+
+ sys.path.insert(1, test)
+
+ return (to_copy, sorted(subtests))
+
+def post_test(ctx, to_copy):
+ '''
+ Remove copied files, and stop test processes.
+ '''
+ for f in to_copy:
+ if os.path.isdir('/tmp/' + f):
+ shutil.rmtree('/tmp/' + f)
+ else:
+ os.remove('/tmp/' + f)
+
+ Process(['ifconfig', 'lo', 'down'], wait=True)
+
+ ctx.stop_test_processes()
+ if ctx.args.valgrind:
+ with open('/tmp/valgrind.log', 'r') as f:
+ dbg(f.read())
+ dbg("\n")
+
+def print_results(results):
+ table = PrettyTable(['Test', colored('Passed', 'green'), colored('Failed', 'red'), \
+ colored('Skipped', 'cyan'), colored('Time', 'yellow')])
+
+ total_pass = 0
+ total_fail = 0
+ total_skip = 0
+ total_time = 0
+
+ for test, result in results.items():
+ if result.time != TEST_MAX_TIMEOUT:
+ failed = result.failures + result.errors
+ passed = result.run - failed
+
+ total_pass += passed
+ total_fail += failed
+ total_skip += result.skipped
+ else:
+ failed = "Timed out"
+ passed = "Timed out"
+
+ total_time += result.time
+
+ time = '%.2f' % result.time
+
+ table.add_row([test, colored(passed, 'green'), colored(failed, 'red'), \
+ colored(result.skipped, 'cyan'), colored(time, 'yellow')])
+
+ total_time = '%.2f' % total_time
+
+ table.add_row(['Total', colored(total_pass, 'green'), colored(total_fail, 'red'), \
+ colored(total_skip, 'cyan'), colored(total_time, 'yellow')])
+
+ dbg(table)
+
+def run_auto_tests(ctx, args):
+ tests = build_test_list(args)
+
+ ctx.start_dbus()
+ ctx.start_haveged()
+
+ # Copy autotests/misc/{certs,secrets,phonesim} so any test can refer to them
+ shutil.copytree(args.testhome + '/autotests/misc/certs', '/tmp/certs')
+ shutil.copytree(args.testhome + '/autotests/misc/secrets', '/tmp/secrets')
+ shutil.copy(args.testhome + '/autotests/misc/phonesim/phonesim.conf', '/tmp')
+
+ if args.shell:
+ #
+ # Shell really isn't meant to be used with multiple tests. If
+ # a set of tests was passed in just start out in the first.
+ #
+ os.chdir(tests[0])
+ os.system('/bin/bash')
+ exit()
+
+ for test in tests:
+ try:
+ copied, subtests = pre_test(ctx, test)
+
+ rqueue = multiprocessing.Queue()
+ p = multiprocessing.Process(target=start_test, args=(ctx, subtests, rqueue))
+ p.start()
+ # Rather than time each subtest we just time the total but
+ # mutiply the default time by the number of tests being run.
+ p.join(TEST_MAX_TIMEOUT * len(subtests))
+
+ if p.is_alive():
+ # Timeout
+ p.terminate()
+
+ ctx.results[os.path.basename(test)] = SimpleResult(run=0,
+ failures=0, errors=0,
+ skipped=0, time=TEST_MAX_TIMEOUT)
+ else:
+ ctx.results[os.path.basename(test)] = rqueue.get()
+
+ except Exception as ex:
+ print(ex)
+ print("Uncaught exception thrown for %s" % test)
+ ctx.results[os.path.basename(test)] = SimpleResult(run=-1, failures=-1,
+ errors=-1, skipped=-1, time=-1)
+ finally:
+ post_test(ctx, copied)
+
+ # Write out kernel log
+ if ctx.args.log:
+ Process(["dmesg"], ctx=ctx, wait=True)
+
+ print_results(ctx.results)
+
+def run_unit_tests(ctx, args):
+ os.chdir(args.testhome + '/unit')
+ units = build_unit_list(args)
+
+ for u in units:
+ if ctx.start_process([u], wait=True).ret != 0:
+ dbg("Unit test %s failed" % os.path.basename(u))
+ else:
+ dbg("Unit test %s passed" % os.path.basename(u))
+
+def run_tests():
+ global config
+
+ with open('/proc/cmdline', 'r') as f:
+ cmdline = f.read()
+
+ start = cmdline.find('--testhome')
+
+ options = shlex.split(cmdline[start:])
+
+ parser = argparse.ArgumentParser()
+ parser.add_argument('--testhome')
+ parser.add_argument('--auto_tests')
+ parser.add_argument('--unit_tests')
+ parser.add_argument('--verbose', default=[])
+ parser.add_argument('--debug')
+ parser.add_argument('--path')
+ parser.add_argument('--valgrind')
+ parser.add_argument('--gdb')
+ parser.add_argument('--shell')
+ parser.add_argument('--log')
+ parser.add_argument('--log-gid')
+ parser.add_argument('--log-uid')
+ parser.add_argument('--hw')
+
+ args = parser.parse_args(options)
+
+ #
+ # This prevents any print() calls in this script from printing unless
+ # --debug is passed. For an 'always print' option use dbg()
+ #
+ if not args.debug:
+ sys.stdout = open(os.devnull, 'w')
+
+ os.environ['PATH'] = '%s/src' % args.testhome
+ os.environ['PATH'] += ':%s/tools' % args.testhome
+ os.environ['PATH'] += ':%s/client' % args.testhome
+ os.environ['PATH'] += ':' + args.path
+
+ sys.path.append(args.testhome + '/autotests/util')
+
+ #
+ # This allows all autotest utils (iwd/hostapd/etc) to access the
+ # TestContext. Any other module or script (in the same interpreter) can
+ # simply import config.ctx and access all live test information,
+ # start/stop processes, see active radios etc.
+ #
+ config = importlib.import_module('config')
+ config.ctx = TestContext(args)
+
+ if args.log:
+ mount('logdir', args.log, '9p', 0, 'trans=virtio,version=9p2000.L')
+
+ if config.ctx.args.unit_tests is None:
+ run_auto_tests(config.ctx, args)
+ else:
+ run_unit_tests(config.ctx, args)
+
+class Main:
+ def __init__(self):
+ self.parser = argparse.ArgumentParser(
+ description='IWD Test Runner')
+
+ self.parser.add_argument('--qemu', '-q',
+ metavar='<QEMU binary>', type=str,
+ help='QEMU binary to use',
+ dest='qemu')
+ self.parser.add_argument('--kernel', '-k', metavar='<kernel>',
+ type=str,
+ help='Path to kernel image',
+ dest='kernel')
+ self.parser.add_argument('--verbose', '-v', metavar='<list>',
+ type=str,
+ help='Comma separated list of applications',
+ dest='verbose',
+ default=[])
+ self.parser.add_argument('--debug', '-d',
+ action='store_true',
+ help='Enable test-runner debugging',
+ dest='debug')
+ self.parser.add_argument('--shell', '-s', action='store_true',
+ help='Boot into shell', dest='shell')
+ self.parser.add_argument('--log', '-l', type=str,
+ help='Directory for log files')
+ self.parser.add_argument('--hw', '-w', type=str, nargs=1,
+ help='Use physical adapters for tests (passthrough)')
+
+ # Prevent --autotest/--unittest from being used together
+ auto_unit_group = self.parser.add_mutually_exclusive_group()
+ auto_unit_group.add_argument('--auto-tests', '-A',
+ metavar='<tests>', type=str, nargs=1,
+ help='List of tests to run',
+ default=None,
+ dest='auto_tests')
+ auto_unit_group.add_argument('--unit-tests', '-U',
+ metavar='<tests>', type=str, nargs='?',
+ const='*',
+ help='List of unit tests to run',
+ dest='unit_tests')
+
+ # Prevent --valgrind/--gdb from being used together
+ valgrind_gdb_group = self.parser.add_mutually_exclusive_group()
+ valgrind_gdb_group.add_argument('--gdb', '-g', metavar='<exec>',
+ type=str, nargs=1,
+ help='Run gdb on specified executable',
+ dest='gdb')
+ valgrind_gdb_group.add_argument('--valgrind', '-V', action='store_true',
+ help='Run valgrind on IWD', dest='valgrind')
+
+ self.args = self.parser.parse_args()
+
+ if self.args.log and self.args.unit_tests:
+ dbg("Cannot use --log with --unit-tests")
+ quit()
+
+ def start(self):
+ usb_adapters = None
+
+ qemu_table = [
+ 'qemu-system-x86_64',
+ '/usr/bin/qemu-system-x86_64'
+ ]
+
+ kernel_table = [
+ 'bzImage',
+ 'arch/x86/boot/bzImage',
+ 'vmlinux',
+ 'arch/x86/boot/vmlinux'
+ ]
+
+ if self.args.qemu is None:
+ qemu_binary = find_binary(qemu_table)
+ else:
+ if path_exists(self.args.qemu):
+ qemu_binary = self.args.qemu
+ else:
+ print("QEMU binary %s does not exist" % \
+ self.args.qemu)
+ quit()
+
+ if self.args.kernel is None:
+ kernel_binary = find_binary(kernel_table)
+ else:
+ if path_exists(self.args.kernel):
+ kernel_binary = self.args.kernel
+ else:
+ print("Kernel image %s does not exist" % \
+ self.args.kernel)
+ quit()
+
+ if self.args.hw:
+ hw_conf = ConfigParser()
+ hw_conf.read(self.args.hw)
+ # TODO: Parse PCI adapters
+ if hw_conf.has_section('USBAdapters'):
+ # The actual key name of the adapter
+ # doesn't matter since all we need is the
+ # bus/address. This gets named by the kernel
+ # anyways once in the VM.
+ usb_adapters = [v for v in hw_conf['USBAdapters'].values()]
+
+ #
+ # Additional arguments not provided to test-runner which are
+ # needed once booted into the kernel.
+ #
+ options = 'init=%s' % os.path.realpath(sys.argv[0])
+
+ # Support running from top level as well as tools
+ if os.getcwd().endswith('tools'):
+ options += ' --testhome %s/../' % os.getcwd()
+ else:
+ options += ' --testhome %s' % os.getcwd()
+
+ options += ' --path "%s"' % os.environ['PATH']
+
+ if self.args.auto_tests:
+ options += ' --auto_tests %s' % ','.join(self.args.auto_tests)
+
+ if self.args.log:
+ if os.environ.get('SUDO_GID', None) is None:
+ print("--log can only be used as root user")
+ quit()
+
+ self.args.log = os.path.abspath(self.args.log)
+ uid = int(os.environ['SUDO_UID'])
+ gid = int(os.environ['SUDO_GID'])
+
+ if not path_exists(self.args.log):
+ os.mkdir(self.args.log)
+ os.chown(self.args.log, uid, gid)
+
+ options += ' --log-gid %u' % gid
+ options += ' --log-uid %u' % uid
+
+ denylist = [
+ 'auto_tests',
+ 'qemu',
+ 'kernel'
+ ]
+
+ #
+ # This passes through most of the command line options to
+ # the kernel command line. Some are not relevant (e.g. qemu)
+ # so similar options are added in the denylist above. This excludes
+ # any unset options which are assumed to be None or False. This
+ # is done so default arguments can be filled once in the VM. If
+ # we pass and basic types (None, False etc.) they are turned into
+ # a string representation ('None', 'False', etc.) which is not
+ # desirable.
+ #
+ for arg in vars(self.args):
+ if arg in denylist or getattr(self.args, arg) in [None, False, []]:
+ continue
+ options += ' --%s %s' % (arg, str(getattr(self.args, arg)))
+
+ kern_log = "ignore_loglevel" if "kernel" in self.args.verbose else "quiet"
+
+ qemu_cmdline = [
+ qemu_binary,
+ '-machine', 'type=q35,accel=kvm:tcg',
+ '-nodefaults', '-no-user-config', '-monitor', 'none',
+ '-display', 'none', '-m', '192M', '-nographic', '-vga',
+ 'none', '-net', 'none', '-no-acpi', '-no-hpet',
+ '-no-reboot', '-fsdev',
+ 'local,id=fsdev-root,path=/,readonly,security_model=none,multidevs=remap',
+ '-device',
+ 'virtio-9p-pci,fsdev=fsdev-root,mount_tag=/dev/root',
+ '-chardev', 'stdio,id=chardev-serial0,signal=off',
+ '-device', 'pci-serial,chardev=chardev-serial0',
+ '-device', 'virtio-rng-pci',
+ '-kernel',
+ kernel_binary,
+ '-append',
+ 'console=ttyS0,115200n8 earlyprintk=serial \
+ rootfstype=9p root=/dev/root \
+ rootflags=trans=virtio,version=9p2000.u \
+ acpi=off pci=noacpi %s ro \
+ mac80211_hwsim.radios=0 %s' % (kern_log, options),
+ '-cpu', 'host'
+ ]
+
+ if usb_adapters:
+ for bus, addr in [s.split(',') for s in usb_adapters]:
+ qemu_cmdline.extend(['-usb',
+ '-device',
+ 'usb-host,hostbus=%s,hostaddr=%s' % \
+ (bus, addr)])
+ if self.args.log:
+ #
+ # Creates a virtfs device that can be mounted. This mount
+ # will point back to the provided log directory and is
+ # writable unlike the rest of the mounted file system.
+ #
+ qemu_cmdline.extend([
+ '-virtfs',
+ 'local,path=%s,mount_tag=logdir,security_model=passthrough,id=logdir' \
+ % self.args.log
+ ])
+
+ os.execlp(qemu_cmdline[0], *qemu_cmdline)
+
+if __name__ == '__main__':
+ if os.getpid() == 1 and os.getppid() == 0:
+ atexit.register(exit_vm)
+ prepare_sandbox()
+ run_tests()
+
+ exit()
+
+ main = Main()
+ main.start()
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 03/26] auto-t: hostapd.py: update to work with test-runner rewrite
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
2020-09-10 23:12 ` [PATCH v5 02/26] auto-t: introduce pure python " James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 04/26] auto-t: testutil.py: " James Prestwood
` (23 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 8731 bytes --]
Before hostapd was initialized using the wiphy_map which has now
gone away. Instead we have a global config module which contains
a single 'ctx'. This is the centeral store for all test information.
This patch converts hostapd.py to lookup instances by already
initialized Hostapd object. The interface parameter was removed
since all tests have been converted to use config= instead.
In addition HostapdCLI was changed to allow no parameters if there
is only a single hostapd instance.
---
autotests/util/hostapd.py | 100 +++++++++++++++++---------------------
1 file changed, 44 insertions(+), 56 deletions(-)
diff --git a/autotests/util/hostapd.py b/autotests/util/hostapd.py
index 0ed4e004..3f0edd9e 100644
--- a/autotests/util/hostapd.py
+++ b/autotests/util/hostapd.py
@@ -1,11 +1,11 @@
#!/usr/bin/python3
import os, os.path
-from wiphy import wiphy_map
import re
import socket
import select
import time
from gi.repository import GLib
+from config import ctx
chan_freq_map = [
None,
@@ -28,31 +28,26 @@ chan_freq_map = [
ctrl_count = 0
mainloop = GLib.MainLoop()
-hostapd_map = {ifname: intf for wname, wiphy in wiphy_map.items()
- for ifname, intf in wiphy.interface_map.items()
- if wiphy.use == 'hostapd'}
-
class HostapdCLI:
- def _init_hostapd(self, interface=None, config=None):
+ def _init_hostapd(self, config=None):
global ctrl_count
+ interface = None
+
+ if not config and len(ctx.hostapd.instances) > 1:
+ raise Exception('config must be provided if more than one hostapd instance exists')
- if not interface and not config:
- raise Exception('interface or config must be provided')
+ hapd = ctx.hostapd[config]
- if not interface:
- for intf in hostapd_map.values():
- if intf.config == config:
- interface = intf
- break
+ self.interface = hapd.intf
+ self.config = hapd.config
- if not interface:
+ if not self.interface:
raise Exception('config %s not found' % config)
- self.ifname = interface.name
- self.socket_path = os.path.dirname(interface.ctrl_interface)
+ self.ifname = self.interface.name
+ self.socket_path = os.path.dirname(self.interface.ctrl_interface)
- self.cmdline = 'hostapd_cli -p"' + self.socket_path + '" -i"' + \
- self.ifname + '"'
+ self.cmdline = ['hostapd_cli', '-p', self.socket_path, '-i', self.ifname]
if not hasattr(self, '_hostapd_restarted'):
self._hostapd_restarted = False
@@ -61,6 +56,7 @@ class HostapdCLI:
str(ctrl_count)
self.ctrl_sock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
self.ctrl_sock.bind(self.local_ctrl)
+
self.ctrl_sock.connect(self.socket_path + '/' + self.ifname)
if 'OK' not in self._ctrl_request('ATTACH'):
@@ -68,8 +64,8 @@ class HostapdCLI:
ctrl_count = ctrl_count + 1
- def __init__(self, interface=None, config=None):
- self._init_hostapd(interface, config)
+ def __init__(self, config=None):
+ self._init_hostapd(config)
def wait_for_event(self, event, timeout=10):
global mainloop
@@ -115,53 +111,49 @@ class HostapdCLI:
def _del_hostapd(self, force=False):
self.ctrl_sock.close()
+ os.remove(self.local_ctrl)
if self._hostapd_restarted:
- if force:
- os.system('killall -9 hostapd')
- else:
- os.system('killall hostapd')
+ ctx.stop_process(ctx.hostapd.process, force)
- os.system('ifconfig %s down' % self.ifname)
- os.system('ifconfig %s up' % self.ifname)
+ self.interface.set_interface_state('down')
+ self.interface.set_interface_state('up')
def __del__(self):
self._del_hostapd()
def wps_push_button(self):
- os.system(self.cmdline + ' wps_pbc')
+ ctx.start_process(self.cmdline + ['wps_pbc'], wait=True)
def wps_pin(self, pin):
- os.system(self.cmdline + ' wps_pin any ' + pin)
+ cmd = self.cmdline + ['wps_pin', 'any', pin]
+ ctx.start_process(cmd, wait=True)
def deauthenticate(self, client_address):
- os.system(self.cmdline + ' deauthenticate ' + client_address)
+ cmd = self.cmdline + ['deauthenticate', client_address]
+ ctx.start_process(cmd, wait=True)
def eapol_reauth(self, client_address):
cmd = 'IFNAME=' + self.ifname + ' EAPOL_REAUTH ' + client_address
- s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
- s.connect(self.socket_path + '/' + self.ifname)
- s.sendall(cmd.encode('utf-8'))
- s.close()
+ self.ctrl_sock.sendall(cmd.encode('utf-8'))
def reload(self):
# Seemingly all three commands needed for the instance to notice
# interface's address change
- cmds = 'reload\ndisable\nenable\n'
- proc = os.popen(self.cmdline, mode='w')
- lines = proc.write(cmds)
- proc.close()
+ ctx.start_process(self.cmdline + ['reload'], wait=True)
+ ctx.start_process(self.cmdline + ['disable'], wait=True)
+ ctx.start_process(self.cmdline + ['enable'], wait=True)
def list_sta(self):
- proc = os.popen(self.cmdline + ' list_sta')
- lines = proc.read()
- proc.close()
+ proc = ctx.start_process(self.cmdline + ['list_sta'])
+ proc.pid.wait()
+ lines = proc.pid.stdout.read().decode('utf-8')
return [line for line in lines.split('\n') if line]
def set_neighbor(self, addr, ssid, nr):
- os.system(self.cmdline + ' set_neighbor ' + addr + ' ssid=\\""' + ssid +
- '"\\" nr=' + nr)
+ cmd = self.cmdline + ['set_neighbor', addr, 'ssid=\\""%s"\\"' % ssid, 'nr=%s' % nr]
+ ctx.start_process(cmd, wait=True)
def send_bss_transition(self, device, nr_list):
# Send a BSS transition to a station (device). nr_list should be an
@@ -170,7 +162,7 @@ class HostapdCLI:
# consistent with the set_neighbor() API, i.e. the same neighbor report
# string could be used in both API's.
pref = 1
- cmd = self.cmdline + ' bss_tm_req ' + device
+ cmd = self.cmdline + ['bss_tm_req', device]
for i in nr_list:
addr = i[0]
nr = i[1]
@@ -180,19 +172,15 @@ class HostapdCLI:
chan_num=nr[10:12]
phy_num=nr[14:16]
- cmd += ' pref=%s neighbor=%s,%s,%s,%s,%s' % \
- (str(pref), addr, bss_info, op_class, chan_num, phy_num)
+ cmd += ['pref=%s' % str(pref), 'neighbor=%s,%s,%s,%s,%s' % \
+ (addr, bss_info, op_class, chan_num, phy_num)]
pref += 1
- os.system(cmd)
-
- @staticmethod
- def kill_all():
- os.system('killall hostapd')
+ ctx.start_process(cmd, wait=True)
def get_config_value(self, key):
# first find the right config file
- with open(hostapd_map[self.ifname].config, 'r') as f:
+ with open(self.config, 'r') as f:
# read in config file and search for key
cfg = f.read()
match = re.search(r'%s=.*' % key, cfg)
@@ -200,6 +188,7 @@ class HostapdCLI:
return match.group(0).split('=')[1]
return None
+
def get_freq(self):
return chan_freq_map[int(self.get_config_value('channel'))]
@@ -209,21 +198,20 @@ class HostapdCLI:
'''
# set flag so hostapd can be killed after the test
self._hostapd_restarted = True
- intf = hostapd_map[self.ifname]
self._del_hostapd(force=True)
- os.system('hostapd -g %s -i %s %s &' %
- (intf.ctrl_interface, intf.name, intf.config))
+ ctx.start_hostapd()
# Give hostapd a second to start and initialize the control interface
time.sleep(1)
# New hostapd process, so re-init
- self._init_hostapd(intf)
+ self._init_hostapd(config=self.config)
def req_beacon(self, addr, request):
'''
Send a RRM Beacon request
'''
- os.system(self.cmdline + ' req_beacon ' + addr + ' ' + request)
+ cmd = self.cmdline + ['req_beacon', addr, request]
+ ctx.start_process(cmd, wait=True)
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 04/26] auto-t: testutil.py: update to work with test-runner rewrite
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
2020-09-10 23:12 ` [PATCH v5 02/26] auto-t: introduce pure python " James Prestwood
2020-09-10 23:12 ` [PATCH v5 03/26] auto-t: hostapd.py: update to work with test-runner rewrite James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 05/26] auto-t: ofono.py: fix timeout cleanup and wait for service James Prestwood
` (22 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1142 bytes --]
Similar to the others, testutil.py was converted to use the global
test context to lookup radios/interfaces rather than the wiphy_map
---
autotests/util/testutil.py | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/autotests/util/testutil.py b/autotests/util/testutil.py
index aaa4e7e1..0a94e23a 100644
--- a/autotests/util/testutil.py
+++ b/autotests/util/testutil.py
@@ -5,8 +5,8 @@ import fcntl
import struct
import select
-import wiphy
import iwd
+from config import ctx
HWSIM_ETHERTYPE = 0x0800
HWSIM_PACKETLEN = 250
@@ -54,8 +54,9 @@ def tx(fromsock, tosock, src, dst):
def test_connected(if0=None, if1=None, group=True):
if if0 is None or if1 is None:
iwd_list = [dev.name for dev in iwd.IWD.get_instance().list_devices()]
- non_iwd_list = [ifname for w in wiphy.wiphy_map.values()
- for ifname in w.interface_map]
+
+ non_iwd_list = [rad.interface.name for rad in ctx.radios if rad.interface is not None]
+
for intf in iwd_list + non_iwd_list:
if if0 is None:
if0 = intf
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 05/26] auto-t: ofono.py: fix timeout cleanup and wait for service
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (2 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 04/26] auto-t: testutil.py: " James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 06/26] auto-t: iwd.py: update to work with test-runner rewrite James Prestwood
` (21 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1929 bytes --]
Ofono.py was not cleaning up the timeout, nor waiting for the ofono.org
service to come up before continuing.
---
autotests/util/ofono.py | 25 ++++++++++++++++++-------
1 file changed, 18 insertions(+), 7 deletions(-)
diff --git a/autotests/util/ofono.py b/autotests/util/ofono.py
index 2bc6338b..f1ed14ee 100644
--- a/autotests/util/ofono.py
+++ b/autotests/util/ofono.py
@@ -1,4 +1,5 @@
import dbus
+import time
from gi.repository import GLib
SIM_AUTH_IFACE = 'org.ofono.SimAuthentication'
@@ -7,6 +8,14 @@ class Ofono(dbus.service.Object):
def __init__(self):
self._bus = dbus.SystemBus()
+ tries = 0
+
+ while not self._bus.name_has_owner('org.ofono'):
+ if tries > 100:
+ raise TimeoutError('Waiting for org.ofono service timed out')
+ tries += 1
+ time.sleep(0.1)
+
def enable_modem(self, path):
self._modem_path = path
self._modem_iface = dbus.Interface(
@@ -37,11 +46,13 @@ class Ofono(dbus.service.Object):
self._wait_timed_out = True
return False
- timeout = GLib.timeout_add_seconds(max_wait, wait_timeout_cb)
- context = mainloop.get_context()
- while (not self._sim_auth_up):
- context.iteration(may_block=True)
- if self._wait_timed_out:
- raise TimeoutError('waiting for SimAuthentication timed out')
+ try:
+ timeout = GLib.timeout_add_seconds(max_wait, wait_timeout_cb)
+ context = mainloop.get_context()
+ while (not self._sim_auth_up):
+ context.iteration(may_block=True)
+ if self._wait_timed_out:
+ raise TimeoutError('waiting for SimAuthentication timed out')
- GLib.source_remove(timeout)
+ finally:
+ GLib.source_remove(timeout)
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 06/26] auto-t: iwd.py: update to work with test-runner rewrite
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (3 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 05/26] auto-t: ofono.py: fix timeout cleanup and wait for service James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 07/26] auto-t: remove wiphy.py James Prestwood
` (20 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 9647 bytes --]
iwd.py was updated to use the TestContext APIs to start/stop
IWD. This makes the process managment consistent between starting
IWD from test-runner or from the IWD() constructor.
The psk agent is now tracked, and destroyed upon __del__. This is
to fix issues where a test throws an exception and never
unregisters the agent, causing future tests to fail.
The configuration directory was also chaged to /tmp by
default. This was done since all tests which used this used /tmp
anyways.
The GLib mainloop was removed, and instead put into test-runner
itself. Now any mainloop operations can use ctx.mainloop instead
---
autotests/util/iwd.py | 123 +++++++++++++++++++-----------------------
1 file changed, 56 insertions(+), 67 deletions(-)
diff --git a/autotests/util/iwd.py b/autotests/util/iwd.py
index ef1ea80e..2041e2d6 100755
--- a/autotests/util/iwd.py
+++ b/autotests/util/iwd.py
@@ -15,7 +15,7 @@ import weakref
from abc import ABCMeta, abstractmethod
from enum import Enum
-import wiphy
+from config import ctx
IWD_STORAGE_DIR = '/var/lib/iwd'
IWD_CONFIG_DIR = '/etc/iwd'
@@ -39,10 +39,6 @@ IWD_STATION_INTERFACE = 'net.connman.iwd.Station'
IWD_AGENT_MANAGER_PATH = '/net/connman/iwd'
IWD_TOP_LEVEL_PATH = '/'
-
-dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
-
-
class UnknownDBusEx(Exception): pass
class InProgressEx(dbus.DBusException): pass
class FailedEx(dbus.DBusException): pass
@@ -108,7 +104,7 @@ class AsyncOpAbstract(object):
self._exception = _convert_dbus_ex(ex)
def _wait_for_async_op(self):
- context = mainloop.get_context()
+ context = ctx.mainloop.get_context()
while not self._is_completed:
context.iteration(may_block=True)
@@ -533,8 +529,8 @@ class Device(IWDDBusAbstract):
if addr not in props['ConnectedPeers']:
return
- GLib.timeout_add(int(30 * 1000), wait_timeout_cb)
- context = mainloop.get_context()
+ GLib.timeout_add(int(50 * 1000), wait_timeout_cb)
+ context = ctx.mainloop.get_context()
while not self._adhoc_prop_found:
context.iteration(may_block=True)
if self._adhoc_timed_out:
@@ -561,8 +557,8 @@ class Device(IWDDBusAbstract):
if addr in props['ConnectedPeers']:
return
- GLib.timeout_add(int(15 * 1000), wait_timeout_cb)
- context = mainloop.get_context()
+ GLib.timeout_add(int(50 * 1000), wait_timeout_cb)
+ context = ctx.mainloop.get_context()
while not self._adhoc_prop_found:
context.iteration(may_block=True)
if self._adhoc_timed_out:
@@ -873,7 +869,11 @@ class DeviceList(collections.Mapping):
class IWD(AsyncOpAbstract):
- ''''''
+ '''
+ Start an IWD instance. By default IWD should already be running, but
+ some tests do require starting IWD using this constructor (by passing
+ start_iwd_daemon=True)
+ '''
_bus = dbus.SystemBus()
_object_manager_if = None
@@ -881,41 +881,22 @@ class IWD(AsyncOpAbstract):
_iwd_proc = None
_devices = None
_instance = None
+ psk_agent = None
- def __init__(self, start_iwd_daemon = False,
- iwd_config_dir = IWD_CONFIG_DIR):
- global mainloop
- mainloop = GLib.MainLoop()
+ def __init__(self, start_iwd_daemon = False, iwd_config_dir = '/tmp'):
+ if start_iwd_daemon and ctx.is_process_running('iwd'):
+ raise Exception("IWD requested to start but is already running")
if start_iwd_daemon:
- args = []
- iwd_wiphys = [wname for wname, wiphy in wiphy.wiphy_map.items()
- if wiphy.use == 'iwd']
- whitelist = ','.join(iwd_wiphys)
-
- if os.environ.get('IWD_TEST_VALGRIND', None) == 'on':
- args.append('valgrind')
- args.append('--leak-check=full')
- args.append('--log-file=/tmp/valgrind.log')
-
- os.environ["CONFIGURATION_DIRECTORY"] = iwd_config_dir;
-
- args.append('iwd')
- args.append('-p')
- args.append(whitelist)
- args.append('-d')
-
- import subprocess
- iwd_proc = subprocess.Popen(args)
-
- self._iwd_proc = iwd_proc
+ self._iwd_proc = ctx.start_iwd(iwd_config_dir)
tries = 0
while not self._bus.name_has_owner(IWD_SERVICE):
- if os.environ['IWD_TEST_TIMEOUTS'] == 'on':
+ if ctx.args.gdb == 'None':
if tries > 100:
if start_iwd_daemon:
- iwd_proc.terminate()
+ ctx.stop_process(self._iwd_proc)
+ self._iwd_proc = None
raise TimeoutError('IWD has failed to start')
tries += 1
time.sleep(0.1)
@@ -927,20 +908,17 @@ class IWD(AsyncOpAbstract):
IWD._instance = weakref.ref(self)
def __del__(self):
- if self._iwd_proc is None:
- return
+ if self.psk_agent:
+ self.unregister_psk_agent(self.psk_agent)
self._object_manager_if = None
self._agent_manager_if = None
self._known_networks = None
self._devices = None
- self._iwd_proc.terminate()
- self._iwd_proc.wait()
-
- self._iwd_proc = None
-
- del os.environ["CONFIGURATION_DIRECTORY"]
+ if self._iwd_proc is not None:
+ ctx.stop_process(self._iwd_proc)
+ self._iwd_proc = None
@property
def _object_manager(self):
@@ -966,15 +944,18 @@ class IWD(AsyncOpAbstract):
self._wait_timed_out = True
return False
- timeout = GLib.timeout_add_seconds(max_wait, wait_timeout_cb)
- context = mainloop.get_context()
- while not eval(condition_str):
- context.iteration(may_block=True)
- if self._wait_timed_out and os.environ['IWD_TEST_TIMEOUTS'] == 'on':
- raise TimeoutError('[' + condition_str + ']'\
- ' condition was not met in '\
- + str(max_wait) + ' sec')
- GLib.source_remove(timeout)
+ try:
+ timeout = GLib.timeout_add_seconds(max_wait, wait_timeout_cb)
+ context = ctx.mainloop.get_context()
+ while not eval(condition_str):
+ context.iteration(may_block=True)
+ if self._wait_timed_out and ctx.args.gdb == None:
+ raise TimeoutError('[' + condition_str + ']'\
+ ' condition was not met in '\
+ + str(max_wait) + ' sec')
+ finally:
+ if not self._wait_timed_out:
+ GLib.source_remove(timeout)
def wait(self, time):
self._wait_timed_out = False
@@ -982,10 +963,14 @@ class IWD(AsyncOpAbstract):
self._wait_timed_out = True
return False
- GLib.timeout_add(int(time * 1000), wait_timeout_cb)
- context = mainloop.get_context()
- while not self._wait_timed_out:
- context.iteration(may_block=True)
+ try:
+ timeout = GLib.timeout_add(int(time * 1000), wait_timeout_cb)
+ context = ctx.mainloop.get_context()
+ while not self._wait_timed_out:
+ context.iteration(may_block=True)
+ finally:
+ if not self._wait_timed_out:
+ GLib.source_remove(timeout)
@staticmethod
def clear_storage():
@@ -1031,14 +1016,16 @@ class IWD(AsyncOpAbstract):
self._wait_timed_out = True
return False
- timeout = GLib.timeout_add_seconds(max_wait, wait_timeout_cb)
- context = mainloop.get_context()
- while len(self._devices) < wait_to_appear:
- context.iteration(may_block=True)
- if self._wait_timed_out:
- raise TimeoutError('IWD has no associated devices')
-
- GLib.source_remove(timeout)
+ try:
+ timeout = GLib.timeout_add_seconds(max_wait, wait_timeout_cb)
+ context = ctx.mainloop.get_context()
+ while len(self._devices) < wait_to_appear:
+ context.iteration(may_block=True)
+ if self._wait_timed_out:
+ raise TimeoutError('IWD has no associated devices')
+ finally:
+ if not self._wait_timed_out:
+ GLib.source_remove(timeout)
return list(self._devices.values())
@@ -1062,6 +1049,7 @@ class IWD(AsyncOpAbstract):
reply_handler=self._success,
error_handler=self._failure)
self._wait_for_async_op()
+ self.psk_agent = psk_agent
def unregister_psk_agent(self, psk_agent):
self._agent_manager.UnregisterAgent(
@@ -1070,6 +1058,7 @@ class IWD(AsyncOpAbstract):
reply_handler=self._success,
error_handler=self._failure)
self._wait_for_async_op()
+ self.psk_agent = None
@staticmethod
def get_instance():
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 07/26] auto-t: remove wiphy.py
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (4 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 06/26] auto-t: iwd.py: update to work with test-runner rewrite James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 08/26] auto-t: fix hidden network test James Prestwood
` (19 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1296 bytes --]
This is no longer needed
---
autotests/util/wiphy.py | 32 --------------------------------
1 file changed, 32 deletions(-)
delete mode 100644 autotests/util/wiphy.py
diff --git a/autotests/util/wiphy.py b/autotests/util/wiphy.py
deleted file mode 100644
index 3f94d87b..00000000
--- a/autotests/util/wiphy.py
+++ /dev/null
@@ -1,32 +0,0 @@
-#! /usr/bin/python3
-import os
-import collections
-
-wiphy_map = {}
-
-Wiphy = collections.namedtuple('Wiphy', ['name', 'use', 'interface_map'])
-
-Intf = collections.namedtuple('Intf',
- ['name', 'wiphy', 'ctrl_interface', 'config'])
-
-def parse_list():
- for entry in os.environ['TEST_WIPHY_LIST'].split('\n'):
- wname, use_str = entry.split('=', 1)
- use = use_str.split(',')
-
- if wname not in wiphy_map:
- wiphy_map[wname] = Wiphy(use=use[0], name=wname, interface_map={})
-
- if len(use) <= 1:
- continue
-
- intf = {}
- intf['name'] = None
- intf['wiphy'] = wiphy_map[wname]
- intf['ctrl_interface'] = None
- intf['config'] = None
- intf.update(dict([param.split('=', 1) for param in use[1:]]))
-
- wiphy_map[wname].interface_map[intf['name']] = Intf(**intf)
-
-parse_list()
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 08/26] auto-t: fix hidden network test
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (5 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 07/26] auto-t: remove wiphy.py James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 09/26] auto-t: fix testSAE autoconnect_test.py James Prestwood
` (18 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 2982 bytes --]
This test was never 100% reliable, and after the test-runner re-write
it became extremely unreliable. The issue came down to the very common
block of code thats present in many tests where we wait for obj.scanning
then not obj.scanning. This is fine when a dbus scan() is explicitly
done before, otherwise it could lead to problems. Without a dbus scan
explicitly called we are assuming a periodic scan will happen. If it
already happen the initial wait for obj.scanning will never return and
time out.
This probably needs to be changed in several tests, but for this specific
case we can remove the waits completely. Since
check_autoconnect_hidden_network has a 30 second wait on
DeviceState.connected this will ultimately time out if anything goes
wrong. There isn't any great reason to wait for scanning (for this test
specifically).
A minor style change was also made when initializing IWD. The values
passed in this test are now the default, so no arguments need to be
passed.
---
.../connect_after_hidden_connect_test.py | 2 +-
autotests/testHiddenNetworks/validation.py | 8 +-------
2 files changed, 2 insertions(+), 8 deletions(-)
diff --git a/autotests/testHiddenNetworks/connect_after_hidden_connect_test.py b/autotests/testHiddenNetworks/connect_after_hidden_connect_test.py
index 478f1505..d9851c6a 100644
--- a/autotests/testHiddenNetworks/connect_after_hidden_connect_test.py
+++ b/autotests/testHiddenNetworks/connect_after_hidden_connect_test.py
@@ -27,7 +27,7 @@ class TestConnectionAfterHiddenNetwork(unittest.TestCase):
self._ex = ex
def test_connection(self):
- wd = IWD(True, '/tmp')
+ wd = IWD(True)
psk_agent = PSKAgent("secret123")
wd.register_psk_agent(psk_agent)
diff --git a/autotests/testHiddenNetworks/validation.py b/autotests/testHiddenNetworks/validation.py
index 9c433623..780af8b3 100644
--- a/autotests/testHiddenNetworks/validation.py
+++ b/autotests/testHiddenNetworks/validation.py
@@ -50,11 +50,6 @@ class TestConnectAutoConnect(unittest.TestCase):
device = devices[0]
if autoconnect:
- condition = 'obj.scanning'
- wd.wait_for_object_condition(device, condition)
- condition = 'not obj.scanning'
- wd.wait_for_object_condition(device, condition)
-
self.check_autoconnect_hidden_network(wd, device, ssid, throws)
else:
if wait_periodic_scan:
@@ -71,7 +66,6 @@ class TestConnectAutoConnect(unittest.TestCase):
def validate(self, ssid, autoconnect, throws = None, use_agent = False,
wait_periodic_scan = False):
- wd = IWD(True, '/tmp')
-
+ wd = IWD(True)
self.validate_connection(wd, ssid, autoconnect, throws, use_agent,
wait_periodic_scan)
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 09/26] auto-t: fix testSAE autoconnect_test.py
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (6 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 08/26] auto-t: fix hidden network test James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 10/26] auto-t: skip ofono tests if ofonod isn't running James Prestwood
` (17 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1499 bytes --]
This test was unreliable since it was assuming a periodic scan would
happen at just the right time. Instead since we are expecting autoconnect
we can just wait for DeviceState.connected then after we are connected
verify the network was correct.
---
autotests/testSAE/autoconnect_test.py | 13 ++++---------
1 file changed, 4 insertions(+), 9 deletions(-)
diff --git a/autotests/testSAE/autoconnect_test.py b/autotests/testSAE/autoconnect_test.py
index a45fd09d..0613d91e 100644
--- a/autotests/testSAE/autoconnect_test.py
+++ b/autotests/testSAE/autoconnect_test.py
@@ -23,20 +23,15 @@ class Test(unittest.TestCase):
devices[2].disconnect()
devices[3].disconnect()
- condition = 'obj.scanning'
- wd.wait_for_object_condition(device, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition, 30)
- condition = 'not obj.scanning'
+ condition = 'obj.connected_network is not None'
wd.wait_for_object_condition(device, condition)
ordered_network = device.get_ordered_network('ssidSAE')
- self.assertEqual(ordered_network.type, NetworkType.psk)
-
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
-
- device.wait_for_connected()
+ self.assertTrue(ordered_network.network_object.connected)
device.disconnect()
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 10/26] auto-t: skip ofono tests if ofonod isn't running
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (7 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 09/26] auto-t: fix testSAE autoconnect_test.py James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 11/26] auto-t: replace hard-coded interfaces James Prestwood
` (16 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 2448 bytes --]
---
autotests/testEAP-AKA-ofono/connection_test.py | 4 ++++
autotests/testEAP-AKA-prime-ofono/connection_test.py | 4 ++++
autotests/testEAP-SIM-ofono/connection_test.py | 4 ++++
3 files changed, 12 insertions(+)
diff --git a/autotests/testEAP-AKA-ofono/connection_test.py b/autotests/testEAP-AKA-ofono/connection_test.py
index 94066f76..27f736f6 100644
--- a/autotests/testEAP-AKA-ofono/connection_test.py
+++ b/autotests/testEAP-AKA-ofono/connection_test.py
@@ -9,6 +9,7 @@ from iwd import IWD
from iwd import NetworkType
from hlrauc import AuthCenter
from ofono import Ofono
+from config import ctx
class Test(unittest.TestCase):
@@ -56,6 +57,9 @@ class Test(unittest.TestCase):
auth.stop()
@classmethod
def setUpClass(cls):
+ if not ctx.is_process_running('ofonod'):
+ cls.skipTest(cls, "ofono not running")
+
IWD.copy_to_storage('ssidEAP-AKA.8021x')
@classmethod
diff --git a/autotests/testEAP-AKA-prime-ofono/connection_test.py b/autotests/testEAP-AKA-prime-ofono/connection_test.py
index 2ff99894..967ba352 100644
--- a/autotests/testEAP-AKA-prime-ofono/connection_test.py
+++ b/autotests/testEAP-AKA-prime-ofono/connection_test.py
@@ -9,6 +9,7 @@ from iwd import IWD
from iwd import NetworkType
from hlrauc import AuthCenter
from ofono import Ofono
+from config import ctx
class Test(unittest.TestCase):
@@ -57,6 +58,9 @@ class Test(unittest.TestCase):
@classmethod
def setUpClass(cls):
+ if not ctx.is_process_running('ofonod'):
+ cls.skipTest(cls, "ofono not running")
+
IWD.copy_to_storage('ssidEAP-AKA.8021x')
@classmethod
diff --git a/autotests/testEAP-SIM-ofono/connection_test.py b/autotests/testEAP-SIM-ofono/connection_test.py
index 95242943..de7c4b48 100644
--- a/autotests/testEAP-SIM-ofono/connection_test.py
+++ b/autotests/testEAP-SIM-ofono/connection_test.py
@@ -9,6 +9,7 @@ from iwd import IWD
from iwd import NetworkType
from hlrauc import AuthCenter
from ofono import Ofono
+from config import ctx
class Test(unittest.TestCase):
@@ -57,6 +58,9 @@ class Test(unittest.TestCase):
@classmethod
def setUpClass(cls):
+ if not ctx.is_process_running('ofonod'):
+ cls.skipTest(cls, "ofono not running")
+
IWD.copy_to_storage('ssidEAP-SIM.8021x')
@classmethod
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 11/26] auto-t: replace hard-coded interfaces
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (8 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 10/26] auto-t: skip ofono tests if ofonod isn't running James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 12/26] auto-t: remove device.wait_for_connected James Prestwood
` (15 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1987 bytes --]
test-runner now supports interface name replacement inside hostapd
config files. Since a given test configuration doesn't know what
interface names there will be $ifaceN can be specified instead e.g.
rsn_preauth_interfaces=$iface0 $iface1
The $ifaceN values will be replace with actual interface names when
the test is started.
This patch also removes ctrl_interface inside the hostapd config
files as this is no longer required.
---
autotests/testPreauth-roam/eaptls-preauth-1.conf | 3 +--
autotests/testPreauth-roam/eaptls-preauth-2.conf | 3 +--
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/autotests/testPreauth-roam/eaptls-preauth-1.conf b/autotests/testPreauth-roam/eaptls-preauth-1.conf
index 0c7f96a7..bc808ed7 100644
--- a/autotests/testPreauth-roam/eaptls-preauth-1.conf
+++ b/autotests/testPreauth-roam/eaptls-preauth-1.conf
@@ -2,7 +2,6 @@ hw_mode=g
channel=1
ssid=TestPreauth
utf8_ssid=1
-ctrl_interface=/var/run/hostapd
wpa=2
wpa_key_mgmt=WPA-EAP
@@ -24,7 +23,7 @@ radius_server_auth_port=1812
nas_identifier=testeap1
rsn_preauth=1
-rsn_preauth_interfaces=wln0 wln1
+rsn_preauth_interfaces=$iface0 $iface1
disable_pmksa_caching=0
# Allow PMK cache to be shared opportunistically among configured interfaces
diff --git a/autotests/testPreauth-roam/eaptls-preauth-2.conf b/autotests/testPreauth-roam/eaptls-preauth-2.conf
index bddcca69..6d3a18e7 100644
--- a/autotests/testPreauth-roam/eaptls-preauth-2.conf
+++ b/autotests/testPreauth-roam/eaptls-preauth-2.conf
@@ -2,7 +2,6 @@ hw_mode=g
channel=2
ssid=TestPreauth
utf8_ssid=1
-ctrl_interface=/var/run/hostapd
wpa=2
wpa_key_mgmt=WPA-EAP
@@ -20,7 +19,7 @@ auth_server_port=1812
auth_server_shared_secret=secret
rsn_preauth=1
-rsn_preauth_interfaces=wln0 wln1
+rsn_preauth_interfaces=$iface0 $iface1
disable_pmksa_caching=0
# Allow PMK cache to be shared opportunistically among configured interfaces
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 12/26] auto-t: remove device.wait_for_connected
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (9 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 11/26] auto-t: replace hard-coded interfaces James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 13/26] tools: post test-runner rewrite cleanup James Prestwood
` (14 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 2328 bytes --]
I had added this without realizing wait_for_object_condition
can do this already using the device object.
---
autotests/testSAE/clogging_test.py | 3 ++-
autotests/util/iwd.py | 25 -------------------------
2 files changed, 2 insertions(+), 26 deletions(-)
diff --git a/autotests/testSAE/clogging_test.py b/autotests/testSAE/clogging_test.py
index e94e2ad6..c46c5821 100644
--- a/autotests/testSAE/clogging_test.py
+++ b/autotests/testSAE/clogging_test.py
@@ -48,7 +48,8 @@ class Test(unittest.TestCase):
wd.wait_for_object_condition(n.network_object, condition)
for d in devices:
- d.wait_for_connected()
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(d, condition)
for d in devices:
d.disconnect()
diff --git a/autotests/util/iwd.py b/autotests/util/iwd.py
index 2041e2d6..ec5e662f 100755
--- a/autotests/util/iwd.py
+++ b/autotests/util/iwd.py
@@ -564,31 +564,6 @@ class Device(IWDDBusAbstract):
if self._adhoc_timed_out:
raise TimeoutError("Timed out waiting for peer %s" % addr)
- def wait_for_connected(self):
- if str(self.state) == "connected":
- return
-
- self._connected_success = False
- self._connected_timed_out = False
-
- def wait_timeout_cb():
- self._connected_timed_out = True
- return False
-
- def connected_prop_changed(iface, changed, invalid):
- if changed.get('State', None):
- if changed['State'] == 'connected':
- self._connected_success = True
-
- self._prop_proxy.connect_to_signal('PropertiesChanged',
- connected_prop_changed)
- GLib.timeout_add(int(15 * 1000), wait_timeout_cb)
- context = mainloop.get_context()
- while not self._connected_success:
- context.iteration(may_block=True)
- if self._connected_timed_out:
- raise TimeoutError("Timed out waiting for connected")
-
def __str__(self, prefix = ''):
return prefix + 'Device: ' + self.device_path + '\n'\
+ prefix + '\tName:\t\t' + self.name + '\n'\
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 13/26] tools: post test-runner rewrite cleanup
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (10 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 12/26] auto-t: remove device.wait_for_connected James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 14/26] doc: update test runner docs James Prestwood
` (13 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1353 bytes --]
Removed test-runner.c, and renamed py_runner to test-runner. Removed
tools/test-runner from .gitignore.
This was done as a separate commit to avoid a nasty diff between the
existing test runner, and the new python version
---
.gitignore | 1 -
Makefile.am | 7 -------
tools/{py_runner.py => test-runner} | 0
3 files changed, 8 deletions(-)
rename tools/{py_runner.py => test-runner} (100%)
diff --git a/.gitignore b/.gitignore
index 93fec4f7..33405af8 100644
--- a/.gitignore
+++ b/.gitignore
@@ -33,7 +33,6 @@ wired/ead.8
wired/ead.service
tools/hwsim
tools/hwsim.1
-tools/test-runner
tools/probe-req
unit/test-cmac-aes
unit/test-arc4
diff --git a/Makefile.am b/Makefile.am
index e83dbeee..5f16920f 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -366,13 +366,6 @@ man_MANS += tools/hwsim.1
endif
endif
-if TOOLS
-noinst_PROGRAMS += tools/test-runner
-
-tools_test_runner_SOURCES = tools/test-runner.c
-tools_test_runner_LDADD = $(ell_ldadd)
-endif
-
unit_tests = unit/test-cmac-aes \
unit/test-hmac-md5 unit/test-hmac-sha1 unit/test-hmac-sha256 \
unit/test-prf-sha1 unit/test-kdf-sha256 \
diff --git a/tools/py_runner.py b/tools/test-runner
similarity index 100%
rename from tools/py_runner.py
rename to tools/test-runner
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 14/26] doc: update test runner docs
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (11 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 13/26] tools: post test-runner rewrite cleanup James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 15/26] auto-t: hwsim: wait for method return James Prestwood
` (12 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 7245 bytes --]
---
doc/test-runner.txt | 117 +++++++++++++-------------------------------
1 file changed, 34 insertions(+), 83 deletions(-)
diff --git a/doc/test-runner.txt b/doc/test-runner.txt
index 2d45bedc..17d7bbb2 100644
--- a/doc/test-runner.txt
+++ b/doc/test-runner.txt
@@ -17,18 +17,21 @@ OS:
Name: Tested ver.:
1. qemu 2.4.1
- 2. Linux kernel 4.20
+ 2. Linux kernel 4.20+
3. dbus-daemon 1.11.18
4. ifconfig 2.10-alpha
5. iw 3.17
6. python 2.7
7. haveged no ver. avail.
- 8. hostapd commit id: 31d3692
+ 8. hostapd recent
9. <iwd>/tools/hwsim 0.0
10. <iwd>/src/iwd 0.0
+ 11. ofono (optional)
+ 12. phonesim (optional)
-Note: You will need ell-key-crypto branch, not the master branch from the tree
-in step 2 above.
+Note: The version for hostapd is listed as 'recent'. Older hostapd versions
+will work but we are continually adopting new features from hostapd and using
+those to test. Its recommended to use a recent release of hostapd.
Note: The test-runner mounts host's file system in readonly mode and executes
the above binaries inside of an emulated environment directly from it.
@@ -100,7 +103,6 @@ By default the test-runner will search for the kernel image in these locations:
An arbitrary kernel image location can be specified by using '--kernel <path>'
parameter into test-runner.
-
Running Automated Tests
=======================
Before running any tests, its expected that the folder /var/lib/iwd exists on
@@ -113,8 +115,8 @@ By default, the automated test configuration directories reside in
'<iwd>/autotests' and have a mandatory prefix of 'test'.
<iwd>/autotests/test1
- /test2
- ...
+ /test2
+ ...
The test configurations along with test cases in <iwd>/autotests/test*
directories will be discovered and executed by test-runner in sequential
@@ -133,6 +135,10 @@ The command line may look as follows:
$ sudo ./test-runner -t test1,test3,/home/test4
+Glob matching is also supported when specifying a test list:
+
+ $ sudo ./test-runner -t testWPA*
+
Creating Test Configurations
============================
@@ -145,7 +151,11 @@ A typical test configuration directory may consist of these types of files:
*Test or *Test.py The set of test cases for IWD functionality
implemented using Python scripting language.
These files must have one of the two predefined
- suffixes: 'Test' or 'Test.py'
+ suffixes: 'test' or 'test.py'. The file name
+ must also be more descriptive than simply 'test'.
+ e.g. 'connection_test' or 'failure_test'. Files
+ named 'test' will not be dynamically loaded due
+ to python module naming overlap.
*.conf A configuration file for an instance of hostapd
(Defined in hw.conf) service.
@@ -164,6 +174,9 @@ A typical contents of a test configuration directory may look as follows:
networkScanTest
networkConnectTest.py
+Note: Any additional files in your test directory will be copied to /tmp inside
+the VM. These additional file should be referenced using /tmp/<file> inside
+any configuration file for hostapd.
Defining Network
----------------
@@ -187,61 +200,26 @@ supplicant running IWD:
# has a range of [1, 100].
num_radios=3
-# Definition of the radio configuration identifiers. These identifiers are used
-# to map the APs and IWD to the radios with the particular hardware properties.
-# This field is optional. If identifier is omitted from this list, then the
-# default radio properties will be used as they are defined in mac80211_hwsim
-# driver to satisfy a total number of radios requested in 'num_radios' field.
-radio_confs=rad0:rad1
-
# Maximum execution interval per Python script file in seconds. This field is
# optional.
# Default: 20 seconds.
#max_test_exec_interval_sec=5
-# List of paths inside of a test configuration directory for which
-# the symlinks will be created inside of /tmp. Such paths can be used
-# to specify an absolute path to the needed files inside of IWD and Hostapd
-# configuration files.
-# Example:
-# <some path>/test1/certs
-# misc
-#
-# certs and misc directories will be respectively mapped to:
-#
-# /tmp/certs
-# misc
-#
-# This field is optional.
-#tmpfs_extra_stuff=certs:misc
-
-# Flag to prevent test-runner from starting IWD. Therefore, it may later be
-# started from the python test cases.
-# This field is optional. Default: 1 (true)
-#start_iwd=0
-
-# Configuration directory to use for IWD daemon. IWD expects 'main.conf' to be
-# inside of the specified directory.
-# This field is optional. Default: /etc/iwd
-#iwd_config_dir=/etc/iwd
-
#
# The following two configuration groups are examples of the radio
# configurations.
#
# This group of settings allows to specify a set of properties for a radio. The
-# name of the group represents a radio identifier and must be predefined in
-# 'radio_confs' field inside of 'SETUP' group. This configuration group is
-# optional.
-# TODO: explain each one of the properties.
+# name of the group represents a radio identifier. These configuration groups
+# are optional.
[rad0]
-channels=2
-p2p_device=1
-use_chanctx=1
-# Properties of the second radio. This configuration group is optional.
-[rad1]
-p2p=0
+# For each radX group you can specify radio properties:
+# Disables the provided interface type(s):
+disable_iftype=ap,p2p
+
+# Disables the provided cipher types:
+disable_cipher=ccmp,bip
# 'HOSTAPD' configuration group identifies a set of access points (AP) for the
# current network topology. Each key/value pair represents a single AP that is
@@ -253,8 +231,12 @@ p2p=0
[HOSTAPD]
rad0=ap1.conf
rad1=ap2.conf
-#~~~~~~~~~~~~~~~~~~ end of hw.conf ~~~~~~~~~~~~~~~~~~~~~~~~~
+# 'radius_server' is a special option used to specify a hostapd instance which
+# acts as a radius server only (not an access point). This option does not
+# require an additional 'num_radios' to be used.
+radius_server=radius.conf
+#~~~~~~~~~~~~~~~~~~ end of hw.conf ~~~~~~~~~~~~~~~~~~~~~~~~~
Configuring Access Points
-------------------------
@@ -358,37 +340,6 @@ For more information on Python unit test framework refer to the following page:
http://pyunit.sourceforge.net/pyunit.html
-
-Examples of the framework usage:
-
-#~~~~~~~~~~~~~~~~~~~~~~~~~ alwaysPassingTest.py ~~~~~~~~~~~~~~~~~~~~~~~~~
-#!/usr/bin/python
-
-import unittest
-
-class TestPassingCase(unittest.TestCase):
-
- def test_pass(self):
- self.assertTrue(True)
-
-if __name__ == '__main__':
- unittest.main()
-#~~~~~~~~~~~~~~~~~~ end of alwaysPassingTest.py ~~~~~~~~~~~~~~~~~~~~~~~~~
-
-#~~~~~~~~~~~~~~~~~~~~~~~~~ alwaysFailingTest.py ~~~~~~~~~~~~~~~~~~~~~~~~~
-#!/usr/bin/python
-
-import unittest
-
-class TestFailingCase(unittest.TestCase):
-
- def test_fail(self):
- self.assertTrue(False)
-
-if __name__ == '__main__':
- unittest.main()
-#~~~~~~~~~~~~~~~~~~ end of alwaysFailingTest.py ~~~~~~~~~~~~~~~~~~~~~~~~~
-
Using hardware passthrough
---------------------------
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 15/26] auto-t: hwsim: wait for method return
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (12 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 14/26] doc: update test runner docs James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 16/26] auto-t: Increase all test timeouts James Prestwood
` (11 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 4262 bytes --]
It is not safe to assume that the python dbus implementation will
wait for a method to return. The documentation says this with
respect to reply_handler/error_handler:
"If both are None, the implementation may request that no reply is sent"
To stay on the safe side we should always include the error/reply
handlers and wait for the operation to complete.
---
autotests/util/hwsim.py | 31 +++++++++++++++++++++++--------
1 file changed, 23 insertions(+), 8 deletions(-)
diff --git a/autotests/util/hwsim.py b/autotests/util/hwsim.py
index 6d2b3e8f..a0ddedcd 100755
--- a/autotests/util/hwsim.py
+++ b/autotests/util/hwsim.py
@@ -58,7 +58,9 @@ class Rule(HwsimDBusAbstract):
@source.setter
def source(self, value):
- self._prop_proxy.Set(self._iface_name, 'Source', value)
+ self._prop_proxy.Set(self._iface_name, 'Source', value, reply_handler=self._success,
+ error_handler=self._failure)
+ self._wait_for_async_op()
@property
def destination(self):
@@ -66,7 +68,9 @@ class Rule(HwsimDBusAbstract):
@destination.setter
def destination(self, value):
- self._prop_proxy.Set(self._iface_name, 'Destination', value)
+ self._prop_proxy.Set(self._iface_name, 'Destination', value, reply_handler=self._success,
+ error_handler=self._failure)
+ self._wait_for_async_op()
@property
def bidirectional(self):
@@ -75,7 +79,8 @@ class Rule(HwsimDBusAbstract):
@bidirectional.setter
def bidirectional(self, value):
self._prop_proxy.Set(self._iface_name, 'Bidirectional',
- dbus.Boolean(value))
+ dbus.Boolean(value), reply_handler=self._success, error_handler=self._failure)
+ self._wait_for_async_op()
@property
def frequency(self):
@@ -84,7 +89,8 @@ class Rule(HwsimDBusAbstract):
@frequency.setter
def frequency(self, value):
self._prop_proxy.Set(self._iface_name, 'Frequency',
- dbus.UInt32(value))
+ dbus.UInt32(value), reply_handler=self._success, error_handler=self._failure)
+ self._wait_for_async_op()
@property
def priority(self):
@@ -93,7 +99,8 @@ class Rule(HwsimDBusAbstract):
@priority.setter
def priority(self, value):
self._prop_proxy.Set(self._iface_name, 'Priority',
- dbus.Int16(value))
+ dbus.Int16(value), reply_handler=self._success, error_handler=self._failure)
+ self._wait_for_async_op()
@property
def signal(self):
@@ -102,7 +109,8 @@ class Rule(HwsimDBusAbstract):
@signal.setter
def signal(self, value):
self._prop_proxy.Set(self._iface_name, 'SignalStrength',
- dbus.Int16(value))
+ dbus.Int16(value), reply_handler=self._success, error_handler=self._failure)
+ self._wait_for_async_op()
@property
def drop(self):
@@ -110,7 +118,9 @@ class Rule(HwsimDBusAbstract):
@drop.setter
def drop(self, value):
- self._prop_proxy.Set(self._iface_name, 'Drop', dbus.Boolean(value))
+ self._prop_proxy.Set(self._iface_name, 'Drop', dbus.Boolean(value),
+ reply_handler=self._success, error_handler=self._failure)
+ self._wait_for_async_op()
@property
def delay(self):
@@ -118,7 +128,9 @@ class Rule(HwsimDBusAbstract):
@delay.setter
def delay(self, value):
- self._prop_proxy.Set(self._iface_name, 'Delay', dbus.UInt32(value))
+ self._prop_proxy.Set(self._iface_name, 'Delay', dbus.UInt32(value),
+ reply_handler=self._success, error_handler=self._failure)
+ self._wait_for_async_op()
def remove(self):
self._iface.Remove(reply_handler=self._success,
@@ -126,6 +138,9 @@ class Rule(HwsimDBusAbstract):
self._wait_for_async_op()
+ def __del__(self):
+ self.remove()
+
def __str__(self, prefix = ''):
return prefix + 'Rule: ' + self.path + '\n' + \
prefix + '\tSource:\t\t' + self.source + '\n' + \
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 16/26] auto-t: Increase all test timeouts
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (13 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 15/26] auto-t: hwsim: wait for method return James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 17/26] auto-t: fix SAE FT test and update James Prestwood
` (10 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 17691 bytes --]
Slower systems may not be able to make some timeouts that tests
mandated. All timeouts were increased significantly to allow tests
to pass on slow systems.
---
autotests/testAPRoam/connection_test.py | 4 ++--
autotests/testBSSBlacklist/connection_test.py | 4 ++--
autotests/testFT-8021x-roam/connection_test.py | 8 ++++----
autotests/testFT-FILS-SHA256/connection_test.py | 4 ++--
autotests/testFT-FILS-SHA384/connection_test.py | 4 ++--
autotests/testFT-PSK-over-DS/connection_test.py | 4 ++--
autotests/testFT-PSK-roam/connection_test.py | 4 ++--
autotests/testFT-SAE-roam/connection_test.py | 8 ++++----
autotests/testHiddenNetworks/validation.py | 6 +++---
autotests/testKnownNetworks/known_network_test.py | 4 ++--
autotests/testPreauth-roam/connection_test.py | 4 ++--
| 2 +-
autotests/testSAE/autoconnect_test.py | 2 +-
autotests/testScan/requested_scan_test.py | 4 ++--
autotests/util/iwd.py | 8 ++++----
autotests/util/testutil.py | 2 +-
tools/test-runner | 2 +-
17 files changed, 37 insertions(+), 37 deletions(-)
diff --git a/autotests/testAPRoam/connection_test.py b/autotests/testAPRoam/connection_test.py
index c8c2ca9f..308bbc28 100644
--- a/autotests/testAPRoam/connection_test.py
+++ b/autotests/testAPRoam/connection_test.py
@@ -60,10 +60,10 @@ class Test(unittest.TestCase):
(bss_radio[2].addresses[0], '8f0000005103060603000000')])
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 15)
+ wd.wait_for_object_condition(device, condition)
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(bss_hostapd[1].list_sta())
diff --git a/autotests/testBSSBlacklist/connection_test.py b/autotests/testBSSBlacklist/connection_test.py
index 825439d4..0a51b8cb 100644
--- a/autotests/testBSSBlacklist/connection_test.py
+++ b/autotests/testBSSBlacklist/connection_test.py
@@ -99,10 +99,10 @@ class Test(unittest.TestCase):
# Now we wait... AutoConnect should take over
condition = 'obj.state == DeviceState.connecting'
- wd.wait_for_object_condition(device, condition, 15)
+ wd.wait_for_object_condition(device, condition)
condition = 'obj.state == DeviceState.connected'
- wd.wait_for_object_condition(device, condition, 15)
+ wd.wait_for_object_condition(device, condition)
# Same as before, make sure we didn't connect to the blacklisted AP.
self.assertNotIn(device.address, bss_hostapd[0].list_sta())
diff --git a/autotests/testFT-8021x-roam/connection_test.py b/autotests/testFT-8021x-roam/connection_test.py
index 65c73492..084bac64 100644
--- a/autotests/testFT-8021x-roam/connection_test.py
+++ b/autotests/testFT-8021x-roam/connection_test.py
@@ -75,12 +75,12 @@ class Test(unittest.TestCase):
rule0.signal = -8000
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 10)
+ wd.wait_for_object_condition(device, condition)
# Check that iwd is on BSS 1 once out of roaming state and doesn't
# go through 'disconnected', 'autoconnect', 'connecting' in between
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(self.bss_hostapd[1].list_sta())
@@ -151,12 +151,12 @@ class Test(unittest.TestCase):
rule0.drop = False
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 20)
+ wd.wait_for_object_condition(device, condition)
# Check that iwd is on BSS 1 once out of roaming state and doesn't
# go through 'disconnected', 'autoconnect', 'connecting' in between
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(self.bss_hostapd[1].list_sta())
diff --git a/autotests/testFT-FILS-SHA256/connection_test.py b/autotests/testFT-FILS-SHA256/connection_test.py
index 16f46210..d87afce0 100644
--- a/autotests/testFT-FILS-SHA256/connection_test.py
+++ b/autotests/testFT-FILS-SHA256/connection_test.py
@@ -105,12 +105,12 @@ class Test(unittest.TestCase):
rule0.signal = -8000
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 10)
+ wd.wait_for_object_condition(device, condition)
# Check that iwd is on BSS 1 once out of roaming state and doesn't
# go through 'disconnected', 'autoconnect', 'connecting' in between
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(self.bss_hostapd[1].list_sta())
diff --git a/autotests/testFT-FILS-SHA384/connection_test.py b/autotests/testFT-FILS-SHA384/connection_test.py
index 16f46210..d87afce0 100644
--- a/autotests/testFT-FILS-SHA384/connection_test.py
+++ b/autotests/testFT-FILS-SHA384/connection_test.py
@@ -105,12 +105,12 @@ class Test(unittest.TestCase):
rule0.signal = -8000
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 10)
+ wd.wait_for_object_condition(device, condition)
# Check that iwd is on BSS 1 once out of roaming state and doesn't
# go through 'disconnected', 'autoconnect', 'connecting' in between
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(self.bss_hostapd[1].list_sta())
diff --git a/autotests/testFT-PSK-over-DS/connection_test.py b/autotests/testFT-PSK-over-DS/connection_test.py
index 2cfdf245..188eca63 100644
--- a/autotests/testFT-PSK-over-DS/connection_test.py
+++ b/autotests/testFT-PSK-over-DS/connection_test.py
@@ -81,12 +81,12 @@ class Test(unittest.TestCase):
rule0.signal = -8000
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 10)
+ wd.wait_for_object_condition(device, condition)
# Check that iwd is on BSS 1 once out of roaming state and doesn't
# go through 'disconnected', 'autoconnect', 'connecting' in between
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(self.bss_hostapd[1].list_sta())
diff --git a/autotests/testFT-PSK-roam/connection_test.py b/autotests/testFT-PSK-roam/connection_test.py
index f40fdf58..37dd80d0 100644
--- a/autotests/testFT-PSK-roam/connection_test.py
+++ b/autotests/testFT-PSK-roam/connection_test.py
@@ -83,12 +83,12 @@ class Test(unittest.TestCase):
rule0.signal = -8000
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 10)
+ wd.wait_for_object_condition(device, condition)
# Check that iwd is on BSS 1 once out of roaming state and doesn't
# go through 'disconnected', 'autoconnect', 'connecting' in between
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(self.bss_hostapd[1].list_sta())
diff --git a/autotests/testFT-SAE-roam/connection_test.py b/autotests/testFT-SAE-roam/connection_test.py
index 3e209560..dcea148c 100644
--- a/autotests/testFT-SAE-roam/connection_test.py
+++ b/autotests/testFT-SAE-roam/connection_test.py
@@ -86,12 +86,12 @@ class Test(unittest.TestCase):
rule0.signal = -8000
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 10)
+ wd.wait_for_object_condition(device, condition)
# Check that iwd is on BSS 1 once out of roaming state and doesn't
# go through 'disconnected', 'autoconnect', 'connecting' in between
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
rule1.signal = -2000
@@ -112,10 +112,10 @@ class Test(unittest.TestCase):
rule2.signal = -1000
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 15)
+ wd.wait_for_object_condition(device, condition)
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(self.bss_hostapd[2].list_sta())
diff --git a/autotests/testHiddenNetworks/validation.py b/autotests/testHiddenNetworks/validation.py
index 780af8b3..7ff7075a 100644
--- a/autotests/testHiddenNetworks/validation.py
+++ b/autotests/testHiddenNetworks/validation.py
@@ -20,12 +20,12 @@ class TestConnectAutoConnect(unittest.TestCase):
device.connect_hidden_network(ssid)
condition = 'obj.state == DeviceState.connected'
- wd.wait_for_object_condition(device, condition, 30)
+ wd.wait_for_object_condition(device, condition)
def check_autoconnect_hidden_network(self, wd, device, ssid, throws):
if throws is None:
condition = 'obj.state == DeviceState.connected'
- wd.wait_for_object_condition(device, condition, 30)
+ wd.wait_for_object_condition(device, condition)
condition = 'obj.connected_network is not None'
wd.wait_for_object_condition(device, condition)
@@ -37,7 +37,7 @@ class TestConnectAutoConnect(unittest.TestCase):
device.disconnect()
condition = 'obj.state == DeviceState.disconnected'
- wd.wait_for_object_condition(device, condition, 30)
+ wd.wait_for_object_condition(device, condition)
def validate_connection(self, wd, ssid, autoconnect, throws, use_agent,
wait_periodic_scan):
diff --git a/autotests/testKnownNetworks/known_network_test.py b/autotests/testKnownNetworks/known_network_test.py
index b2b3d105..a94a2151 100644
--- a/autotests/testKnownNetworks/known_network_test.py
+++ b/autotests/testKnownNetworks/known_network_test.py
@@ -56,7 +56,7 @@ class Test(unittest.TestCase):
IWD.copy_to_storage('known_networks/ssidPSK.psk')
condition = 'len(obj.list_known_networks()) == 5'
- wd.wait_for_object_condition(wd, condition, 1)
+ wd.wait_for_object_condition(wd, condition)
expected = ['ssidNew', 'ssidOpen', 'ssidPSK', 'ssidEAP-TLS',
'Hotspot Network']
@@ -65,7 +65,7 @@ class Test(unittest.TestCase):
IWD.remove_from_storage('ssidPSK.psk')
condition = 'len(obj.list_known_networks()) == 4'
- wd.wait_for_object_condition(wd, condition, 1)
+ wd.wait_for_object_condition(wd, condition)
for net in known_networks:
net.forget()
diff --git a/autotests/testPreauth-roam/connection_test.py b/autotests/testPreauth-roam/connection_test.py
index ed043c3f..839e04b0 100644
--- a/autotests/testPreauth-roam/connection_test.py
+++ b/autotests/testPreauth-roam/connection_test.py
@@ -91,14 +91,14 @@ class Test(unittest.TestCase):
rule0.signal = -8000
condition = 'obj.state == DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 15)
+ wd.wait_for_object_condition(device, condition)
# TODO: verify that the PMK from preauthentication was used
# Check that iwd is on BSS 1 once out of roaming state and doesn't
# go through 'disconnected', 'autoconnect', 'connecting' in between
condition = 'obj.state != DeviceState.roaming'
- wd.wait_for_object_condition(device, condition, 5)
+ wd.wait_for_object_condition(device, condition)
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(bss_hostapd[1].list_sta())
--git a/autotests/testRSSIAgent/connection_test.py b/autotests/testRSSIAgent/connection_test.py
index f8140850..ab0bc3a5 100644
--- a/autotests/testRSSIAgent/connection_test.py
+++ b/autotests/testRSSIAgent/connection_test.py
@@ -71,7 +71,7 @@ class Test(unittest.TestCase):
rule.signal = centre
agent.level = -1
condition = 'obj.level == ' + str(level)
- wd.wait_for_object_condition(agent, condition, 3)
+ wd.wait_for_object_condition(agent, condition)
self.assertTrue(agent.calls > 0)
self.assertEqual(agent.device_path, device.device_path)
diff --git a/autotests/testSAE/autoconnect_test.py b/autotests/testSAE/autoconnect_test.py
index 0613d91e..48b4b6cd 100644
--- a/autotests/testSAE/autoconnect_test.py
+++ b/autotests/testSAE/autoconnect_test.py
@@ -24,7 +24,7 @@ class Test(unittest.TestCase):
devices[3].disconnect()
condition = 'obj.state == DeviceState.connected'
- wd.wait_for_object_condition(device, condition, 30)
+ wd.wait_for_object_condition(device, condition)
condition = 'obj.connected_network is not None'
wd.wait_for_object_condition(device, condition)
diff --git a/autotests/testScan/requested_scan_test.py b/autotests/testScan/requested_scan_test.py
index 11282ce1..3858b4f5 100644
--- a/autotests/testScan/requested_scan_test.py
+++ b/autotests/testScan/requested_scan_test.py
@@ -36,9 +36,9 @@ class Test(unittest.TestCase):
device = devices[0]
condition = 'obj.scanning'
- wd.wait_for_object_condition(device, condition, 20)
+ wd.wait_for_object_condition(device, condition)
condition = 'not obj.scanning'
- wd.wait_for_object_condition(device, condition, 20)
+ wd.wait_for_object_condition(device, condition)
device.scan()
diff --git a/autotests/util/iwd.py b/autotests/util/iwd.py
index ec5e662f..e74a6faf 100755
--- a/autotests/util/iwd.py
+++ b/autotests/util/iwd.py
@@ -867,8 +867,8 @@ class IWD(AsyncOpAbstract):
tries = 0
while not self._bus.name_has_owner(IWD_SERVICE):
- if ctx.args.gdb == 'None':
- if tries > 100:
+ if not ctx.args.gdb:
+ if tries > 200:
if start_iwd_daemon:
ctx.stop_process(self._iwd_proc)
self._iwd_proc = None
@@ -913,7 +913,7 @@ class IWD(AsyncOpAbstract):
IWD_AGENT_MANAGER_INTERFACE)
return self._agent_manager_if
- def wait_for_object_condition(self, obj, condition_str, max_wait = 15):
+ def wait_for_object_condition(self, obj, condition_str, max_wait = 50):
self._wait_timed_out = False
def wait_timeout_cb():
self._wait_timed_out = True
@@ -982,7 +982,7 @@ class IWD(AsyncOpAbstract):
def remove_from_storage(file_name):
os.system('rm -rf ' + IWD_STORAGE_DIR + '/\'' + file_name + '\'')
- def list_devices(self, wait_to_appear = 0, max_wait = 15):
+ def list_devices(self, wait_to_appear = 0, max_wait = 50):
if not wait_to_appear:
return list(self._devices.values())
diff --git a/autotests/util/testutil.py b/autotests/util/testutil.py
index 0a94e23a..f9cfbf29 100644
--- a/autotests/util/testutil.py
+++ b/autotests/util/testutil.py
@@ -82,7 +82,7 @@ def test_connected(if0=None, if1=None, group=True):
rec.append(False)
while not all(rec):
- r, w, x = select.select([sock0, sock1], [], [], 1.0)
+ r, w, x = select.select([sock0, sock1], [], [], 10)
if not r:
raise Exception('timeout waiting for packets: ' + repr(rec))
diff --git a/tools/test-runner b/tools/test-runner
index d09f08d2..d7f831ac 100755
--- a/tools/test-runner
+++ b/tools/test-runner
@@ -43,7 +43,7 @@ config = None
intf_id = 0
rad_id = 0
-TEST_MAX_TIMEOUT = 45
+TEST_MAX_TIMEOUT = 120
dbus.mainloop.glib.DBusGMainLoop(set_as_default=True)
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 17/26] auto-t: fix SAE FT test and update
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (14 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 16/26] auto-t: Increase all test timeouts James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 18/26] auto-t: update testClientNonInteractive to use TestContext James Prestwood
` (9 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 5186 bytes --]
Updated testFT-SAE-roam to use the TestContext APIs as well as
fixed the failure which was introduced after requiring stricter
AKM logic for SAE networks. The new failure was due to the hostapd
config not including the standard SAE AKM which is actually
required by the spec.
---
autotests/testFT-SAE-roam/connection_test.py | 22 ++++++++++++--------
autotests/testFT-SAE-roam/ft-psk-3.conf | 2 --
autotests/testFT-SAE-roam/ft-sae-1.conf | 4 +---
autotests/testFT-SAE-roam/ft-sae-2.conf | 4 +---
4 files changed, 15 insertions(+), 17 deletions(-)
diff --git a/autotests/testFT-SAE-roam/connection_test.py b/autotests/testFT-SAE-roam/connection_test.py
index dcea148c..99f58456 100644
--- a/autotests/testFT-SAE-roam/connection_test.py
+++ b/autotests/testFT-SAE-roam/connection_test.py
@@ -11,9 +11,12 @@ from iwd import NetworkType
from hwsim import Hwsim
from hostapd import HostapdCLI
import testutil
+from config import ctx
class Test(unittest.TestCase):
def test_roam_success(self):
+ wd = IWD()
+
hwsim = Hwsim()
rule0 = hwsim.rules.create()
@@ -28,8 +31,6 @@ class Test(unittest.TestCase):
rule2.source = self.bss_radio[2].addresses[0]
rule2.bidirectional = True
- wd = IWD()
-
psk_agent = PSKAgent("EasilyGuessedPassword")
wd.register_psk_agent(psk_agent)
@@ -158,17 +159,20 @@ class Test(unittest.TestCase):
hwsim.get_radio('rad1'),
hwsim.get_radio('rad2') ]
- # Set interface addresses to those expected by hostapd config files
- os.system('ifconfig "' + cls.bss_hostapd[0].ifname +
- '" down hw ether 12:00:00:00:00:01 up')
- os.system('ifconfig "' + cls.bss_hostapd[1].ifname +
- '" down hw ether 12:00:00:00:00:02 up')
- os.system('ifconfig "' + cls.bss_hostapd[2].ifname +
- '" down hw ether 12:00:00:00:00:03 up')
+ ctx.start_process(['ifconfig', cls.bss_hostapd[0].ifname, 'down', 'hw', \
+ 'ether', '12:00:00:00:00:01', 'up'], wait=True)
+ ctx.start_process(['ifconfig', cls.bss_hostapd[1].ifname, 'down', 'hw', \
+ 'ether', '12:00:00:00:00:02', 'up'], wait=True)
+ ctx.start_process(['ifconfig', cls.bss_hostapd[2].ifname, 'down', 'hw', \
+ 'ether', '12:00:00:00:00:03', 'up'], wait=True)
+ # Set interface addresses to those expected by hostapd config files
cls.bss_hostapd[0].reload()
+ cls.bss_hostapd[0].wait_for_event("AP-ENABLED")
cls.bss_hostapd[1].reload()
+ cls.bss_hostapd[1].wait_for_event("AP-ENABLED")
cls.bss_hostapd[2].reload()
+ cls.bss_hostapd[2].wait_for_event("AP-ENABLED")
# Fill in the neighbor AP tables in both BSSes. By default each
# instance knows only about current BSS, even inside one hostapd
diff --git a/autotests/testFT-SAE-roam/ft-psk-3.conf b/autotests/testFT-SAE-roam/ft-psk-3.conf
index 6a2d4cad..8918cf9e 100644
--- a/autotests/testFT-SAE-roam/ft-psk-3.conf
+++ b/autotests/testFT-SAE-roam/ft-psk-3.conf
@@ -16,8 +16,6 @@ wpa_passphrase=EasilyGuessedPassword
wpa_ptk_rekey=30
wpa_group_rekey=80
ieee80211w=2
-rsn_preauth=1
-rsn_preauth_interfaces=lo
disable_pmksa_caching=0
# Allow PMK cache to be shared opportunistically among configured interfaces
# and BSSes (i.e., all configurations within a single hostapd process).
diff --git a/autotests/testFT-SAE-roam/ft-sae-1.conf b/autotests/testFT-SAE-roam/ft-sae-1.conf
index 6642c398..aa40494a 100644
--- a/autotests/testFT-SAE-roam/ft-sae-1.conf
+++ b/autotests/testFT-SAE-roam/ft-sae-1.conf
@@ -10,14 +10,12 @@ nas_identifier=dummy1
wpa=2
# Can support SAE and FT-SAE (space separated list) and/or EAP at the same
# time but we want to force FT
-wpa_key_mgmt=FT-SAE
+wpa_key_mgmt=FT-SAE SAE
sae_password=EasilyGuessedPassword|mac=ff:ff:ff:ff:ff:ff
wpa_pairwise=CCMP
wpa_ptk_rekey=30
wpa_group_rekey=80
ieee80211w=2
-rsn_preauth=1
-rsn_preauth_interfaces=lo
disable_pmksa_caching=0
# Allow PMK cache to be shared opportunistically among configured interfaces
# and BSSes (i.e., all configurations within a single hostapd process).
diff --git a/autotests/testFT-SAE-roam/ft-sae-2.conf b/autotests/testFT-SAE-roam/ft-sae-2.conf
index fa841db8..5bff6c9d 100644
--- a/autotests/testFT-SAE-roam/ft-sae-2.conf
+++ b/autotests/testFT-SAE-roam/ft-sae-2.conf
@@ -10,14 +10,12 @@ nas_identifier=dummy2
wpa=2
# Can support SAE and FT-SAE (space separated list) and/or EAP at the same
# time but we want to force FT
-wpa_key_mgmt=FT-SAE
+wpa_key_mgmt=FT-SAE SAE
sae_password=EasilyGuessedPassword|mac=ff:ff:ff:ff:ff:ff
wpa_pairwise=CCMP
wpa_ptk_rekey=30
wpa_group_rekey=80
ieee80211w=2
-rsn_preauth=1
-rsn_preauth_interfaces=lo
disable_pmksa_caching=0
# Allow PMK cache to be shared opportunistically among configured interfaces
# and BSSes (i.e., all configurations within a single hostapd process).
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 18/26] auto-t: update testClientNonInteractive to use TestContext
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (15 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 17/26] auto-t: fix SAE FT test and update James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 19/26] auto-t: update testAgent " James Prestwood
` (8 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 3698 bytes --]
This makes any iwctl output hidden unless enabled via the verbose
option.
---
.../connect_command_test.py | 26 +++++++++----------
1 file changed, 12 insertions(+), 14 deletions(-)
diff --git a/autotests/testClientNonInteractive/connect_command_test.py b/autotests/testClientNonInteractive/connect_command_test.py
index 886d7a76..a503bd24 100644
--- a/autotests/testClientNonInteractive/connect_command_test.py
+++ b/autotests/testClientNonInteractive/connect_command_test.py
@@ -8,6 +8,7 @@ import iwd
from iwd import IWD
import testutil
import subprocess
+from config import ctx
class Test(unittest.TestCase):
@@ -34,11 +35,10 @@ class Test(unittest.TestCase):
# Use --dontaks cmd-line option
with self.assertRaises(subprocess.CalledProcessError):
- subprocess.check_call(['iwctl', '-d', 'station',
- device.name, 'connect', ssid])
+ ctx.start_process(['iwctl', 'd', 'station', device.name, 'connect', ssid], check=True)
- subprocess.check_call(['iwctl', '-P', 'passphrase',
- 'station', device.name, 'connect', ssid])
+ ctx.start_process(['iwctl', '-P', 'passphrase', 'station', device.name, 'connect', ssid],
+ check=True)
self.check_connection_success(ssid)
@@ -47,9 +47,8 @@ class Test(unittest.TestCase):
device = self.wd.list_devices(1)[0]
- subprocess.check_call(['iwctl', '-u', 'user', '-p', 'password',
- 'station', device.name, 'connect', ssid])
-
+ ctx.start_process(['iwctl', '-u', 'user', '-p', 'password', 'station', \
+ device.name, 'connect', ssid], check=True)
self.check_connection_success(ssid)
def test_connection_with_password(self):
@@ -57,8 +56,8 @@ class Test(unittest.TestCase):
device = self.wd.list_devices(1)[0]
- subprocess.check_call(['iwctl', '-p', 'password',
- 'station', device.name, 'connect', ssid])
+ ctx.start_process(['iwctl', '-p', 'password', 'station', device.name, 'connect', ssid],
+ check=True)
self.check_connection_success(ssid)
@@ -68,8 +67,8 @@ class Test(unittest.TestCase):
device = self.wd.list_devices(1)[0]
with self.assertRaises(subprocess.CalledProcessError):
- subprocess.check_call(['iwctl', '-P', 'incorrect_passphrase',
- 'station', device.name, 'connect', ssid])
+ ctx.start_process(['iwctl', '-P', 'incorrect_passphrase', 'station', device.name, \
+ 'connect', ssid], check=True)
def test_invalid_command_line_option(self):
ssid = 'ssidPassphrase'
@@ -77,14 +76,13 @@ class Test(unittest.TestCase):
device = self.wd.list_devices(1)[0]
with self.assertRaises(subprocess.CalledProcessError):
- subprocess.check_call(['iwctl', '-z',
- 'station', device.name, 'connect', ssid])
+ ctx.start_process(['iwctl', '-z', 'station', device.name, 'connect', ssid], check=True)
def test_invalid_command(self):
device = self.wd.list_devices(1)[0]
with self.assertRaises(subprocess.CalledProcessError):
- subprocess.check_call(['iwctl', 'inexistent', 'command'])
+ ctx.start_process(['iwctl', 'inexistent', 'command'], check=True)
@classmethod
def setUpClass(cls):
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 19/26] auto-t: update testAgent to use TestContext
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (16 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 18/26] auto-t: update testClientNonInteractive to use TestContext James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 20/26] auto-t: fix testBSSBlacklist James Prestwood
` (7 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1185 bytes --]
This hides iwctl output unless verbose is specified
---
autotests/testAgent/agent_test.py | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/autotests/testAgent/agent_test.py b/autotests/testAgent/agent_test.py
index 80d2ce83..60465d3f 100644
--- a/autotests/testAgent/agent_test.py
+++ b/autotests/testAgent/agent_test.py
@@ -10,6 +10,7 @@ from iwd import PSKAgent
from iwd import NetworkType
import testutil
import subprocess
+from config import ctx
class Test(unittest.TestCase):
@@ -57,7 +58,7 @@ class Test(unittest.TestCase):
def test_connection_with_other_agent(self):
wd = IWD()
- iwctl = subprocess.Popen(['iwctl', '-P', 'secret_ssid2'])
+ iwctl = ctx.start_process(['iwctl', '-P', 'secret_ssid2']).pid
# Let iwctl to start and register its agent.
wd.wait(2)
@@ -72,7 +73,7 @@ class Test(unittest.TestCase):
wd = IWD()
- iwctl = subprocess.Popen(['iwctl', '-P', 'secret_ssid2'])
+ iwctl = ctx.start_process(['iwctl', '-P', 'secret_ssid2']).pid
# Let iwctl to start and register its agent.
wd.wait(2)
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 20/26] auto-t: fix testBSSBlacklist
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (17 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 19/26] auto-t: update testAgent " James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 21/26] auto-t: fix testAP James Prestwood
` (6 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 5893 bytes --]
There were a number of fixes here. The waits were changed to wait
on the device state instead of the network state and hwsim rules
were removed after the test as to not interfere with future tests.
One of the rules was setting the signal to -10000 wich was causing
the ranking to be zero.
---
.../testBSSBlacklist/all_blacklisted_test.py | 11 ++++++++---
autotests/testBSSBlacklist/bad_pass_test.py | 8 +++++++-
autotests/testBSSBlacklist/connection_test.py | 9 +++++++--
autotests/testBSSBlacklist/temp_blacklist_test.py | 15 ++++++++++++++-
4 files changed, 36 insertions(+), 7 deletions(-)
diff --git a/autotests/testBSSBlacklist/all_blacklisted_test.py b/autotests/testBSSBlacklist/all_blacklisted_test.py
index 099804a8..d3ed7acc 100644
--- a/autotests/testBSSBlacklist/all_blacklisted_test.py
+++ b/autotests/testBSSBlacklist/all_blacklisted_test.py
@@ -37,7 +37,7 @@ class Test(unittest.TestCase):
rule2 = hwsim.rules.create()
rule2.source = bss_radio[2].addresses[0]
rule2.bidirectional = True
- rule2.signal = -10000
+ rule2.signal = -9000
wd = IWD(True)
@@ -72,17 +72,22 @@ class Test(unittest.TestCase):
rule0.drop = False
rule1.drop = False
+ rule2.drop = False
# This connect should work
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
self.assertIn(device.address, bss_hostapd[0].list_sta())
wd.unregister_psk_agent(psk_agent)
+ rule0.remove()
+ rule1.remove()
+ rule2.remove()
+
@classmethod
def setUpClass(cls):
pass
diff --git a/autotests/testBSSBlacklist/bad_pass_test.py b/autotests/testBSSBlacklist/bad_pass_test.py
index 220ffcc8..b2ceb998 100644
--- a/autotests/testBSSBlacklist/bad_pass_test.py
+++ b/autotests/testBSSBlacklist/bad_pass_test.py
@@ -39,7 +39,7 @@ class Test(unittest.TestCase):
rule2.bidirectional = True
rule2.signal = -4000
- wd = IWD(True, '/tmp')
+ wd = IWD(True)
psk_agent = PSKAgent("wrong_password")
wd.register_psk_agent(psk_agent)
@@ -80,6 +80,12 @@ class Test(unittest.TestCase):
condition = 'obj.connected'
wd.wait_for_object_condition(ordered_network.network_object, condition)
+ wd.unregister_psk_agent(psk_agent)
+
+ rule0.remove()
+ rule1.remove()
+ rule2.remove()
+
@classmethod
def setUpClass(cls):
pass
diff --git a/autotests/testBSSBlacklist/connection_test.py b/autotests/testBSSBlacklist/connection_test.py
index 0a51b8cb..8083fc92 100644
--- a/autotests/testBSSBlacklist/connection_test.py
+++ b/autotests/testBSSBlacklist/connection_test.py
@@ -10,7 +10,6 @@ from iwd import PSKAgent
from iwd import NetworkType
from hostapd import HostapdCLI
-
from hwsim import Hwsim
import time
@@ -42,7 +41,7 @@ class Test(unittest.TestCase):
rule2.bidirectional = True
rule2.signal = -4000
- wd = IWD(True, '/tmp')
+ wd = IWD(True)
psk_agent = PSKAgent("secret123")
wd.register_psk_agent(psk_agent)
@@ -67,6 +66,8 @@ class Test(unittest.TestCase):
condition = 'not obj.connected'
wd.wait_for_object_condition(ordered_network.network_object, condition)
+ rule0.drop = True
+
ordered_network.network_object.connect(wait=False)
# Have AP1 drop all packets, should result in a connection timeout
@@ -140,6 +141,10 @@ class Test(unittest.TestCase):
wd.unregister_psk_agent(psk_agent)
+ rule0.remove()
+ rule1.remove()
+ rule2.remove()
+
@classmethod
def setUpClass(cls):
IWD.copy_to_storage('main.conf')
diff --git a/autotests/testBSSBlacklist/temp_blacklist_test.py b/autotests/testBSSBlacklist/temp_blacklist_test.py
index b2b57035..2a9590b8 100644
--- a/autotests/testBSSBlacklist/temp_blacklist_test.py
+++ b/autotests/testBSSBlacklist/temp_blacklist_test.py
@@ -39,7 +39,7 @@ class Test(unittest.TestCase):
rule2.bidirectional = True
rule2.signal = -2000
- wd = IWD(True, '/tmp')
+ wd = IWD(True)
psk_agent = PSKAgent("secret123")
wd.register_psk_agent(psk_agent)
@@ -63,6 +63,9 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(dev1, condition)
+
self.assertIn(dev1.address, bss_hostapd[2].list_sta())
# dev1 now connected, this should max out the first AP, causing the next
@@ -85,6 +88,9 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(dev2, condition)
+
# We should have temporarily blacklisted the first BSS, and connected
# to this one.
self.assertIn(dev2.address, bss_hostapd[1].list_sta())
@@ -111,10 +117,17 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(dev2, condition)
+
self.assertIn(dev2.address, bss_hostapd[2].list_sta())
wd.unregister_psk_agent(psk_agent)
+ rule0.remove()
+ rule1.remove()
+ rule2.remove()
+
@classmethod
def setUpClass(cls):
IWD.copy_to_storage('main.conf')
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 21/26] auto-t: fix testAP
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (18 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 20/26] auto-t: fix testBSSBlacklist James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 22/26] auto-t: fix testAPRoam James Prestwood
` (5 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1596 bytes --]
As with other tests, wait on device state instead of the network
object. The connectivity test was also changed to not check for
group traffic since AP does not negotiate the IGTK at this time.
---
autotests/testAP/connection_test.py | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/autotests/testAP/connection_test.py b/autotests/testAP/connection_test.py
index 36302da1..c3744c55 100644
--- a/autotests/testAP/connection_test.py
+++ b/autotests/testAP/connection_test.py
@@ -24,8 +24,8 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(dev, condition)
wd.unregister_psk_agent(psk_agent)
@@ -74,12 +74,11 @@ class Test(unittest.TestCase):
networks['TestAP2'].network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(networks['TestAP2'].network_object,
- condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(dev2, condition)
testutil.test_iface_operstate(dev2.name)
- testutil.test_ifaces_connected(dev1.name, dev2.name)
+ testutil.test_ifaces_connected(dev1.name, dev2.name, group=False)
wd.unregister_psk_agent(psk_agent)
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 22/26] auto-t: fix testAPRoam
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (19 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 21/26] auto-t: fix testAP James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 23/26] auto-t: harden testHotspot James Prestwood
` (4 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 2568 bytes --]
This test occationally fails due to no hwsim rules. Basically we
were just expecting iwd to connect to one of 3 access points but
the ranking was equal, so it chose the first in the scan list.
Now a signal strength is assigned to each AP to steer IWD into
connecting to the expected AP.
---
autotests/testAPRoam/connection_test.py | 29 +++++++++++++++++++++++--
1 file changed, 27 insertions(+), 2 deletions(-)
diff --git a/autotests/testAPRoam/connection_test.py b/autotests/testAPRoam/connection_test.py
index 308bbc28..3f6296b7 100644
--- a/autotests/testAPRoam/connection_test.py
+++ b/autotests/testAPRoam/connection_test.py
@@ -24,6 +24,22 @@ class Test(unittest.TestCase):
hwsim.get_radio('rad1'),
hwsim.get_radio('rad2') ]
+ rule0 = hwsim.rules.create()
+ rule0.source = bss_radio[0].addresses[0]
+ rule0.bidirectional = True
+
+ rule1 = hwsim.rules.create()
+ rule1.source = bss_radio[1].addresses[0]
+ rule1.bidirectional = True
+
+ rule2 = hwsim.rules.create()
+ rule2.source = bss_radio[2].addresses[0]
+ rule2.bidirectional = True
+
+ rule0.signal = -2000
+ rule1.signal = -2500
+ rule2.signal = -3000
+
wd = IWD()
psk_agent = PSKAgent("secret123")
@@ -49,8 +65,10 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
+
+ bss_hostapd[0].wait_for_event('AP-STA-CONNECTED')
self.assertTrue(bss_hostapd[0].list_sta())
self.assertFalse(bss_hostapd[1].list_sta())
@@ -65,6 +83,9 @@ class Test(unittest.TestCase):
condition = 'obj.state != DeviceState.roaming'
wd.wait_for_object_condition(device, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
+
self.assertEqual(device.state, iwd.DeviceState.connected)
self.assertTrue(bss_hostapd[1].list_sta())
device.disconnect()
@@ -74,6 +95,10 @@ class Test(unittest.TestCase):
wd.unregister_psk_agent(psk_agent)
+ rule0.remove()
+ rule1.remove()
+ rule2.remove()
+
@classmethod
def setUpClass(cls):
pass
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 23/26] auto-t: harden testHotspot
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (20 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 22/26] auto-t: fix testAPRoam James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 24/26] auto-t: fix testWPA2 to wait for device object James Prestwood
` (3 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 3930 bytes --]
This test occationally failed, and it uses the old style of waiting
for connected on the network object instead of the device object.
The hwsim rule was also moved ahead of the IWD() initializer which
ensures that IWD doesn't scan before the rule can be set/processed.
---
autotests/testHotspot/anqp_delay_test.py | 11 ++++++-----
autotests/testHotspot/hessid_test.py | 4 ++--
autotests/testHotspot/hotspot_test.py | 4 ++--
autotests/testHotspot/roaming_test.py | 4 ++--
4 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/autotests/testHotspot/anqp_delay_test.py b/autotests/testHotspot/anqp_delay_test.py
index 431f16c1..adfa57fc 100644
--- a/autotests/testHotspot/anqp_delay_test.py
+++ b/autotests/testHotspot/anqp_delay_test.py
@@ -23,10 +23,8 @@ class Test(unittest.TestCase):
rule0 = hwsim.rules.create()
rule0.source = bss_radio.addresses[0]
rule0.bidirectional = True
- # Force the case where ANQP does not finish before Connect() comes in
- rule0.delay = 100
- wd = IWD(True, '/tmp')
+ wd = IWD(True)
hapd = HostapdCLI(config='ssidHotspot.conf')
@@ -53,12 +51,15 @@ class Test(unittest.TestCase):
condition = 'not obj.connected'
wd.wait_for_object_condition(ordered_network.network_object, condition)
+ # Force the case where ANQP does not finish before Connect() comes in
+ rule0.delay = 100
+
ordered_network.network_object.connect()
rule0.delay = 1
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
testutil.test_iface_operstate()
testutil.test_ifaces_connected(device.name, hapd.ifname)
diff --git a/autotests/testHotspot/hessid_test.py b/autotests/testHotspot/hessid_test.py
index f511fd62..d22c5d61 100644
--- a/autotests/testHotspot/hessid_test.py
+++ b/autotests/testHotspot/hessid_test.py
@@ -42,8 +42,8 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
testutil.test_iface_operstate()
testutil.test_ifaces_connected(device.name, hapd.ifname)
diff --git a/autotests/testHotspot/hotspot_test.py b/autotests/testHotspot/hotspot_test.py
index 0f18fae1..4fb6b071 100644
--- a/autotests/testHotspot/hotspot_test.py
+++ b/autotests/testHotspot/hotspot_test.py
@@ -42,8 +42,8 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
testutil.test_iface_operstate()
testutil.test_ifaces_connected(device.name, hapd.ifname)
diff --git a/autotests/testHotspot/roaming_test.py b/autotests/testHotspot/roaming_test.py
index a7274a82..52d89f84 100644
--- a/autotests/testHotspot/roaming_test.py
+++ b/autotests/testHotspot/roaming_test.py
@@ -42,8 +42,8 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
testutil.test_iface_operstate()
testutil.test_ifaces_connected(device.name, hapd.ifname)
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 24/26] auto-t: fix testWPA2 to wait for device object
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (21 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 23/26] auto-t: harden testHotspot James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 25/26] auto-t: fix testPreauth-roam James Prestwood
` (2 subsequent siblings)
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 766 bytes --]
---
autotests/testWPA2/connection_test.py | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/autotests/testWPA2/connection_test.py b/autotests/testWPA2/connection_test.py
index dcb4722e..3ff9b50f 100644
--- a/autotests/testWPA2/connection_test.py
+++ b/autotests/testWPA2/connection_test.py
@@ -38,8 +38,8 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
testutil.test_iface_operstate()
testutil.test_ifaces_connected()
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 25/26] auto-t: fix testPreauth-roam
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (22 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 24/26] auto-t: fix testWPA2 to wait for device object James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:12 ` [PATCH v5 26/26] auto-t: fix testFT-PSK-roam James Prestwood
2020-09-10 23:21 ` [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write Denis Kenzior
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 1527 bytes --]
Changed to wait for DeviceState instead of network object as well
as moved hwsim rules ahead of the IWD initializer to avoid IWD
scanning before the rules are fully in place.
---
autotests/testPreauth-roam/connection_test.py | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/autotests/testPreauth-roam/connection_test.py b/autotests/testPreauth-roam/connection_test.py
index 839e04b0..5eee3078 100644
--- a/autotests/testPreauth-roam/connection_test.py
+++ b/autotests/testPreauth-roam/connection_test.py
@@ -44,14 +44,14 @@ class Test(unittest.TestCase):
bss_hostapd[1].set_neighbor(bss_radio[0].addresses[0], 'TestPreauth',
bss0_nr)
- wd = IWD()
-
- device = wd.list_devices(1)[0]
-
# Check that iwd selects BSS 0 first
rule0.signal = -2500
rule1.signal = -3500
+ wd = IWD()
+
+ device = wd.list_devices(1)[0]
+
condition = 'not obj.scanning'
wd.wait_for_object_condition(device, condition)
@@ -76,8 +76,8 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
self.assertTrue(bss_hostapd[0].list_sta())
self.assertFalse(bss_hostapd[1].list_sta())
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* [PATCH v5 26/26] auto-t: fix testFT-PSK-roam
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (23 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 25/26] auto-t: fix testPreauth-roam James Prestwood
@ 2020-09-10 23:12 ` James Prestwood
2020-09-10 23:21 ` [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write Denis Kenzior
25 siblings, 0 replies; 27+ messages in thread
From: James Prestwood @ 2020-09-10 23:12 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 3995 bytes --]
Switched around hwsim rules with the IWD initializer to avoid
IWD periodically scanning before hwsim rules are in place. Removed
some unneeded code during teardown.
---
autotests/testFT-PSK-roam/connection_test.py | 39 ++++++--------------
autotests/testFT-PSK-roam/hw.conf | 1 +
2 files changed, 13 insertions(+), 27 deletions(-)
diff --git a/autotests/testFT-PSK-roam/connection_test.py b/autotests/testFT-PSK-roam/connection_test.py
index 37dd80d0..ccbaccb8 100644
--- a/autotests/testFT-PSK-roam/connection_test.py
+++ b/autotests/testFT-PSK-roam/connection_test.py
@@ -24,7 +24,11 @@ class Test(unittest.TestCase):
rule1.source = self.bss_radio[1].addresses[0]
rule1.bidirectional = True
- wd = self.wd
+ # Check that iwd selects BSS 0 first
+ rule0.signal = -2000
+ rule1.signal = -2500
+
+ wd = IWD(True)
psk_agent = PSKAgent("EasilyGuessedPassword")
wd.register_psk_agent(psk_agent)
@@ -33,10 +37,6 @@ class Test(unittest.TestCase):
# prevent autoconnect
device.disconnect()
- # Check that iwd selects BSS 0 first
- rule0.signal = -2000
- rule1.signal = -2500
-
condition = 'not obj.scanning'
wd.wait_for_object_condition(device, condition)
@@ -61,8 +61,8 @@ class Test(unittest.TestCase):
ordered_network.network_object.connect()
- condition = 'obj.connected'
- wd.wait_for_object_condition(ordered_network.network_object, condition)
+ condition = 'obj.state == DeviceState.connected'
+ wd.wait_for_object_condition(device, condition)
self.assertTrue(self.bss_hostapd[0].list_sta())
self.assertFalse(self.bss_hostapd[1].list_sta())
@@ -109,7 +109,11 @@ class Test(unittest.TestCase):
rule1.source = self.bss_radio[1].addresses[0]
rule1.bidirectional = True
- wd = self.wd
+ # Check that iwd selects BSS 0 first
+ rule0.signal = -2000
+ rule1.signal = -2500
+
+ wd = IWD(True)
psk_agent = PSKAgent("EasilyGuessedPassword")
wd.register_psk_agent(psk_agent)
@@ -118,10 +122,6 @@ class Test(unittest.TestCase):
# prevent autoconnect
device.disconnect()
- # Check that iwd selects BSS 0 first
- rule0.signal = -2000
- rule1.signal = -2500
-
condition = 'not obj.scanning'
wd.wait_for_object_condition(device, condition)
@@ -180,8 +180,6 @@ class Test(unittest.TestCase):
testutil.test_ifaces_connected(self.bss_hostapd[1].ifname, device.name)
self.assertRaises(Exception, testutil.test_ifaces_connected,
(self.bss_hostapd[0].ifname, device.name))
- def setUp(self):
- self.wd = IWD()
def tearDown(self):
os.system('ifconfig "' + self.bss_hostapd[0].ifname + '" down')
@@ -189,19 +187,6 @@ class Test(unittest.TestCase):
os.system('ifconfig "' + self.bss_hostapd[0].ifname + '" up')
os.system('ifconfig "' + self.bss_hostapd[1].ifname + '" up')
- hwsim = Hwsim()
- device = self.wd.list_devices(1)[0]
- try:
- device.disconnect()
- except:
- pass
-
- condition = 'obj.state == DeviceState.disconnected'
- self.wd.wait_for_object_condition(device, condition)
-
- for rule in list(hwsim.rules.keys()):
- del hwsim.rules[rule]
-
@classmethod
def setUpClass(cls):
hwsim = Hwsim()
diff --git a/autotests/testFT-PSK-roam/hw.conf b/autotests/testFT-PSK-roam/hw.conf
index d2d50863..d08534c4 100644
--- a/autotests/testFT-PSK-roam/hw.conf
+++ b/autotests/testFT-PSK-roam/hw.conf
@@ -3,6 +3,7 @@ num_radios=3
max_test_exec_interval_sec=45
tmpfs_extra_stuff=main.conf
needs_hwsim=1
+start_iwd=0
[HOSTAPD]
rad0=ft-psk-ccmp-1.conf
--
2.26.2
^ permalink raw reply related [flat|nested] 27+ messages in thread* Re: [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write
2020-09-10 23:12 [PATCH v5 01/26] auto-t: prepare autotests for test-runner re-write James Prestwood
` (24 preceding siblings ...)
2020-09-10 23:12 ` [PATCH v5 26/26] auto-t: fix testFT-PSK-roam James Prestwood
@ 2020-09-10 23:21 ` Denis Kenzior
25 siblings, 0 replies; 27+ messages in thread
From: Denis Kenzior @ 2020-09-10 23:21 UTC (permalink / raw)
To: iwd
[-- Attachment #1: Type: text/plain, Size: 4152 bytes --]
Hi James,
On 9/10/20 6:12 PM, James Prestwood wrote:
> The tests basically remained the same with a few minor changes.
> The wiphy_map and in turn hostapd_map are no longer used. This
> was already partially converted a long time ago when the 'config'
> parameter was added to HostapdCLI. This patch fully converts all
> autotests to use 'config' rather than looking up by interface.
>
> Some test scripts were named 'test.py' which was fine before but
> the new rewrite actually loads each python test as a module. The
> name 'test' is too ambiguous and causes issues due to a native
> python module with the same name. All of these files were
> renamed to 'connection_test.py'.
> ---
> .../{test.py => connection_test.py} | 0
> .../testAP/{test.py => connection_test.py} | 9 ++--
> autotests/testAP/failure_test.py | 6 +--
> autotests/testAPRoam/connection_test.py | 2 -
> .../testBSSBlacklist/all_blacklisted_test.py | 4 --
> autotests/testBSSBlacklist/bad_pass_test.py | 4 --
> autotests/testBSSBlacklist/connection_test.py | 1 -
> .../testBSSBlacklist/temp_blacklist_test.py | 4 --
> .../connect_command_test.py | 42 +++++++------------
> .../disconnect_by_ap_test.py | 4 +-
> autotests/testEAP-AKA/connection_test.py | 1 -
> .../testEAP-PEAPv0-CryptoBinding/ISK_test.py | 1 -
> .../NoISK_test.py | 1 -
> .../testEAP-PEAPv0-CryptoBinding/main.conf | 2 -
> autotests/testEAP-PWD/connection_test.py | 1 -
> autotests/testEAP-SIM/connection_test.py | 1 -
> .../connection_test.py | 11 ++---
> .../testEAP-TTLS-CHAP/connection_test.py | 1 -
> .../testEAP-TTLS-MSCHAP/connection_test.py | 1 -
> autotests/testEAP-TTLS-PAP/connection_test.py | 1 -
> autotests/testEAP-WPS-Frag/wps_frag_test.py | 4 +-
> autotests/testEAP-WPS/four_digit_pin_test.py | 4 +-
> autotests/testEAP-WPS/pin_test.py | 4 +-
> autotests/testEAP-WPS/push_button_test.py | 5 +--
> autotests/testFILS/fils_256_test.py | 1 -
> autotests/testFILS/fils_384_test.py | 1 -
> .../{test.py => connection_test.py} | 2 +-
> .../{test.py => connection_test.py} | 2 +-
> .../{test.py => connection_test.py} | 2 +-
> .../{test.py => connection_test.py} | 2 +-
> .../{test.py => connection_test.py} | 17 +++++---
> .../{test.py => connection_test.py} | 2 +-
> .../{test.py => connection_test.py} | 2 +-
> autotests/testOWE/connection_test.py | 1 -
> autotests/testOWE/renegotiate_test.py | 1 -
> .../{test.py => connection_test.py} | 2 +-
> .../{test.py => connection_test.py} | 0
> autotests/testSAE/connection_test.py | 10 ++---
> autotests/testSAE/group_20_connection_test.py | 10 ++---
> .../testSAQuery-spoofing/connection_test.py | 2 +-
> autotests/testSAQuery/connection_test.py | 2 +-
> 41 files changed, 56 insertions(+), 117 deletions(-)
> rename autotests/testAP-no-support/{test.py => connection_test.py} (100%)
> rename autotests/testAP/{test.py => connection_test.py} (94%)
> delete mode 100644 autotests/testEAP-PEAPv0-CryptoBinding/main.conf
> rename autotests/testFT-8021x-roam/{test.py => connection_test.py} (99%)
> rename autotests/testFT-FILS-SHA256/{test.py => connection_test.py} (99%)
> rename autotests/testFT-FILS-SHA384/{test.py => connection_test.py} (99%)
> rename autotests/testFT-PSK-over-DS/{test.py => connection_test.py} (99%)
> rename autotests/testFT-PSK-roam/{test.py => connection_test.py} (96%)
> rename autotests/testFT-SAE-roam/{test.py => connection_test.py} (99%)
> rename autotests/testHT-VHT/{test.py => connection_test.py} (98%)
> rename autotests/testPreauth-roam/{test.py => connection_test.py} (98%)
> rename autotests/testRSSIAgent/{test.py => connection_test.py} (100%)
>
All applied, thanks.
Regards,
-Denis
^ permalink raw reply [flat|nested] 27+ messages in thread