All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 1/2] auto-t: add API wait_for_object_change
@ 2021-02-05 19:06 James Prestwood
  2021-02-05 19:06 ` [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change James Prestwood
  2021-02-05 20:44 ` [PATCH 1/2] auto-t: add API wait_for_object_change Denis Kenzior
  0 siblings, 2 replies; 7+ messages in thread
From: James Prestwood @ 2021-02-05 19:06 UTC (permalink / raw)
  To: iwd

[-- Attachment #1: Type: text/plain, Size: 2634 bytes --]

This is similar to wait_for_object_condition, but will not allow
any intermediate state changes between the initial and expected
conditions. This is useful for roaming tests when the expected
state change is 'connected' --> 'roaming' with no changes in
between.
---
 autotests/util/iwd.py | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)

diff --git a/autotests/util/iwd.py b/autotests/util/iwd.py
index 6b153ea7..8ea1dfb9 100755
--- a/autotests/util/iwd.py
+++ b/autotests/util/iwd.py
@@ -1034,6 +1034,48 @@ class IWD(AsyncOpAbstract):
     def wait_for_object_condition(self, *args, **kwargs):
         self._wait_for_object_condition(*args, **kwargs)
 
+    def wait_for_object_change(self, obj, from_str, to_str, max_wait = 50):
+        '''
+            Expects condition 'from_str' to evaluate true while waiting for 'to_str'. If
+            at any point during the wait 'from_str' evaluates false, an exception is
+            raised.
+
+            This allows an object to be checked for a state transition without any
+            intermediate state changes.
+        '''
+        self._wait_timed_out = False
+
+        def wait_timeout_cb():
+            self._wait_timed_out = True
+            return False
+
+        # Does initial condition pass?
+        if not eval(from_str):
+            raise Exception("initial condition [%s] not met" % from_str)
+
+        try:
+            timeout = GLib.timeout_add_seconds(max_wait, wait_timeout_cb)
+            context = ctx.mainloop.get_context()
+            while True:
+                context.iteration(may_block=True)
+
+                # If neither the initial or expected condition evaluate the
+                # object must be in another unexpected state.
+                if not eval(from_str) and not eval(to_str):
+                    raise Exception('unexpected condition between [%s] and [%s]' % from_str, to_str)
+
+                # Initial condition does not evaluate but expected does, pass
+                if not eval(from_str) and eval(to_str):
+                    break
+
+                if self._wait_timed_out and ctx.args.gdb == None:
+                    raise TimeoutError('[' + to_str + ']'\
+                                       ' condition was not met in '\
+                                       + str(max_wait) + ' sec')
+        finally:
+            if not self._wait_timed_out:
+                GLib.source_remove(timeout)
+
     def wait(self, time):
         self._wait_timed_out = False
         def wait_timeout_cb():
-- 
2.26.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change
  2021-02-05 19:06 [PATCH 1/2] auto-t: add API wait_for_object_change James Prestwood
@ 2021-02-05 19:06 ` James Prestwood
  2021-02-05 23:50   ` Andrew Zaborowski
  2021-02-05 20:44 ` [PATCH 1/2] auto-t: add API wait_for_object_change Denis Kenzior
  1 sibling, 1 reply; 7+ messages in thread
From: James Prestwood @ 2021-02-05 19:06 UTC (permalink / raw)
  To: iwd

[-- Attachment #1: Type: text/plain, Size: 10105 bytes --]

Every single roaming test had one of two problems with watching the
state change between roaming --> connected. Either the test used
wait_for_object_condition to wait for 'connected' which could allow
other states in between. Or it simply used an assert. The assert
wouldn't allow other state changes, but at the cost of potentially
failing due to IWD not having made it to the 'connected' state yet.

Now we have wait_for_object_change which takes two conditions:
initial (from_str) and expected (to_str). This API will not allow
any other conditions except these, and will wait for the expected
condition before continuing. This allows roaming test to reliably
wait for the roaming --> connected state change.
---
 autotests/testFT-8021x-roam/connection_test.py  | 8 +++-----
 autotests/testFT-FILS-SHA256/connection_test.py | 8 +++-----
 autotests/testFT-FILS-SHA384/connection_test.py | 6 +++---
 autotests/testFT-PSK-over-DS/connection_test.py | 8 +++-----
 autotests/testFT-PSK-roam/connection_test.py    | 7 +++----
 autotests/testFT-SAE-roam/connection_test.py    | 6 +++---
 autotests/testPreauth-roam/connection_test.py   | 6 +++---
 autotests/testRoamRetry/fast_retry_test.py      | 6 +++---
 autotests/testRoamRetry/stop_retry_test.py      | 6 +++---
 9 files changed, 27 insertions(+), 34 deletions(-)

diff --git a/autotests/testFT-8021x-roam/connection_test.py b/autotests/testFT-8021x-roam/connection_test.py
index 6e905f0d..c50c90ee 100644
--- a/autotests/testFT-8021x-roam/connection_test.py
+++ b/autotests/testFT-8021x-roam/connection_test.py
@@ -79,11 +79,9 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition)
-
-        condition = 'obj.state == DeviceState.connected'
-        wd.wait_for_object_condition(device, condition)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
         self.assertTrue(self.bss_hostapd[1].list_sta())
 
diff --git a/autotests/testFT-FILS-SHA256/connection_test.py b/autotests/testFT-FILS-SHA256/connection_test.py
index 267639cd..81201e08 100644
--- a/autotests/testFT-FILS-SHA256/connection_test.py
+++ b/autotests/testFT-FILS-SHA256/connection_test.py
@@ -109,11 +109,9 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition)
-
-        condition = 'obj.state == DeviceState.connected'
-        wd.wait_for_object_condition(device, condition)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
         self.assertTrue(self.bss_hostapd[1].list_sta())
 
diff --git a/autotests/testFT-FILS-SHA384/connection_test.py b/autotests/testFT-FILS-SHA384/connection_test.py
index 12a85e3f..81201e08 100644
--- a/autotests/testFT-FILS-SHA384/connection_test.py
+++ b/autotests/testFT-FILS-SHA384/connection_test.py
@@ -109,10 +109,10 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertEqual(device.state, iwd.DeviceState.connected)
         self.assertTrue(self.bss_hostapd[1].list_sta())
 
         testutil.test_iface_operstate(device.name)
diff --git a/autotests/testFT-PSK-over-DS/connection_test.py b/autotests/testFT-PSK-over-DS/connection_test.py
index b6cad5a4..ad186d07 100644
--- a/autotests/testFT-PSK-over-DS/connection_test.py
+++ b/autotests/testFT-PSK-over-DS/connection_test.py
@@ -85,11 +85,9 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition)
-
-        condition = 'obj.state == DeviceState.connected'
-        wd.wait_for_object_condition(device, condition)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
         self.assertTrue(self.bss_hostapd[1].list_sta())
 
diff --git a/autotests/testFT-PSK-roam/connection_test.py b/autotests/testFT-PSK-roam/connection_test.py
index a7acf495..c0eec0ae 100644
--- a/autotests/testFT-PSK-roam/connection_test.py
+++ b/autotests/testFT-PSK-roam/connection_test.py
@@ -87,11 +87,10 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
-        condition = 'obj.state == DeviceState.connected'
-        wd.wait_for_object_condition(device, condition)
         self.assertTrue(self.bss_hostapd[1].list_sta())
 
         testutil.test_iface_operstate(device.name)
diff --git a/autotests/testFT-SAE-roam/connection_test.py b/autotests/testFT-SAE-roam/connection_test.py
index a8d1da6c..f2c694dd 100644
--- a/autotests/testFT-SAE-roam/connection_test.py
+++ b/autotests/testFT-SAE-roam/connection_test.py
@@ -91,15 +91,15 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
         rule1.signal = -2000
 
         # wait for IWD's signal levels to recover
         wd.wait(5)
 
-        self.assertEqual(device.state, iwd.DeviceState.connected)
         self.assertTrue(self.bss_hostapd[1].list_sta())
 
         testutil.test_iface_operstate(device.name)
diff --git a/autotests/testPreauth-roam/connection_test.py b/autotests/testPreauth-roam/connection_test.py
index 5eee3078..8a50abb5 100644
--- a/autotests/testPreauth-roam/connection_test.py
+++ b/autotests/testPreauth-roam/connection_test.py
@@ -97,10 +97,10 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertEqual(device.state, iwd.DeviceState.connected)
         self.assertTrue(bss_hostapd[1].list_sta())
 
         testutil.test_iface_operstate(device.name)
diff --git a/autotests/testRoamRetry/fast_retry_test.py b/autotests/testRoamRetry/fast_retry_test.py
index 32d80002..ce2c5603 100644
--- a/autotests/testRoamRetry/fast_retry_test.py
+++ b/autotests/testRoamRetry/fast_retry_test.py
@@ -117,10 +117,10 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition, max_wait=5)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertEqual(device.state, iwd.DeviceState.connected)
         self.assertTrue(bss_hostapd[1].list_sta())
 
         device.disconnect()
diff --git a/autotests/testRoamRetry/stop_retry_test.py b/autotests/testRoamRetry/stop_retry_test.py
index a2729d6c..38f58398 100644
--- a/autotests/testRoamRetry/stop_retry_test.py
+++ b/autotests/testRoamRetry/stop_retry_test.py
@@ -92,10 +92,10 @@ class Test(unittest.TestCase):
 
         # Check that iwd is on BSS 1 once out of roaming state and doesn't
         # go through 'disconnected', 'autoconnect', 'connecting' in between
-        condition = 'obj.state != DeviceState.roaming'
-        wd.wait_for_object_condition(device, condition)
+        from_condition = 'obj.state == DeviceState.roaming'
+        to_condition = 'obj.state == DeviceState.connected'
+        wd.wait_for_object_change(device, from_condition, to_condition)
 
-        self.assertEqual(device.state, iwd.DeviceState.connected)
         self.assertTrue(bss_hostapd[1].list_sta())
 
         # Now make sure that we don't roam anymore. In order to catch this via
-- 
2.26.2

^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH 1/2] auto-t: add API wait_for_object_change
  2021-02-05 19:06 [PATCH 1/2] auto-t: add API wait_for_object_change James Prestwood
  2021-02-05 19:06 ` [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change James Prestwood
@ 2021-02-05 20:44 ` Denis Kenzior
  1 sibling, 0 replies; 7+ messages in thread
From: Denis Kenzior @ 2021-02-05 20:44 UTC (permalink / raw)
  To: iwd

[-- Attachment #1: Type: text/plain, Size: 501 bytes --]

Hi James,

On 2/5/21 1:06 PM, James Prestwood wrote:
> This is similar to wait_for_object_condition, but will not allow
> any intermediate state changes between the initial and expected
> conditions. This is useful for roaming tests when the expected
> state change is 'connected' --> 'roaming' with no changes in
> between.
> ---
>   autotests/util/iwd.py | 42 ++++++++++++++++++++++++++++++++++++++++++
>   1 file changed, 42 insertions(+)
> 

All applied, thanks.

Regards,
-Denis

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change
  2021-02-05 19:06 ` [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change James Prestwood
@ 2021-02-05 23:50   ` Andrew Zaborowski
  2021-02-05 23:58     ` James Prestwood
  0 siblings, 1 reply; 7+ messages in thread
From: Andrew Zaborowski @ 2021-02-05 23:50 UTC (permalink / raw)
  To: iwd

[-- Attachment #1: Type: text/plain, Size: 720 bytes --]

Hi James,

On Fri, 5 Feb 2021 at 20:06, James Prestwood <prestwoj@gmail.com> wrote:
> Every single roaming test had one of two problems with watching the
> state change between roaming --> connected. Either the test used
> wait_for_object_condition to wait for 'connected' which could allow
> other states in between. Or it simply used an assert. The assert
> wouldn't allow other state changes, but at the cost of potentially
> failing due to IWD not having made it to the 'connected' state yet.

Can you give an example of a sequence of states?  I can't see a
scenario where there would be a difference between the state before
ce5e1e1933dbb11c503702ca0ee7a3d0f142ff4b and after this patch.

Best regards

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change
  2021-02-05 23:50   ` Andrew Zaborowski
@ 2021-02-05 23:58     ` James Prestwood
  2021-02-08 13:16       ` Andrew Zaborowski
  0 siblings, 1 reply; 7+ messages in thread
From: James Prestwood @ 2021-02-05 23:58 UTC (permalink / raw)
  To: iwd

[-- Attachment #1: Type: text/plain, Size: 1341 bytes --]

Hi Andrew,

On Sat, 2021-02-06 at 00:50 +0100, Andrew Zaborowski wrote:
> Hi James,
> 
> On Fri, 5 Feb 2021 at 20:06, James Prestwood <prestwoj@gmail.com>
> wrote:
> > Every single roaming test had one of two problems with watching the
> > state change between roaming --> connected. Either the test used
> > wait_for_object_condition to wait for 'connected' which could allow
> > other states in between. Or it simply used an assert. The assert
> > wouldn't allow other state changes, but at the cost of potentially
> > failing due to IWD not having made it to the 'connected' state yet.
> 
> Can you give an example of a sequence of states?  I can't see a
> scenario where there would be a difference between the state before
> ce5e1e1933dbb11c503702ca0ee7a3d0f142ff4b and after this patch.

Commit ce5e1e1 was a mistake, although some FT tests already did this.
As the comments state in most FT tests we want to make sure IWD doesnt
fully disconnect e.g. because roaming failed. Using
wait_for_object_condition allows any state in between so we don't want
that, and a simple assert may not allow enough time for IWD to fully
roam. Hence this API which waits for state without allowing
intermediate states.

Not sure of any example besides roaming->disconnected->connected.

Thanks,
James
> 
> Best regards

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change
  2021-02-05 23:58     ` James Prestwood
@ 2021-02-08 13:16       ` Andrew Zaborowski
  2021-02-08 16:54         ` James Prestwood
  0 siblings, 1 reply; 7+ messages in thread
From: Andrew Zaborowski @ 2021-02-08 13:16 UTC (permalink / raw)
  To: iwd

[-- Attachment #1: Type: text/plain, Size: 1403 bytes --]

On Sat, 6 Feb 2021 at 00:59, James Prestwood <prestwoj@gmail.com> wrote:
> On Sat, 2021-02-06 at 00:50 +0100, Andrew Zaborowski wrote:
> > Hi James,
> >
> > On Fri, 5 Feb 2021 at 20:06, James Prestwood <prestwoj@gmail.com>
> > wrote:
> > > Every single roaming test had one of two problems with watching the
> > > state change between roaming --> connected. Either the test used
> > > wait_for_object_condition to wait for 'connected' which could allow
> > > other states in between. Or it simply used an assert. The assert
> > > wouldn't allow other state changes, but at the cost of potentially
> > > failing due to IWD not having made it to the 'connected' state yet.
> >
> > Can you give an example of a sequence of states?  I can't see a
> > scenario where there would be a difference between the state before
> > ce5e1e1933dbb11c503702ca0ee7a3d0f142ff4b and after this patch.
>
> Commit ce5e1e1 was a mistake, although some FT tests already did this.
> As the comments state in most FT tests we want to make sure IWD doesnt
> fully disconnect e.g. because roaming failed. Using
> wait_for_object_condition allows any state in between so we don't want
> that, and a simple assert may not allow enough time for IWD to fully
> roam.

Can this really happen, since the assert follows a
wd.wait_for_object_condition(device, 'obj.state !=
DeviceState.roaming')?

Best regards

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change
  2021-02-08 13:16       ` Andrew Zaborowski
@ 2021-02-08 16:54         ` James Prestwood
  0 siblings, 0 replies; 7+ messages in thread
From: James Prestwood @ 2021-02-08 16:54 UTC (permalink / raw)
  To: iwd

[-- Attachment #1: Type: text/plain, Size: 2081 bytes --]

On Mon, 2021-02-08 at 14:16 +0100, Andrew Zaborowski wrote:
> On Sat, 6 Feb 2021 at 00:59, James Prestwood <prestwoj@gmail.com>
> wrote:
> > On Sat, 2021-02-06 at 00:50 +0100, Andrew Zaborowski wrote:
> > > Hi James,
> > > 
> > > On Fri, 5 Feb 2021 at 20:06, James Prestwood <prestwoj@gmail.com>
> > > wrote:
> > > > Every single roaming test had one of two problems with watching
> > > > the
> > > > state change between roaming --> connected. Either the test
> > > > used
> > > > wait_for_object_condition to wait for 'connected' which could
> > > > allow
> > > > other states in between. Or it simply used an assert. The
> > > > assert
> > > > wouldn't allow other state changes, but at the cost of
> > > > potentially
> > > > failing due to IWD not having made it to the 'connected' state
> > > > yet.
> > > 
> > > Can you give an example of a sequence of states?  I can't see a
> > > scenario where there would be a difference between the state
> > > before
> > > ce5e1e1933dbb11c503702ca0ee7a3d0f142ff4b and after this patch.
> > 
> > Commit ce5e1e1 was a mistake, although some FT tests already did
> > this.
> > As the comments state in most FT tests we want to make sure IWD
> > doesnt
> > fully disconnect e.g. because roaming failed. Using
> > wait_for_object_condition allows any state in between so we don't
> > want
> > that, and a simple assert may not allow enough time for IWD to
> > fully
> > roam.
> 
> Can this really happen, since the assert follows a
> wd.wait_for_object_condition(device, 'obj.state !=
> DeviceState.roaming')?

Yeah your right. I'm not sure what I was seeing then with that assert
failing. Probably another unrelated issue with the tests I guess. Well
its merged now... I do think having this API is more explicit and cuts
down slightly on the code. I think if I included a wait for the initial
condition too we could also avoid waiting for 'connected' before which
would make this connected->roaming check be a single API call.

Anyways, thanks for pointing that out.

> 
> Best regards

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2021-02-08 16:54 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-05 19:06 [PATCH 1/2] auto-t: add API wait_for_object_change James Prestwood
2021-02-05 19:06 ` [PATCH 2/2] auto-t: update roaming tests with wait_for_object_change James Prestwood
2021-02-05 23:50   ` Andrew Zaborowski
2021-02-05 23:58     ` James Prestwood
2021-02-08 13:16       ` Andrew Zaborowski
2021-02-08 16:54         ` James Prestwood
2021-02-05 20:44 ` [PATCH 1/2] auto-t: add API wait_for_object_change Denis Kenzior

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.