Problems about handling duplicate packets when frame retry is enabled

Problems about handling duplicate packets when frame retry is enabled

[Xue, Wenqian]

Hi, all!

I’m using two raspberry pis with openlabs 802.15.4 radio hooked up for some experiments (Linux kernel is v4.7). Some stats like RSSI and LQI are collected and recorded by modifying wpan-ping.c. During the tests, I find some problems about handling duplicate packets.

When frame retry is enabled, it is found that the 802.15.4 module/Linux kernel could not discard duplicates, and would send the duplicates to the upper level (wpan-ping level). I check the mac receive code (/net/mac802154/rx.c), the functions __ieee802154_rx_handle_packet() and ieee802154_subif_frame() have no sub-function to handle duplicates. I also see that the similar function has been implemented in /net/mac80211/rx.c, it is the function ieee80211_rx_h_check_dup().
The duplicates need to be discarded for our experiments, the results will become more complicated otherwise. I add some codes in the function ieee802154_subif_frame(), the added codes are as below:

//part of codes in the function ieee802154_subif_frame()
switch (mac_cb(skb)->type) {
case IEEE802154_FC_TYPE_DATA:
+if (hdr->seq != sdata->last_seq) {
+    sdata->last_seq == hdr->seq;
+    return ieee802154_deliver_skb(skb);
+} else {
+    break;
+}
default:
pr_warn("ieee802154: bad frame received (type = %d)\n", mac_cb(skb)->type);
mac_cb(skb)->type);
goto fail;
//----------------------------------------------------------

The main mechanism to handle duplicates is through the comparison of sequence number between current packet (indicated by seq) and last one (indicated by last_seq). The variable last_seq is defined in the struct sdata. By the modification, the duplicates are indeed no longer received on upper layer. I'm a newbie for linux kernel, I'd like to know if the modification is ok?
If there are errors or will lead to any unexpected affects to the kernel, please feel free to point it out, thank you very much!

Best Regards,
Xue Wenqian

Re: Problems about handling duplicate packets when frame retry is enabled

[Alexander Aring]

Hi,

On 08/08/2016 11:11 AM, Xue, Wenqian wrote:
> Hi, all!
> 
> I’m using two raspberry pis with openlabs 802.15.4 radio hooked up for some experiments (Linux kernel is v4.7). Some stats like RSSI and LQI are collected and recorded by modifying wpan-ping.c. During the tests, I find some problems about handling duplicate packets.
> 
> When frame retry is enabled, it is found that the 802.15.4 module/Linux kernel could not discard duplicates, and would send the duplicates to the upper level (wpan-ping level). I check the mac receive code (/net/mac802154/rx.c), the functions __ieee802154_rx_handle_packet() and ieee802154_subif_frame() have no sub-function to handle duplicates. I also see that the similar function has been implemented in /net/mac80211/rx.c, it is the function ieee80211_rx_h_check_dup().
> The duplicates need to be discarded for our experiments, the results will become more complicated otherwise. I add some codes in the function ieee802154_subif_frame(), the added codes are as below:
> 
> //part of codes in the function ieee802154_subif_frame()
> switch (mac_cb(skb)->type) {
> case IEEE802154_FC_TYPE_DATA:
> +if (hdr->seq != sdata->last_seq) {
> +    sdata->last_seq == hdr->seq;
> +    return ieee802154_deliver_skb(skb);
> +} else {
> +    break;
> +}
> default:
> pr_warn("ieee802154: bad frame received (type = %d)\n", mac_cb(skb)->type);
> mac_cb(skb)->type);
> goto fail;
> //----------------------------------------------------------
> 
> The main mechanism to handle duplicates is through the comparison of sequence number between current packet (indicated by seq) and last one (indicated by last_seq). The variable last_seq is defined in the struct sdata. By the modification, the duplicates are indeed no longer received on upper layer. I'm a newbie for linux kernel, I'd like to know if the modification is ok?
> If there are errors or will lead to any unexpected affects to the kernel, please feel free to point it out, thank you very much!
> 

This smells something that for what reason ever the Linux transceiver cannot
handle AACK anymore.

Do you using monitor interfaces for receiving? This could be the reason
why AACK is disabled.

- Alex

Re: 答复: Problems about handling duplicate packets when frame retry is enabled

[Alexander Aring]

Hi,

On 08/11/2016 11:50 AM, Xue, Wenqian wrote:
> Hi, Alexander,
> 
> Thank you for your quick reply.
> I'm not sure if AACK is disabled (how can I know it?), but I'm using node interface not monitor interface.
>

yea, I should get it before because monitor interfaces doesn't run the
code which you changed. But monitor interfaces goes into promiscuous
mode and AACK is disabled there.

> When without kernel modification and setting frame retries=3 for both RPis, lots of error 'sequence number did not match" will occur. While, when with the modification to deal with duplicates, the error will not occur any more. I'm not sure if AACK could discard duplicates automatically.
> Thank you!
> 

Your modification handles in duplicates in a form which is not
acceptable. It will simple drop duplicates but how frame retries works
is the following:

1. Node A transmits frame for Node B with ackrequest bit set.
2. Node B receive that frame and see's "ackrequest bit is set" and
   sending an ACK back.
3. If Node A received an ack for the sequence number then no
   retransmission will occur.
OR
3. If Node A doesn't received an ack it will retransmit the frame.


On your nodes and if you are really using two at86rf233 transceivers,
this mechanism is completely done by hardware.

Transmit: ARET
Receive: AACK

On the receive side it's always imporant that the receiving node
supports AACK handling if Nodes sending frames with "ackrequest bit"
set.

---

First, you should sniff somehow the traffic if it's possible, to see
what's going on the traffic and you see that Node B sends ACKs back, if
not -> missing AACK handling.

On linux the openlabs transceiver (at86rf233) supports AACK handling...
don't know if you break it with your changes.

---

Also I detected that the ieee802154 sockets doesn't care about "iwpan
dev wpan0 set ackreq_default" setting [0]. This socket interface should
only be used as DGRAM sockets and there is a big TODO to make an
enhanced version of this socket interface. :-)

But that you have retransmits smells that the [0] is set.

---

I am here totally confused that you don't have AACK handling which the
at86rf233 supports, do you really using two openlabs with mainline Linux
kernel? Or do you talking with some contiki/RIOT etc. node?

- Alex

[0] http://lxr.free-electrons.com/source/net/ieee802154/socket.c#L667

Re: 答复: 答复: Problems about handling duplicate packets when frame retry is enabled

[Alexander Aring]

Hi,

On 08/19/2016 03:10 PM, Xue, Wenqian wrote:
> Hi, Alexander,
> 
> Sorry for so late reply, as I have some other business at hand.
> 
>> -----邮件原件-----
>> 发件人: Alexander Aring [mailto:aar@pengutronix.de]
>> 发送时间: 2016年8月13日 0:29
>> 收件人: Xue, Wenqian/薛 文倩 <xuewenqian@cn.fujitsu.com>
>> 抄送: linux-wpan@vger.kernel.org
>> 主题: Re: 答复: Problems about handling duplicate packets when frame retry is
>> enabled
>>
>>
...
>>
>> Your modification handles in duplicates in a form which is not acceptable. It will
>> simple drop duplicates but how frame retries works is the following:
> 
> If your meaning for my modification is that the result will be the same whether retry is enabled or not? Initially, my motivation is to prevent the received retry packet from being transmitted up to the wpan-ping level, then to avoid the error "Sequence number did not match".
> 

I looked deeper for that handling in 802.15.4-2011 (which is the version
which is currently free available):

5.1.6.6 Transmission scenarios

Your scenario is "Lost acknowledgment frame.", I don't see anything
which let me accept your patch to drop multiple received frames with the
same sequence number. The receiving node mac802154 stack is "recipient
next layer".

Or did I miss something?

>>
>> 1. Node A transmits frame for Node B with ackrequest bit set.
>> 2. Node B receive that frame and see's "ackrequest bit is set" and
>>   sending an ACK back.
>> 3. If Node A received an ack for the sequence number then no
>>   retransmission will occur.
>> OR
>> 3. If Node A doesn't received an ack it will retransmit the frame.
>>
>>
>> On your nodes and if you are really using two at86rf233 transceivers, this
>> mechanism is completely done by hardware.
>>
>> Transmit: ARET
>> Receive: AACK
>>
>> On the receive side it's always imporant that the receiving node supports AACK
>> handling if Nodes sending frames with "ackrequest bit"
>> set.
>>
>> ---
>>
>> First, you should sniff somehow the traffic if it's possible, to see what's going on
>> the traffic and you see that Node B sends ACKs back, if not -> missing AACK
>> handling.
>>
>> On linux the openlabs transceiver (at86rf233) supports AACK handling...
>> don't know if you break it with your changes.
>>
> 
> I get it for your detailed description for frame re-transmission mechanism. And yes, I really use the two at86rf233 transceivers, I also used an ieee802.15.4 sniffer to sniff the traffic, ackrequest bit is indeed set, and I could see the captured ack.
> 
> I just did some tests with the kernel v4.7 (without my duplicate modification), I did 3 cases, as below and the attachment:
> 1. set the max_frame_retries=3 for nodeA and nodeB, lots of "Sequence number did not match" errors occur during the test
> 2. set the max_frame_retries=0 for nodeA, and set the max_frame_retries=3 for nodeB, lots of "Sequence number did not match" errors occur during the test
> 3. set the max_frame_retries=3 for nodeA, and set the max_frame_retries=0 for nodeB, no above error occur
> In my goal, the retry needs to be enabled for both sides, so I'd like to know the root cause for such case.
> I also made a simple description for the test cases to make you clearer for my problem, a few typical captured packets are shown in the attachment. 
> Hope you could give some suggestions for the problem, thank you very much!
> 

Sure, retransmits can be 3 at both sides. But somehow you have a missing
acknowledgment frame. The sniff example shows me this was never send.

It could be that you transmit very fast and a force state change aborts
the transmitting of ACK frame in RX_AACK_BUSY state.

Can you try this:

diff --git a/drivers/net/ieee802154/at86rf230.c b/drivers/net/ieee802154/at86rf230.c
index 9f10da6..97f75b5 100644
--- a/drivers/net/ieee802154/at86rf230.c
+++ b/drivers/net/ieee802154/at86rf230.c
@@ -64,7 +64,7 @@ struct at86rf2xx_chip_data {
  *
  * We assume the max_frame_retries (7) value of 802.15.4 here.
  */
-#define AT86RF2XX_MAX_TX_RETRIES       7
+#define AT86RF2XX_MAX_TX_RETRIES       20
 /* We use the recommended 5 minutes timeout to recalibrate */
 #define AT86RF2XX_CAL_LOOP_TIMEOUT     (5 * 60 * HZ)

>> ---
>>
>> Also I detected that the ieee802154 sockets doesn't care about "iwpan dev
>> wpan0 set ackreq_default" setting [0]. This socket interface should only be used
>> as DGRAM sockets and there is a big TODO to make an enhanced version of this
>> socket interface. :-)
>>
>> But that you have retransmits smells that the [0] is set.
>>
> 
> Actually, I don't set the ackreq_default, just keep it as default.
> 
>> ---
>>
>> I am here totally confused that you don't have AACK handling which the
>> at86rf233 supports, do you really using two openlabs with mainline Linux
>> kernel? Or do you talking with some contiki/RIOT etc. node?
> 
> Sorry to make you so confused, if the email still confused you, feel free to email me. Thank you!
> 

No, I am confused because the ack handling which is handled by _hardware_
isn't working at your side, I think it's because the force state change
and I mess something with the timing settings. If so, then sorry. :-/

It should work everything with AACK handling when it's done by hardware.

- Alex

Re: 答复: 答复: Problems about handling duplicate packets when frame retry is enabled

[Alexander Aring]

Hi,

On 08/20/2016 11:36 PM, Alexander Aring wrote:
> 
> Hi,
> 
> On 08/19/2016 03:10 PM, Xue, Wenqian wrote:
>> Hi, Alexander,
>>
>> Sorry for so late reply, as I have some other business at hand.
>>
>>> -----邮件原件-----
>>> 发件人: Alexander Aring [mailto:aar@pengutronix.de]
>>> 发送时间: 2016年8月13日 0:29
>>> 收件人: Xue, Wenqian/薛 文倩 <xuewenqian@cn.fujitsu.com>
>>> 抄送: linux-wpan@vger.kernel.org
>>> 主题: Re: 答复: Problems about handling duplicate packets when frame retry is
>>> enabled
>>>
>>>
> ...
>>>
>>> Your modification handles in duplicates in a form which is not acceptable. It will
>>> simple drop duplicates but how frame retries works is the following:
>>
>> If your meaning for my modification is that the result will be the same whether retry is enabled or not? Initially, my motivation is to prevent the received retry packet from being transmitted up to the wpan-ping level, then to avoid the error "Sequence number did not match".
>>
> 
> I looked deeper for that handling in 802.15.4-2011 (which is the version
> which is currently free available):
> 
> 5.1.6.6 Transmission scenarios
> 
> Your scenario is "Lost acknowledgment frame.", I don't see anything
> which let me accept your patch to drop multiple received frames with the
> same sequence number. The receiving node mac802154 stack is "recipient
> next layer".
> 
> Or did I miss something?
> 

I see currently 802.11 has some handling which you like to get in.
See [0]. But I don't see that such handling is described in 802.15.4 :-/

For me it's clean that it can be that duplicate frames can reach the
next layer and the next layer need to care about that. That's what
"Transmission scenario" - "Lost acknowledgment frame" shows me.

- Alex

[0] http://lxr.free-electrons.com/source/net/mac80211/rx.c#L1143

Re: 答复: 答复: 答复: Problems about handling duplicate packets when frame retry is enabled

[Alexander Aring]

On 09/04/2016 05:07 PM, Xue, Wenqian wrote:
> Hi, Alexander
> 
>> -----邮件原件-----
>> 发件人: Alexander Aring [mailto:aar@pengutronix.de]
>> 发送时间: 2016年9月1日 23:05
>> 收件人: Xue, Wenqian/薛 文倩 <xuewenqian@cn.fujitsu.com>
>> 抄送: linux-wpan@vger.kernel.org
>> 主题: Re: 答复: 答复: Problems about handling duplicate packets when frame
>> retry is enabled
>>
>>
>> Hi,
>>
>> On 08/20/2016 11:36 PM, Alexander Aring wrote:
>>>
>>> Hi,
>>>
>>> On 08/19/2016 03:10 PM, Xue, Wenqian wrote:
>>>> Hi, Alexander,
>>>>
>>>> Sorry for so late reply, as I have some other business at hand.
>>>>
>>>>> -----邮件原件-----
>>>>> 发件人: Alexander Aring [mailto:aar@pengutronix.de]
>>>>> 发送时间: 2016年8月13日 0:29
>>>>> 收件人: Xue, Wenqian/薛 文倩 <xuewenqian@cn.fujitsu.com>
>>>>> 抄送: linux-wpan@vger.kernel.org
>>>>> 主题: Re: 答复: Problems about handling duplicate packets when frame
>>>>> retry is enabled
>>>>>
>>>>>
>>> ...
>>>>>
>>>>> Your modification handles in duplicates in a form which is not
>>>>> acceptable. It will simple drop duplicates but how frame retries works is the
>> following:
>>>>
>>>> If your meaning for my modification is that the result will be the same
>> whether retry is enabled or not? Initially, my motivation is to prevent the
>> received retry packet from being transmitted up to the wpan-ping level, then to
>> avoid the error "Sequence number did not match".
>>>>
>>>
>>> I looked deeper for that handling in 802.15.4-2011 (which is the
>>> version which is currently free available):
>>>
>>> 5.1.6.6 Transmission scenarios
>>>
>>> Your scenario is "Lost acknowledgment frame.", I don't see anything
>>> which let me accept your patch to drop multiple received frames with
>>> the same sequence number. The receiving node mac802154 stack is
>>> "recipient next layer".
>>>
>>> Or did I miss something?
>>>
>>
>> I see currently 802.11 has some handling which you like to get in.
>> See [0]. But I don't see that such handling is described in 802.15.4 :-/
>>
>> For me it's clean that it can be that duplicate frames can reach the next layer
>> and the next layer need to care about that. That's what "Transmission scenario"
>> - "Lost acknowledgment frame" shows me.
>>
> I also checked the spec when I met the problem first, the spec really does not mention the handling for the duplicates. I thought it could be implemented in reality and should prevent the duplicates to the next layer. If according to your opinion " the next layer need to care about that ", the wpan-ping.c should care about the duplicates, right? Thank you!
> 

I think yes, some upper layers need to care about that or not. Here you
have like in "ping6" also a duplicate warning which indicates that you
received duplicates and _maybe_ there exists an issue in the lower
layers and I think you really should try to detect why you have an lost
acknowledgment frame.

- Alex

Re: 答复: 答复: 答复: Problems about handling duplicate packets when frame retry is enabled

[Alexander Aring]

Hi,

On 09/04/2016 05:06 PM, Xue, Wenqian wrote:
> Hi, Alexander
> Sorry for so late reply, and thank you for your kind attention.
> 
>> -----邮件原件-----
>> 发件人: Alexander Aring [mailto:aar@pengutronix.de]
>> 发送时间: 2016年8月21日 5:37
>> 收件人: Xue, Wenqian/薛 文倩 <xuewenqian@cn.fujitsu.com>
>> 抄送: linux-wpan@vger.kernel.org
>> 主题: Re: 答复: 答复: Problems about handling duplicate packets when frame
>> retry is enabled
>>
>>
>> Hi,
>>
>> On 08/19/2016 03:10 PM, Xue, Wenqian wrote:
>>> Hi, Alexander,
>>>
>>> Sorry for so late reply, as I have some other business at hand.
>>>
>>>> -----邮件原件-----
>>>> 发件人: Alexander Aring [mailto:aar@pengutronix.de]
>>>> 发送时间: 2016年8月13日 0:29
>>>> 收件人: Xue, Wenqian/薛 文倩 <xuewenqian@cn.fujitsu.com>
>>>> 抄送: linux-wpan@vger.kernel.org
>>>> 主题: Re: 答复: Problems about handling duplicate packets when frame
>>>> retry is enabled
>>>>
>>>>
>> ...
>>>>
>>>> Your modification handles in duplicates in a form which is not
>>>> acceptable. It will simple drop duplicates but how frame retries works is the
>> following:
>>>
>>> If your meaning for my modification is that the result will be the same
>> whether retry is enabled or not? Initially, my motivation is to prevent the
>> received retry packet from being transmitted up to the wpan-ping level, then to
>> avoid the error "Sequence number did not match".
>>>
>>
>> I looked deeper for that handling in 802.15.4-2011 (which is the version which
>> is currently free available):
>>
>> 5.1.6.6 Transmission scenarios
>>
>> Your scenario is "Lost acknowledgment frame.", I don't see anything which let
>> me accept your patch to drop multiple received frames with the same sequence
>> number. The receiving node mac802154 stack is "recipient next layer".
>>
>> Or did I miss something?
>>
>>>>
>>>> 1. Node A transmits frame for Node B with ackrequest bit set.
>>>> 2. Node B receive that frame and see's "ackrequest bit is set" and
>>>>   sending an ACK back.
>>>> 3. If Node A received an ack for the sequence number then no
>>>>   retransmission will occur.
>>>> OR
>>>> 3. If Node A doesn't received an ack it will retransmit the frame.
>>>>
>>>>
>>>> On your nodes and if you are really using two at86rf233 transceivers,
>>>> this mechanism is completely done by hardware.
>>>>
>>>> Transmit: ARET
>>>> Receive: AACK
>>>>
>>>> On the receive side it's always imporant that the receiving node
>>>> supports AACK handling if Nodes sending frames with "ackrequest bit"
>>>> set.
>>>>
>>>> ---
>>>>
>>>> First, you should sniff somehow the traffic if it's possible, to see
>>>> what's going on the traffic and you see that Node B sends ACKs back,
>>>> if not -> missing AACK handling.
>>>>
>>>> On linux the openlabs transceiver (at86rf233) supports AACK handling...
>>>> don't know if you break it with your changes.
>>>>
>>>
>>> I get it for your detailed description for frame re-transmission mechanism.
>> And yes, I really use the two at86rf233 transceivers, I also used an ieee802.15.4
>> sniffer to sniff the traffic, ackrequest bit is indeed set, and I could see the
>> captured ack.
>>>
>>> I just did some tests with the kernel v4.7 (without my duplicate modification),
>> I did 3 cases, as below and the attachment:
>>> 1. set the max_frame_retries=3 for nodeA and nodeB, lots of "Sequence
>>> number did not match" errors occur during the test 2. set the
>>> max_frame_retries=0 for nodeA, and set the max_frame_retries=3 for
>>> nodeB, lots of "Sequence number did not match" errors occur during the test
>> 3. set the max_frame_retries=3 for nodeA, and set the max_frame_retries=0 for
>> nodeB, no above error occur In my goal, the retry needs to be enabled for both
>> sides, so I'd like to know the root cause for such case.
>>> I also made a simple description for the test cases to make you clearer for my
>> problem, a few typical captured packets are shown in the attachment.
>>> Hope you could give some suggestions for the problem, thank you very much!
>>>
>>
>> Sure, retransmits can be 3 at both sides. But somehow you have a missing
>> acknowledgment frame. The sniff example shows me this was never send.
>>
>> It could be that you transmit very fast and a force state change aborts the
>> transmitting of ACK frame in RX_AACK_BUSY state.
>>
>> Can you try this:
>>
>> diff --git a/drivers/net/ieee802154/at86rf230.c
>> b/drivers/net/ieee802154/at86rf230.c
>> index 9f10da6..97f75b5 100644
>> --- a/drivers/net/ieee802154/at86rf230.c
>> +++ b/drivers/net/ieee802154/at86rf230.c
>> @@ -64,7 +64,7 @@ struct at86rf2xx_chip_data {
>>  *
>>  * We assume the max_frame_retries (7) value of 802.15.4 here.
>>  */
>> -#define AT86RF2XX_MAX_TX_RETRIES       7
>> +#define AT86RF2XX_MAX_TX_RETRIES       20
>> /* We use the recommended 5 minutes timeout to recalibrate */
>> #define AT86RF2XX_CAL_LOOP_TIMEOUT     (5 * 60 * HZ)
>>
> I tried the test, but the result is still the same, lots of errors "Sequence number did not match" occur.
> Actually, I made some modification for at86rf230.c and wpan-ping.c for our experiment. To figure out the problem, I also tried other tests:
> 1. without any modification for at86rf230.c and wpan-ping.c, and enable the max_frame_retries (3) for both sides, the error "Sequence number did not match" also occurs, but not so frequently, it goes well overall.
> 2. with my modification for at86rf230.c, without the modification for wpan-ping.c, and enable the max_frame_retries (3) for both sides, lots of errors "Sequence number did not match" will occur.
> So, I guess it may be caused by my modification to the at86rf230.c. But I cannot figure out the problem, could you help me? Thank you!
> Below shows my modification, the motivation is to try to get the mac sequence number, rssi and lqi of the received packet, and output the values into a debug file:
> 

Sorry, I have no idea. You modification could insert some additional
operation which makes something slower.

I change not my opinion, I think something will do a FORCE TX_ON
statechange, if this is going while RX_AACK_BUSY (don't know the exact
namens), then it will abort a sending of ack frame, the frame can be
received before.

You can remove the if-branch at [0] which will never make some force
state change, when we receive currently. To be clear, this patch:

diff --git a/drivers/net/ieee802154/at86rf230.c b/drivers/net/ieee802154/at86rf230.c
index 9f10da6..d4c6d72 100644
--- a/drivers/net/ieee802154/at86rf230.c
+++ b/drivers/net/ieee802154/at86rf230.c
@@ -447,8 +447,6 @@ at86rf230_async_state_assert(void *context)
                            ctx->to_state == STATE_TRX_OFF) {
                                u8 state = ctx->to_state;
 
-                               if (lp->tx_retry >= AT86RF2XX_MAX_TX_RETRIES)
-                                       state = STATE_FORCE_TRX_OFF;
                                lp->tx_retry++;
 
                                at86rf230_async_state_change(lp, ctx, state,

---

I currently have no idea what's going on your side and my only one guess
is that FORCE change will abort the transmitting of an ACK frame, but I
also didn't saw some behaviour like that on my side. I also doesn't have
a two RPi2 setup.

- Alex

[0] http://lxr.free-electrons.com/source/drivers/net/ieee802154/at86rf230.c#L450