Re: 6lowpan status

Re: 6lowpan status

[Alexander Aring]

Hi,

On Mon, Sep 01, 2014 at 01:36:14PM +0300, Jukka Rissanen wrote:
> Hi Alex,
> 
> I saw your RFC mail (9 Aug) about generic compression layer. Was there
> any functionality changes by this RFC or was it just refactoring the
> code?
> 

It's not just refactoring. The point is that there are many of RFC's
outside which declare a next header compession format (like UDP). UDP is
the only one next header compression which we currenlty support. If we
don't add a layer to add a new header compression format, all people
hack this into the iphc.c file. I don't know what you think about this
but I really don't like that.

This code adds a layer for this and supports all other known next header
compression formats in the way that we drop the packets afterwards
instead of deliver garabage to next layer (IPv6).

The layer avoids that everybody hacks static code into iphc file.

Please let me know if this is okay for you, then I will send a v2 with
changes.

> Anyway, looks ok to me although I just browsed quickly through the code,
> did not really review it :) Just wondering do we need more
> subdirectories under 6lowpan or could we just put everything into
> 6lowpan dir (just to keep it simple)? No big preference here thou.
> 

Don't know, but there is a unkown number of possible next header compression
formats outside, we should split it in a new subdirectorie. What I
actually mean, don't hack every next header compression format in a
single file.

For address compression/uncompression this is fine, there is no unknown
numbers of compression ways.

I know only IPHC and HC1. HC1 should be support for receiving side and
it's obsolete, but this is rfc 4944 so it's only related to 802.15.4.

Some or later we need to add support for this, but this should not used
by bluetooth 6LoWPAN.

> I saw the mail from Varka Bhadram (6 Aug) about RFC 6775 support. That
> looked interesting. Did you get the code from him (if yes, could you
> also send it to me)? Do you know why this guy have not send this stuff
> to upstream?
> 

The 802.15.4 mac layer has some neighbor discovery issues which are not
solved currently. It's about two kinds of mac addresses, for bluetooth
it should be fine. A small description about this problem is here [1].

We need some way to trigger 6LoWPAN/Layer 2 data across the IPv6 layer
and this isn't easy. Everything what we do on runtime decisions makes
the IPv6 stack slower and we should avoid that.



I currently working on a big rework of the 802.15.4 layer and will try
to bring this mainline at end of this month. Our mac layer is currenlty
in an not useable state. :(

I don't have time to make some effort after IPv6, currently I work on
the rework and will solve the two kind mac addresses issues with the
neighbor discovery cache, then we are in some state which we don't have
some broken support of functionality.


Maybe simple talk with Varka about what he did there, because I don't
really know what he exactly did there. The code what I saw isn't
mainline able. [0] What I know that RFC 6775 and the actually
implementation need to handle the two kinds of mac addresses (which is
broken currenlty) and need support of context based address compression.

I don't have time to add support for this. I need to rescue the 802.15.4
layer. Sorry! Maybe after that... or maybe somebody will do this work.
It sounds for me crazy to support RFC6775 which has dependencies on
other functionality which we doesn't support right now. For bluetooth
the mac layer works great, we only needs support for context based
address compression then.


I CC the new mailinglist for IEEE 802.15.4/IEEE 802.15.4 6LoWPAN
"linux-wpan" and bluetooth. We should not talk about this under the
hood.

- Alex

[0] https://www.mail-archive.com/linux-zigbee-devel%40lists.sourceforge.net/msg02018.html
[1] http://www.spinics.net/lists/netdev/msg291767.html

Re: 6lowpan status

[Jukka Rissanen]

Hi Alex,

On ma, 2014-09-01 at 13:38 +0200, Alexander Aring wrote:
> Hi,
> 
> On Mon, Sep 01, 2014 at 01:36:14PM +0300, Jukka Rissanen wrote:
> > Hi Alex,
> > 
> > I saw your RFC mail (9 Aug) about generic compression layer. Was there
> > any functionality changes by this RFC or was it just refactoring the
> > code?
> > 
> 
> It's not just refactoring. The point is that there are many of RFC's
> outside which declare a next header compession format (like UDP). UDP is
> the only one next header compression which we currenlty support. If we
> don't add a layer to add a new header compression format, all people
> hack this into the iphc.c file. I don't know what you think about this
> but I really don't like that.
> 
> This code adds a layer for this and supports all other known next header
> compression formats in the way that we drop the packets afterwards
> instead of deliver garabage to next layer (IPv6).
> 
> The layer avoids that everybody hacks static code into iphc file.

Ok, sounds like a plan.

> 
> Please let me know if this is okay for you, then I will send a v2 with
> changes.

Sure, just send new version.

> 
> > Anyway, looks ok to me although I just browsed quickly through the code,
> > did not really review it :) Just wondering do we need more
> > subdirectories under 6lowpan or could we just put everything into
> > 6lowpan dir (just to keep it simple)? No big preference here thou.
> > 
> 
> Don't know, but there is a unkown number of possible next header compression
> formats outside, we should split it in a new subdirectorie. What I
> actually mean, don't hack every next header compression format in a
> single file.

I was not really thinking that all the stuff would be in single file. I
was just pondering is there really a need for sub-directory, after all
the number of files in net/6lowpan directory is currently very small so
one option could be to put the nhc files directly into 6lowpan directory
(and perhaps adding them nhc_ prefix). Anyway, this is minor thing and I
am fine with having sub-directories.

> 
> For address compression/uncompression this is fine, there is no unknown
> numbers of compression ways.
> 
> I know only IPHC and HC1. HC1 should be support for receiving side and
> it's obsolete, but this is rfc 4944 so it's only related to 802.15.4.
> 
> Some or later we need to add support for this, but this should not used
> by bluetooth 6LoWPAN.
> 
> > I saw the mail from Varka Bhadram (6 Aug) about RFC 6775 support. That
> > looked interesting. Did you get the code from him (if yes, could you
> > also send it to me)? Do you know why this guy have not send this stuff
> > to upstream?
> > 
> 
> The 802.15.4 mac layer has some neighbor discovery issues which are not
> solved currently. It's about two kinds of mac addresses, for bluetooth
> it should be fine. A small description about this problem is here [1].
> 
> We need some way to trigger 6LoWPAN/Layer 2 data across the IPv6 layer
> and this isn't easy. Everything what we do on runtime decisions makes
> the IPv6 stack slower and we should avoid that.

Yep, agreed.

> 
> 
> 
> I currently working on a big rework of the 802.15.4 layer and will try
> to bring this mainline at end of this month. Our mac layer is currenlty
> in an not useable state. :(
> 
> I don't have time to make some effort after IPv6, currently I work on
> the rework and will solve the two kind mac addresses issues with the
> neighbor discovery cache, then we are in some state which we don't have
> some broken support of functionality.
> 
> 
> Maybe simple talk with Varka about what he did there, because I don't
> really know what he exactly did there. The code what I saw isn't
> mainline able. [0] What I know that RFC 6775 and the actually
> implementation need to handle the two kinds of mac addresses (which is
> broken currenlty) and need support of context based address compression.
> 
> I don't have time to add support for this. I need to rescue the 802.15.4
> layer. Sorry! Maybe after that... or maybe somebody will do this work.
> It sounds for me crazy to support RFC6775 which has dependencies on
> other functionality which we doesn't support right now. For bluetooth
> the mac layer works great, we only needs support for context based
> address compression then.

Ok. Currently I am also busy with some other stuff but I try to find
some time to investigate the RFC6775 more. Cannot really promise any
dates etc.

> 
> 
> I CC the new mailinglist for IEEE 802.15.4/IEEE 802.15.4 6LoWPAN
> "linux-wpan" and bluetooth. We should not talk about this under the
> hood.
> 
> - Alex
> 
> [0] https://www.mail-archive.com/linux-zigbee-devel%40lists.sourceforge.net/msg02018.html
> [1] http://www.spinics.net/lists/netdev/msg291767.html


Cheers,
Jukka

Re: 6lowpan status

[Alexander Aring]

Hi Jukka,

On Tue, Sep 02, 2014 at 09:08:10AM +0300, Jukka Rissanen wrote:
> Hi Alex,
> 
> On ma, 2014-09-01 at 13:38 +0200, Alexander Aring wrote:
> > Hi,
> > 
> > On Mon, Sep 01, 2014 at 01:36:14PM +0300, Jukka Rissanen wrote:
> > > Hi Alex,
> > > 
> > > I saw your RFC mail (9 Aug) about generic compression layer. Was there
> > > any functionality changes by this RFC or was it just refactoring the
> > > code?
> > > 
> > 
> > It's not just refactoring. The point is that there are many of RFC's
> > outside which declare a next header compession format (like UDP). UDP is
> > the only one next header compression which we currenlty support. If we
> > don't add a layer to add a new header compression format, all people
> > hack this into the iphc.c file. I don't know what you think about this
> > but I really don't like that.
> > 
> > This code adds a layer for this and supports all other known next header
> > compression formats in the way that we drop the packets afterwards
> > instead of deliver garabage to next layer (IPv6).
> > 
> > The layer avoids that everybody hacks static code into iphc file.
> 
> Ok, sounds like a plan.
> 
> > 
> > Please let me know if this is okay for you, then I will send a v2 with
> > changes.
> 
> Sure, just send new version.
> 

ok.

> > 
> > > Anyway, looks ok to me although I just browsed quickly through the code,
> > > did not really review it :) Just wondering do we need more
> > > subdirectories under 6lowpan or could we just put everything into
> > > 6lowpan dir (just to keep it simple)? No big preference here thou.
> > > 
> > 
> > Don't know, but there is a unkown number of possible next header compression
> > formats outside, we should split it in a new subdirectorie. What I
> > actually mean, don't hack every next header compression format in a
> > single file.
> 
> I was not really thinking that all the stuff would be in single file. I
> was just pondering is there really a need for sub-directory, after all
> the number of files in net/6lowpan directory is currently very small so
> one option could be to put the nhc files directly into 6lowpan directory
> (and perhaps adding them nhc_ prefix). Anyway, this is minor thing and I
> am fine with having sub-directories.
> 

ok. I that was the behaviour in one of my first series. I will change it
according to that. I agree with you here.

> > 
> > For address compression/uncompression this is fine, there is no unknown
> > numbers of compression ways.
> > 
> > I know only IPHC and HC1. HC1 should be support for receiving side and
> > it's obsolete, but this is rfc 4944 so it's only related to 802.15.4.
> > 
> > Some or later we need to add support for this, but this should not used
> > by bluetooth 6LoWPAN.
> > 
> > > I saw the mail from Varka Bhadram (6 Aug) about RFC 6775 support. That
> > > looked interesting. Did you get the code from him (if yes, could you
> > > also send it to me)? Do you know why this guy have not send this stuff
> > > to upstream?
> > > 
> > 
> > The 802.15.4 mac layer has some neighbor discovery issues which are not
> > solved currently. It's about two kinds of mac addresses, for bluetooth
> > it should be fine. A small description about this problem is here [1].
> > 
> > We need some way to trigger 6LoWPAN/Layer 2 data across the IPv6 layer
> > and this isn't easy. Everything what we do on runtime decisions makes
> > the IPv6 stack slower and we should avoid that.
> 
> Yep, agreed.
> 

That's the big challenge to make an acceptable mainline solution. :-)

At the end it should be acked by IPv6 community, they should scream if a
solution makes runtime decisions which slower down the stack too much.

> > 
> > 
> > 
> > I currently working on a big rework of the 802.15.4 layer and will try
> > to bring this mainline at end of this month. Our mac layer is currenlty
> > in an not useable state. :(
> > 
> > I don't have time to make some effort after IPv6, currently I work on
> > the rework and will solve the two kind mac addresses issues with the
> > neighbor discovery cache, then we are in some state which we don't have
> > some broken support of functionality.
> > 
> > 
> > Maybe simple talk with Varka about what he did there, because I don't
> > really know what he exactly did there. The code what I saw isn't
> > mainline able. [0] What I know that RFC 6775 and the actually
> > implementation need to handle the two kinds of mac addresses (which is
> > broken currenlty) and need support of context based address compression.
> > 
> > I don't have time to add support for this. I need to rescue the 802.15.4
> > layer. Sorry! Maybe after that... or maybe somebody will do this work.
> > It sounds for me crazy to support RFC6775 which has dependencies on
> > other functionality which we doesn't support right now. For bluetooth
> > the mac layer works great, we only needs support for context based
> > address compression then.
> 
> Ok. Currently I am also busy with some other stuff but I try to find
> some time to investigate the RFC6775 more. Cannot really promise any
> dates etc.
> 

Ok, null problem.

- Alex

Re: 6lowpan status

[Alexander Aring]

> > > 
> > > The 802.15.4 mac layer has some neighbor discovery issues which are not
> > > solved currently. It's about two kinds of mac addresses, for bluetooth
> > > it should be fine. A small description about this problem is here [1].
> > > 
> > > We need some way to trigger 6LoWPAN/Layer 2 data across the IPv6 layer
> > > and this isn't easy. Everything what we do on runtime decisions makes
> > > the IPv6 stack slower and we should avoid that.
> > 
> > Yep, agreed.
> > 
> 
> That's the big challenge to make an acceptable mainline solution. :-)
> 
> At the end it should be acked by IPv6 community, they should scream if a
> solution makes runtime decisions which slower down the stack too much.
> 

btw. I need L2 data in the neighbor discovery cache to handle the "two
kinds mac addresses". I will try to find a way after the rework of the mac
layer. This was discussed at [0].

- Alex

[0] http://marc.info/?l=linux-netdev&m=140724093516806&w=2

6lowpan raw socket problems

[Simon Vincent]

I am having problems with raw sockets on a 6lowpan 802.15.4 interface. 
The code works correctly on an ethernet ipv6 interface but not on a 
6lowpan interface. In the 6lowpan interface I receive packets but I find 
the src and dest addresses are incorrect.

I open the socket like this:
socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6);

Add options for IPV6_RECVPKTINFO, IPV6_CHECKSUM, IPV6_RECVHOPLIMIT.

I then receive a packet using recvmsg and attempt to get the source 
address. The source address is rubbish. Also trying to get the 
destination address from IPV6_PKTINFO using CMSG_FIRSTHDR results in 
rubbish.

Can anyone explain why? Is there something different I have to do when 
using 802.15.4 and 6lowpan?

Regards
Simon

Re: 6lowpan raw socket problems

[Alexander Aring]

Hi Simon,

On Tue, Sep 02, 2014 at 03:21:56PM +0100, Simon Vincent wrote:
> I am having problems with raw sockets on a 6lowpan 802.15.4 interface. The
> code works correctly on an ethernet ipv6 interface but not on a 6lowpan
> interface. In the 6lowpan interface I receive packets but I find the src and
> dest addresses are incorrect.
> 
> I open the socket like this:
> socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6);
> 
> Add options for IPV6_RECVPKTINFO, IPV6_CHECKSUM, IPV6_RECVHOPLIMIT.
> 
> I then receive a packet using recvmsg and attempt to get the source address.
> The source address is rubbish. Also trying to get the destination address
> from IPV6_PKTINFO using CMSG_FIRSTHDR results in rubbish.
> 
> Can anyone explain why? Is there something different I have to do when using
> 802.15.4 and 6lowpan?
> 

Can you please send a test programm for this as example? Then I/We will
look at why this happen.

I think this would be the best to find why you have this issue. Doesn't
need to be the full program only a small example.


What I can say actually is:
To send ICMPv6 packets from userspace you need RAW-Sockets. This is how
ping6 do it and ping6 works on my side. Maybe there is a issue because
some of your sockopts.

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

The program can be found at this github. branch all_fixes.
https://github.com/smee204/unstrung/tree/all_fixes

The socket is created in iface.cpp network_interface::setup()
The receive problem is in iface.cpp network_interface::receive(const 
time_t now)

Thanks

Simon

On 02/09/14 15:43, Alexander Aring wrote:
> Hi Simon,
>
> On Tue, Sep 02, 2014 at 03:21:56PM +0100, Simon Vincent wrote:
>> I am having problems with raw sockets on a 6lowpan 802.15.4 interface. The
>> code works correctly on an ethernet ipv6 interface but not on a 6lowpan
>> interface. In the 6lowpan interface I receive packets but I find the src and
>> dest addresses are incorrect.
>>
>> I open the socket like this:
>> socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6);
>>
>> Add options for IPV6_RECVPKTINFO, IPV6_CHECKSUM, IPV6_RECVHOPLIMIT.
>>
>> I then receive a packet using recvmsg and attempt to get the source address.
>> The source address is rubbish. Also trying to get the destination address
>> from IPV6_PKTINFO using CMSG_FIRSTHDR results in rubbish.
>>
>> Can anyone explain why? Is there something different I have to do when using
>> 802.15.4 and 6lowpan?
>>
> Can you please send a test programm for this as example? Then I/We will
> look at why this happen.
>
> I think this would be the best to find why you have this issue. Doesn't
> need to be the full program only a small example.
>
>
> What I can say actually is:
> To send ICMPv6 packets from userspace you need RAW-Sockets. This is how
> ping6 do it and ping6 works on my side. Maybe there is a issue because
> some of your sockopts.
>
> - Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

Hi,

On Tue, Sep 02, 2014 at 03:51:13PM +0100, Simon Vincent wrote:
> The program can be found at this github. branch all_fixes.
> https://github.com/smee204/unstrung/tree/all_fixes
> 
> The socket is created in iface.cpp network_interface::setup()
> The receive problem is in iface.cpp network_interface::receive(const time_t
> now)
> 
Thanks I will look at it, but I mean a small example not a complete RPL
framework. Nevertheless I think this is a good start to add a testing
framework into the wpan-tools. I will create a small test application
for raw sockets and different settings of sockopts.

6LoWPAN is only an adaptation layer we don't have any changes in the
IPv6 stack. Maybe you can also say something about which IPv6 address
type it was with the address compression according the address type?

For example link-local which is based on the mac address (SLAAC). I
already said we have issues with short addresses and 802.15.4 6LoWPAN. I
hope you don't want to try something with short addresses.

- Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

Hi Simon,

I saw this right now [0].

This is how an eui48 is converted to eui64. But we have already a
neui64. Are you sure this code check on this?

Normally it should check if the dev addr_len is 64 bit long, then it's a
eui64 address.

I don't see any test on this. That's why it works on a ethernet device
and not on a 802.15.4 lowpan device type.

- Alex

[0] https://github.com/smee204/unstrung/blob/all_fixes/lib/libndmgmt/iface.cpp#L91

Re: 6lowpan raw socket problems

[Simon Vincent]

This is not a problem. There is a fix elsewhere for that.

I am using local-link addresses.

I am seeing funny things like:
lowpan0: received packet from fe80::203:9a00:41:c864 -> 
7707:ffff:300:0:9a:300:7b3b:3a1a

Src address fe80::203:9a00:41:c864 does not exist on the network and 
destination address 7707:ffff:300:0:9a:300:7b3b:3a1a does not.

I will play around with different socket options and see if that makes a 
difference.

Simon

On 02/09/14 16:37, Alexander Aring wrote:
> Hi Simon,
>
> I saw this right now [0].
>
> This is how an eui48 is converted to eui64. But we have already a
> neui64. Are you sure this code check on this?
>
> Normally it should check if the dev addr_len is 64 bit long, then it's a
> eui64 address.
>
> I don't see any test on this. That's why it works on a ethernet device
> and not on a 802.15.4 lowpan device type.
>
> - Alex
>
> [0] https://github.com/smee204/unstrung/blob/all_fixes/lib/libndmgmt/iface.cpp#L91

Re: 6lowpan raw socket problems

[Alexander Aring]

On Tue, Sep 02, 2014 at 05:06:32PM +0100, Simon Vincent wrote:
> This is not a problem. There is a fix elsewhere for that.
> 
> I am using local-link addresses.
> 
> I am seeing funny things like:
> lowpan0: received packet from fe80::203:9a00:41:c864 ->
> 7707:ffff:300:0:9a:300:7b3b:3a1a
> 
> Src address fe80::203:9a00:41:c864 does not exist on the network and
> destination address 7707:ffff:300:0:9a:300:7b3b:3a1a does not.
> 

Where do you capture this? You know that wireshark on a lowpan interface
is broken?

- Alex

Re: 6lowpan raw socket problems

[Martin Townsend]

On 02/09/14 17:26, Alexander Aring wrote:
> On Tue, Sep 02, 2014 at 05:06:32PM +0100, Simon Vincent wrote:
>> This is not a problem. There is a fix elsewhere for that.
>>
>> I am using local-link addresses.
>>
>> I am seeing funny things like:
>> lowpan0: received packet from fe80::203:9a00:41:c864 ->
>> 7707:ffff:300:0:9a:300:7b3b:3a1a
>>
>> Src address fe80::203:9a00:41:c864 does not exist on the network and
>> destination address 7707:ffff:300:0:9a:300:7b3b:3a1a does not.
>>
> Where do you capture this? You know that wireshark on a lowpan interface
> is broken?
>
> - Alex
> --
> To unsubscribe from this list: send the line "unsubscribe linux-wpan" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
Hi Alex,

I'm working with Simon and these are from printf statements, we've 
stepped through the code and seen the same problem.

It would be nice to have a better testing environment, you mentioned 
that you have a virtual setup, does this allow for multiple nodes 
sending receiving packets like the contiki one? if so do you have 
instructions on how to setup?

Many Thanks,
Martin.

Re: 6lowpan raw socket problems

[Alexander Aring]

On Tue, Sep 02, 2014 at 07:53:50PM +0100, Martin Townsend wrote:
> On 02/09/14 17:26, Alexander Aring wrote:
> >On Tue, Sep 02, 2014 at 05:06:32PM +0100, Simon Vincent wrote:
> >>This is not a problem. There is a fix elsewhere for that.
> >>
> >>I am using local-link addresses.
> >>
> >>I am seeing funny things like:
> >>lowpan0: received packet from fe80::203:9a00:41:c864 ->
> >>7707:ffff:300:0:9a:300:7b3b:3a1a
> >>
> >>Src address fe80::203:9a00:41:c864 does not exist on the network and
> >>destination address 7707:ffff:300:0:9a:300:7b3b:3a1a does not.
> >>
> >Where do you capture this? You know that wireshark on a lowpan interface
> >is broken?
> >
> >- Alex
> >--
> >To unsubscribe from this list: send the line "unsubscribe linux-wpan" in
> >the body of a message to majordomo@vger.kernel.org
> >More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Hi Alex,
> 
> I'm working with Simon and these are from printf statements, we've stepped
> through the code and seen the same problem.
> 
> It would be nice to have a better testing environment, you mentioned that
> you have a virtual setup, does this allow for multiple nodes sending
> receiving packets like the contiki one? if so do you have instructions on
> how to setup?
> 

for testing you could use fakelb or tony's serial virtual interface [0].
I didn't test both of them, but we need some testing interface of
course.

The virtual serial interface require the serial driver.

Sorry for my late reply.

[0] https://github.com/tcheneau/virtual-ieee802154-serial

Re: 6lowpan raw socket problems

[Simon Vincent]

I have been looking into this issue again and it looks like RPL packets 
cause the problem. It looks a bit like the header is not decompressed 
correctly.

When I send a RPL packet I can see lowpan_header_compress is called 
which compresses the header. I can not find where the header is 
decompressed when the packet is received. By the time the packet gets to 
rawv6_recvmsg the src and dest addresses in the header are corrupt.

Is this expected?

Regards

Simon

On 08/09/14 11:43, Alexander Aring wrote:
> On Tue, Sep 02, 2014 at 07:53:50PM +0100, Martin Townsend wrote:
>> On 02/09/14 17:26, Alexander Aring wrote:
>>> On Tue, Sep 02, 2014 at 05:06:32PM +0100, Simon Vincent wrote:
>>>> This is not a problem. There is a fix elsewhere for that.
>>>>
>>>> I am using local-link addresses.
>>>>
>>>> I am seeing funny things like:
>>>> lowpan0: received packet from fe80::203:9a00:41:c864 ->
>>>> 7707:ffff:300:0:9a:300:7b3b:3a1a
>>>>
>>>> Src address fe80::203:9a00:41:c864 does not exist on the network and
>>>> destination address 7707:ffff:300:0:9a:300:7b3b:3a1a does not.
>>>>
>>> Where do you capture this? You know that wireshark on a lowpan interface
>>> is broken?
>>>
>>> - Alex
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-wpan" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> Hi Alex,
>>
>> I'm working with Simon and these are from printf statements, we've stepped
>> through the code and seen the same problem.
>>
>> It would be nice to have a better testing environment, you mentioned that
>> you have a virtual setup, does this allow for multiple nodes sending
>> receiving packets like the contiki one? if so do you have instructions on
>> how to setup?
>>
> for testing you could use fakelb or tony's serial virtual interface [0].
> I didn't test both of them, but we need some testing interface of
> course.
>
> The virtual serial interface require the serial driver.
>
> Sorry for my late reply.
>
> [0] https://github.com/tcheneau/virtual-ieee802154-serial

Re: 6lowpan raw socket problems

[Alexander Aring]

Hi Simon,

On Wed, Sep 17, 2014 at 02:30:22PM +0100, Simon Vincent wrote:
> I have been looking into this issue again and it looks like RPL packets
> cause the problem. It looks a bit like the header is not decompressed
> correctly.
> 

You mean now IPv6 header here? I think yes, there should no other header
compression running for this case currently.

> When I send a RPL packet I can see lowpan_header_compress is called which
> compresses the header. I can not find where the header is decompressed when
> the packet is received. By the time the packet gets to rawv6_recvmsg the src
> and dest addresses in the header are corrupt.
> 

Okay, you say here issues with source and destination addresses. I have
no idea now what's going on your side.

I need more information which types of addresses you use. Which kind of
address compression methods, note we don't support stateful address
compression but you are welcome to send patches. :-)

My idea is now, enable some more debug information while running
decompress/compress methods.

At [0], you could add "#define DEBUG" then you see more information
about the compression/uncompression while receiving/transmit. Don't know
if you already know about that.

Very simple is also to check if DAC or SAC bit in iphc1 is set. If this
is set you used stateful compression, which is not supported (should
occur a warning).

I don't know right your setup if you use linux<->linux this should never
happen. Other question is also, do you use short addresses? Handling of
short addresses, except the broadcast address is currently also not
supported. :-(

I can also not say that there is more bugs than this, Werner Almesberger
detected some issue about ICMPv6 payload size. Some issue with ICMPv6 payload
which are lesser than 4 bytes was filtered by kernel. That was wrong for
RPL... there was some crazy issue [1]. What I here want to say is, that
probaly IPv6 implementation can't also deal with it, it's hard to debug
I know. Use a little bit ctags for stepping into the code and instrument
some printk foo, more or later you will find it. :)

Sorry but I can't currently look more into this issue. Maybe simple add
the "#define DEBUG" into [1], then write some mail which information
from dmesg.

> Is this expected?
> 

No, normally all things should run fine. Otherwise you detect a bug. :-)

And of course you are also welcome to send patches for it. I know there
was some guy/girl which said the traffic class compression is broken,
but this is currently hard understandable code, need some time to
decrypt it, maybe a full rework for traffic class compression/decompress
is needed. Maybe some cases for traffic class will break your address
compression/decompression, overwriting some 6LoWPAN inline data.

- Alex

[0] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/6lowpan/iphc.c?id=refs/tags/v3.17-rc5#n51
[1] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d1c53c8e870cdedb6fc9550f41c558bab45b5219

Re: 6lowpan raw socket problems

[Alexander Aring]

On Wed, Sep 17, 2014 at 04:33:49PM +0100, Simon Vincent wrote:
> Hi Alex,
> 
> It looks like the problem is internal to Linux. I am running on a single
> Linux box, outputting a RPL message which is put onto a 802154 fakelb
> driver. The RPL message is also picked up by the code in my application that
> is listening on a RAW socket. This is where the corrupt addresses are seen.
> 
> All addresses are local link. Short addressing is used for the broadcast
> address only. I am not using stateful compression.
> 
> Is the same buffer used for the 6lowpan and ipv6 or is a copy taken to do
> the ipv6 header compression? If so the message is output on the 802154 but
> first the ipv6 header is compressed. The message is also passed to my RAW
> ipv6 socket from the ipv6 layer using the same buffer which now contains the
> compressed header?
> 
> 
>                     |--->   recvmsg RAW skt
> sendmsg RAW skt --->|
>                     |--->   6lowpan/802.15.4
> 
> My debug can see the header getting compressed at the 6lowpan layer but

<--- compressed means tx working and no rx.

> there is no receive debug triggered so the RAW socket is getting the message
> from the ipv6 layer.
> 

mhhh, I have a theory. The linux networking stack is too clever. It see's
that these addresses are on the _same_ HOST and running it only one some
loopback interfaces. It's like ping6 ::1 on a lowpan interface, if you
want to do that, we never replace the IPv6 header with an 6LoWPAN header
because the current behaviour doesn't send it to the lower layers.
In this case "linux networking stack" see's "ahhh the same host -> doing
magic optimization". Then the IPv6 stack doesn't call the necessary 
callbacks to do 6LoWPAN magic.

Please instrument some driver layer of fakelb driver [1] then we can check
this theory, When you instrument some printk's at driver layer we can
see if your frames ever reached the lower layer. I am right not sure if
this could be true, I never tested so much the fakelb driver.

Also please instrument this function [0], it's necessary to run
decompression when receiving. If my theory is true, then it's a
complicated issue and don't know right now a easy workaround.

- Alex

[0] http://lxr.free-electrons.com/source/net/ieee802154/6lowpan_rtnl.c#L456
[1] http://lxr.free-electrons.com/source/drivers/net/ieee802154/fakelb.c#L81

Re: 6lowpan raw socket problems

[Alexander Aring]

Hi,

On Wed, Sep 17, 2014 at 06:03:53PM +0200, Alexander Aring wrote:
> On Wed, Sep 17, 2014 at 04:33:49PM +0100, Simon Vincent wrote:
> > Hi Alex,
> > 
> > It looks like the problem is internal to Linux. I am running on a single
> > Linux box, outputting a RPL message which is put onto a 802154 fakelb
> > driver. The RPL message is also picked up by the code in my application that
> > is listening on a RAW socket. This is where the corrupt addresses are seen.
> > 
> > All addresses are local link. Short addressing is used for the broadcast
> > address only. I am not using stateful compression.
> > 
> > Is the same buffer used for the 6lowpan and ipv6 or is a copy taken to do
> > the ipv6 header compression? If so the message is output on the 802154 but
> > first the ipv6 header is compressed. The message is also passed to my RAW
> > ipv6 socket from the ipv6 layer using the same buffer which now contains the
> > compressed header?
> > 
> > 
> >                     |--->   recvmsg RAW skt
> > sendmsg RAW skt --->|
> >                     |--->   6lowpan/802.15.4
> > 
> > My debug can see the header getting compressed at the 6lowpan layer but
> 
> <--- compressed means tx working and no rx.
> 
> > there is no receive debug triggered so the RAW socket is getting the message
> > from the ipv6 layer.
> > 
> 
> mhhh, I have a theory. The linux networking stack is too clever. It see's
> that these addresses are on the _same_ HOST and running it only one some
> loopback interfaces. It's like ping6 ::1 on a lowpan interface, if you
> want to do that, we never replace the IPv6 header with an 6LoWPAN header
> because the current behaviour doesn't send it to the lower layers.
> In this case "linux networking stack" see's "ahhh the same host -> doing
> magic optimization". Then the IPv6 stack doesn't call the necessary 
> callbacks to do 6LoWPAN magic.
> 
> Please instrument some driver layer of fakelb driver [1] then we can check
> this theory, When you instrument some printk's at driver layer we can
> see if your frames ever reached the lower layer. I am right not sure if
> this could be true, I never tested so much the fakelb driver.
> 
> Also please instrument this function [0], it's necessary to run
> decompression when receiving. If my theory is true, then it's a
> complicated issue and don't know right now a easy workaround.
> 

I also remembered an other guy/girl which had some issues with 6lowpan
and fakelb driver, but I did a quick test and it worked for me.

Which kernel version do you using, net-next kernel?

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

We are using kernel 3.16 from davem net-next.

On 17/09/14 17:17, Alexander Aring wrote:
> Hi,
>
> On Wed, Sep 17, 2014 at 06:03:53PM +0200, Alexander Aring wrote:
>> On Wed, Sep 17, 2014 at 04:33:49PM +0100, Simon Vincent wrote:
>>> Hi Alex,
>>>
>>> It looks like the problem is internal to Linux. I am running on a single
>>> Linux box, outputting a RPL message which is put onto a 802154 fakelb
>>> driver. The RPL message is also picked up by the code in my application that
>>> is listening on a RAW socket. This is where the corrupt addresses are seen.
>>>
>>> All addresses are local link. Short addressing is used for the broadcast
>>> address only. I am not using stateful compression.
>>>
>>> Is the same buffer used for the 6lowpan and ipv6 or is a copy taken to do
>>> the ipv6 header compression? If so the message is output on the 802154 but
>>> first the ipv6 header is compressed. The message is also passed to my RAW
>>> ipv6 socket from the ipv6 layer using the same buffer which now contains the
>>> compressed header?
>>>
>>>
>>>                      |--->   recvmsg RAW skt
>>> sendmsg RAW skt --->|
>>>                      |--->   6lowpan/802.15.4
>>>
>>> My debug can see the header getting compressed at the 6lowpan layer but
>> <--- compressed means tx working and no rx.
>>
>>> there is no receive debug triggered so the RAW socket is getting the message
>>> from the ipv6 layer.
>>>
>> mhhh, I have a theory. The linux networking stack is too clever. It see's
>> that these addresses are on the _same_ HOST and running it only one some
>> loopback interfaces. It's like ping6 ::1 on a lowpan interface, if you
>> want to do that, we never replace the IPv6 header with an 6LoWPAN header
>> because the current behaviour doesn't send it to the lower layers.
>> In this case "linux networking stack" see's "ahhh the same host -> doing
>> magic optimization". Then the IPv6 stack doesn't call the necessary
>> callbacks to do 6LoWPAN magic.
>>
>> Please instrument some driver layer of fakelb driver [1] then we can check
>> this theory, When you instrument some printk's at driver layer we can
>> see if your frames ever reached the lower layer. I am right not sure if
>> this could be true, I never tested so much the fakelb driver.
>>
>> Also please instrument this function [0], it's necessary to run
>> decompression when receiving. If my theory is true, then it's a
>> complicated issue and don't know right now a easy workaround.
>>
> I also remembered an other guy/girl which had some issues with 6lowpan
> and fakelb driver, but I did a quick test and it worked for me.
>
> Which kernel version do you using, net-next kernel?
>
> - Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

Hi Alex,

It looks like the problem is internal to Linux. I am running on a single 
Linux box, outputting a RPL message which is put onto a 802154 fakelb 
driver. The RPL message is also picked up by the code in my application 
that is listening on a RAW socket. This is where the corrupt addresses 
are seen.

All addresses are local link. Short addressing is used for the broadcast 
address only. I am not using stateful compression.

Is the same buffer used for the 6lowpan and ipv6 or is a copy taken to 
do the ipv6 header compression? If so the message is output on the 
802154 but first the ipv6 header is compressed. The message is also 
passed to my RAW ipv6 socket from the ipv6 layer using the same buffer 
which now contains the compressed header?


                                     |--->   recvmsg RAW skt
sendmsg RAW skt --->|
                                     |--->   6lowpan/802.15.4

My debug can see the header getting compressed at the 6lowpan layer but 
there is no receive debug triggered so the RAW socket is getting the 
message from the ipv6 layer.

Thanks

Simon

On 17/09/14 14:57, Alexander Aring wrote:
> Hi Simon,
>
> On Wed, Sep 17, 2014 at 02:30:22PM +0100, Simon Vincent wrote:
>> I have been looking into this issue again and it looks like RPL packets
>> cause the problem. It looks a bit like the header is not decompressed
>> correctly.
>>
> You mean now IPv6 header here? I think yes, there should no other header
> compression running for this case currently.
>> When I send a RPL packet I can see lowpan_header_compress is called which
>> compresses the header. I can not find where the header is decompressed when
>> the packet is received. By the time the packet gets to rawv6_recvmsg the src
>> and dest addresses in the header are corrupt.
>>
> Okay, you say here issues with source and destination addresses. I have
> no idea now what's going on your side.
>
> I need more information which types of addresses you use. Which kind of
> address compression methods, note we don't support stateful address
> compression but you are welcome to send patches. :-)
>
> My idea is now, enable some more debug information while running
> decompress/compress methods.
>
> At [0], you could add "#define DEBUG" then you see more information
> about the compression/uncompression while receiving/transmit. Don't know
> if you already know about that.
>
> Very simple is also to check if DAC or SAC bit in iphc1 is set. If this
> is set you used stateful compression, which is not supported (should
> occur a warning).
>
> I don't know right your setup if you use linux<->linux this should never
> happen. Other question is also, do you use short addresses? Handling of
> short addresses, except the broadcast address is currently also not
> supported. :-(
>
> I can also not say that there is more bugs than this, Werner Almesberger
> detected some issue about ICMPv6 payload size. Some issue with ICMPv6 payload
> which are lesser than 4 bytes was filtered by kernel. That was wrong for
> RPL... there was some crazy issue [1]. What I here want to say is, that
> probaly IPv6 implementation can't also deal with it, it's hard to debug
> I know. Use a little bit ctags for stepping into the code and instrument
> some printk foo, more or later you will find it. :)
>
> Sorry but I can't currently look more into this issue. Maybe simple add
> the "#define DEBUG" into [1], then write some mail which information
> from dmesg.
>
>> Is this expected?
>>
> No, normally all things should run fine. Otherwise you detect a bug. :-)
>
> And of course you are also welcome to send patches for it. I know there
> was some guy/girl which said the traffic class compression is broken,
> but this is currently hard understandable code, need some time to
> decrypt it, maybe a full rework for traffic class compression/decompress
> is needed. Maybe some cases for traffic class will break your address
> compression/decompression, overwriting some 6LoWPAN inline data.
>
> - Alex
>
> [0]https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/6lowpan/iphc.c?id=refs/tags/v3.17-rc5#n51
> [1]https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d1c53c8e870cdedb6fc9550f41c558bab45b5219

Re: 6lowpan raw socket problems

[Simon Vincent]

I think what is happening is similar to your theory. The packets are getting sent to the 802.15.4 driver and the compress header function but as we have the raw socket recvmsg the packet is also sent internally to this socket. 
This is because a raw socket receives all messages that are sent and received. 
It does appear to be a complex problem. I just need to prove that this is what is happening by inserting more debug.

Simon

On 17 Sep 2014 17:17, Alexander Aring <alex.aring@gmail.com> wrote:
>
> Hi, 
>
> On Wed, Sep 17, 2014 at 06:03:53PM +0200, Alexander Aring wrote: 
> > On Wed, Sep 17, 2014 at 04:33:49PM +0100, Simon Vincent wrote: 
> > > Hi Alex, 
> > > 
> > > It looks like the problem is internal to Linux. I am running on a single 
> > > Linux box, outputting a RPL message which is put onto a 802154 fakelb 
> > > driver. The RPL message is also picked up by the code in my application that 
> > > is listening on a RAW socket. This is where the corrupt addresses are seen. 
> > > 
> > > All addresses are local link. Short addressing is used for the broadcast 
> > > address only. I am not using stateful compression. 
> > > 
> > > Is the same buffer used for the 6lowpan and ipv6 or is a copy taken to do 
> > > the ipv6 header compression? If so the message is output on the 802154 but 
> > > first the ipv6 header is compressed. The message is also passed to my RAW 
> > > ipv6 socket from the ipv6 layer using the same buffer which now contains the 
> > > compressed header? 
> > > 
> > > 
> > >                     |--->   recvmsg RAW skt 
> > > sendmsg RAW skt --->| 
> > >                     |--->   6lowpan/802.15.4 
> > > 
> > > My debug can see the header getting compressed at the 6lowpan layer but 
> > 
> > <--- compressed means tx working and no rx. 
> > 
> > > there is no receive debug triggered so the RAW socket is getting the message 
> > > from the ipv6 layer. 
> > > 
> > 
> > mhhh, I have a theory. The linux networking stack is too clever. It see's 
> > that these addresses are on the _same_ HOST and running it only one some 
> > loopback interfaces. It's like ping6 ::1 on a lowpan interface, if you 
> > want to do that, we never replace the IPv6 header with an 6LoWPAN header 
> > because the current behaviour doesn't send it to the lower layers. 
> > In this case "linux networking stack" see's "ahhh the same host -> doing 
> > magic optimization". Then the IPv6 stack doesn't call the necessary 
> > callbacks to do 6LoWPAN magic. 
> > 
> > Please instrument some driver layer of fakelb driver [1] then we can check 
> > this theory, When you instrument some printk's at driver layer we can 
> > see if your frames ever reached the lower layer. I am right not sure if 
> > this could be true, I never tested so much the fakelb driver. 
> > 
> > Also please instrument this function [0], it's necessary to run 
> > decompression when receiving. If my theory is true, then it's a 
> > complicated issue and don't know right now a easy workaround. 
> > 
>
> I also remembered an other guy/girl which had some issues with 6lowpan 
> and fakelb driver, but I did a quick test and it worked for me. 
>
> Which kernel version do you using, net-next kernel? 
>
> - Alex 

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 08:46:55AM +0100, Simon Vincent wrote:
> I think what is happening is similar to your theory. The packets are getting sent to the 802.15.4 driver and the compress header function but as we have the raw socket recvmsg the packet is also sent internally to this socket. 
> This is because a raw socket receives all messages that are sent and received. 
> It does appear to be a complex problem. I just need to prove that this is what is happening by inserting more debug.
> 
mh, ok. Good to know that there is maybe such issue with the fakelb
driver and 6LoWPAN. Maybe there exists some netdev flag setting or
something else to avoid this.

You could describe this problem at netdev@vger.kernel.org and maybe the
guys knows a solution.

At the moment we use's some callbacks for replacing IPv6 with 6LoWPAN
header which was not made to make this there. This seems to be a global
architecture problem of the 6LoWPAN stack.

Or we need a another fakelb driver solution, I remember the tun/tap
interfaces which allow to send/recv packets from/to userspace. They need
to have similar issues when they have two interfaces and link-local
addresses. I don't know how they deal with that. This idea is just a shoot
into the dark. ;-)

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

Sorry if I confused you, It is not a fakelb problem I get the same 
problem if I run using real 802.15.4 hardware (mrf24j40). The packet is 
put out by the driver but also sent to the RAW socket.

Simon

On 18/09/14 09:33, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 08:46:55AM +0100, Simon Vincent wrote:
>> I think what is happening is similar to your theory. The packets are getting sent to the 802.15.4 driver and the compress header function but as we have the raw socket recvmsg the packet is also sent internally to this socket.
>> This is because a raw socket receives all messages that are sent and received.
>> It does appear to be a complex problem. I just need to prove that this is what is happening by inserting more debug.
>>
> mh, ok. Good to know that there is maybe such issue with the fakelb
> driver and 6LoWPAN. Maybe there exists some netdev flag setting or
> something else to avoid this.
>
> You could describe this problem at netdev@vger.kernel.org and maybe the
> guys knows a solution.
>
> At the moment we use's some callbacks for replacing IPv6 with 6LoWPAN
> header which was not made to make this there. This seems to be a global
> architecture problem of the 6LoWPAN stack.
>
> Or we need a another fakelb driver solution, I remember the tun/tap
> interfaces which allow to send/recv packets from/to userspace. They need
> to have similar issues when they have two interfaces and link-local
> addresses. I don't know how they deal with that. This idea is just a shoot
> into the dark. ;-)
>
> - Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 09:37:40AM +0100, Simon Vincent wrote:
> Sorry if I confused you, It is not a fakelb problem I get the same problem
> if I run using real 802.15.4 hardware (mrf24j40). The packet is put out by
> the driver but also sent to the RAW socket.
> 
Two mrf24j40 on the same host?

- Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 10:45:15AM +0200, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 09:37:40AM +0100, Simon Vincent wrote:
> > Sorry if I confused you, It is not a fakelb problem I get the same problem
> > if I run using real 802.15.4 hardware (mrf24j40). The packet is put out by
> > the driver but also sent to the RAW socket.
> > 
> Two mrf24j40 on the same host?

I mean, my theory is that there are two IPv6 _interfaces_ are on the
same host. Then Linux do some optimization and necessary callbacks for
replacing 6LoWPAN header header will never called. Linux do this
optimization because Linux see it's local.

Then doesn't matter if two mrf24j40, fakelb, etc... or some combinations
of them. :-)

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

There is just a single mrf24j40 on the host.
Some of the messages are sent to multicast addresses so this could be 
causing it!
In particular the multicast address is ff02::1a. (all RPL nodes) I guess 
it is likely this has not been implemented in the Linux kernel...

Simon


On 18/09/14 09:54, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 10:45:15AM +0200, Alexander Aring wrote:
>> On Thu, Sep 18, 2014 at 09:37:40AM +0100, Simon Vincent wrote:
>>> Sorry if I confused you, It is not a fakelb problem I get the same problem
>>> if I run using real 802.15.4 hardware (mrf24j40). The packet is put out by
>>> the driver but also sent to the RAW socket.
>>>
>> Two mrf24j40 on the same host?
> I mean, my theory is that there are two IPv6 _interfaces_ are on the
> same host. Then Linux do some optimization and necessary callbacks for
> replacing 6LoWPAN header header will never called. Linux do this
> optimization because Linux see it's local.
>
> Then doesn't matter if two mrf24j40, fakelb, etc... or some combinations
> of them. :-)
>
> - Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 10:03:15AM +0100, Simon Vincent wrote:
> There is just a single mrf24j40 on the host.
> Some of the messages are sent to multicast addresses so this could be
> causing it!
> In particular the multicast address is ff02::1a. (all RPL nodes) I guess it
> is likely this has not been implemented in the Linux kernel...
> 

and we comming back to some other issue which Werner Almesberger
detected. [0] :-)

There are also some IPv6 implementation issues why RPL sometimes not
really working.

I cc Werner here, hope that's okay.

- Alex

[0] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d1c53c8e870cdedb6fc9550f41c558bab45b5219

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 11:44:01AM +0200, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 10:03:15AM +0100, Simon Vincent wrote:
> > There is just a single mrf24j40 on the host.
> > Some of the messages are sent to multicast addresses so this could be
> > causing it!
> > In particular the multicast address is ff02::1a. (all RPL nodes) I guess it
> > is likely this has not been implemented in the Linux kernel...
> > 
> 
> and we comming back to some other issue which Werner Almesberger
> detected. [0] :-)
> 
> There are also some IPv6 implementation issues why RPL sometimes not
> really working.
> 
> I cc Werner here, hope that's okay.
> 
> - Alex
> 
> [0] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d1c53c8e870cdedb6fc9550f41c558bab45b5219

and now really cc Werner, here. Sorry.

Re: 6lowpan raw socket problems

[Simon Vincent]

I have created a small test program that shows this problem. It looks 
like a race condition as sometimes the addresses are not corrupt.

It looks like if the RAW socket gets the packet before the packet hits 
the 6lowpan layer the addresses are fine. If the packet hits the 6lowpan 
layer before the RAW socket gets the packet then the addresses are corrupt.

The test program can be found here.
https://github.com/xsilon/sockdebug

I will continue debugging!

Simon

On 18/09/14 10:45, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 11:44:01AM +0200, Alexander Aring wrote:
>> On Thu, Sep 18, 2014 at 10:03:15AM +0100, Simon Vincent wrote:
>>> There is just a single mrf24j40 on the host.
>>> Some of the messages are sent to multicast addresses so this could be
>>> causing it!
>>> In particular the multicast address is ff02::1a. (all RPL nodes) I guess it
>>> is likely this has not been implemented in the Linux kernel...
>>>
>> and we comming back to some other issue which Werner Almesberger
>> detected. [0] :-)
>>
>> There are also some IPv6 implementation issues why RPL sometimes not
>> really working.
>>
>> I cc Werner here, hope that's okay.
>>
>> - Alex
>>
>> [0] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d1c53c8e870cdedb6fc9550f41c558bab45b5219
> and now really cc Werner, here. Sorry.

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
> I have created a small test program that shows this problem. It looks like a
> race condition as sometimes the addresses are not corrupt.
> 

Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
Don't know right now.

> It looks like if the RAW socket gets the packet before the packet hits the
> 6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
> before the RAW socket gets the packet then the addresses are corrupt.
> 
> The test program can be found here.
> https://github.com/xsilon/sockdebug
> 
> I will continue debugging!
> 
ok, good luck.

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

It looks like in 6lowpan_iphc.c lowpan_header_compress the original ipv6 
header is removed and the compressed header is attached.

These four lines are responsible.
     skb_pull(skb, sizeof(struct ipv6hdr));
     skb_reset_transport_header(skb);
     memcpy(skb_push(skb, hc06_ptr - head), head, hc06_ptr - head);
     skb_reset_network_header(skb);

I don't think we can do this as the skb is being used in other parts of 
the ip stack. Hence when the ipv6 header is read elsewhere the addresses 
become corrupt as they have been overwritten by the 6lowpan compressed 
header.

Any ideas on how to fix this?

Simon

On 18/09/14 15:19, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
>> I have created a small test program that shows this problem. It looks like a
>> race condition as sometimes the addresses are not corrupt.
>>
> Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
> Don't know right now.
>
>> It looks like if the RAW socket gets the packet before the packet hits the
>> 6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
>> before the RAW socket gets the packet then the addresses are corrupt.
>>
>> The test program can be found here.
>> https://github.com/xsilon/sockdebug
>>
>> I will continue debugging!
>>
> ok, good luck.
>
> - Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 04:54:53PM +0100, Simon Vincent wrote:
> It looks like in 6lowpan_iphc.c lowpan_header_compress the original ipv6
> header is removed and the compressed header is attached.
> 
> These four lines are responsible.
>     skb_pull(skb, sizeof(struct ipv6hdr));
>     skb_reset_transport_header(skb);
>     memcpy(skb_push(skb, hc06_ptr - head), head, hc06_ptr - head);
>     skb_reset_network_header(skb);
> 
> I don't think we can do this as the skb is being used in other parts of the
> ip stack. Hence when the ipv6 header is read elsewhere the addresses become
> corrupt as they have been overwritten by the 6lowpan compressed header.
> 
> Any ideas on how to fix this?
> 
yes, but I don't believe that this makes trouble. <--- or only makes
trouble by replacing data, see below.

It's called by a callback of header_ops [0].

This is for generating the mac header with address information from
neighbor discovery cache (mainly destination address) and source
addresse (mainly netdev->dev_addr).

Another example of this function is ethernet. [1]

On [1] you will se that the ethernet header will created there.

- Get data from skb for ethhdr (ethernet header)
  struct ethhdr *eth = (struct ethhdr *)skb_push(skb, ETH_HLEN);

- memcpy(eth->h_source, saddr, ETH_ALEN); <-- source address

- memcpy(eth->h_dest, daddr, ETH_ALEN); <-- destination address.


So they using the callback there to manipulate the skb here.

Another idea is that, maybe we can ADD data but not REPLACING existing
data with that. I don't know right now.


But I moved this handling out of the create callback of header_ops. This
also fix the issue by running wireshark&co on a lowpan interface.

What I did is only to save the address information in the reserved room
of skb in this callback. [2]

Then running replacing header in the xmit callback of lowpan device. It
looks very different now! I have also splitted the lowpan implementation
in three files "main.c, tx.c, rx.c". Please note that. This code is part
of the rework and I want to fix the wireshark&co issue there.

I don't have a solution for this right now which applies on current
mainline, sorry! But the xmit callback is 100% secure by replacing skb
header data.

If you like you can try to apply it on mainline. What I said is that
this also fix the IPv6 capturing on a lowpan interface. I want also say
to you that I added a comment there "TODO ask david or marc if this run
into trouble", because I am not sure if the reserved skb room can be
overwritten sometimes, then we stuck into the same issue. (Worked on my
side currently), but I also don't do much raw socket RPL messages. ;-)

What I know is that we can't use skb->cb here to save the information,
this would be overwritten by traffic control.

- Alex

[0] http://lxr.free-electrons.com/source/include/linux/netdevice.h#L255
[1] http://lxr.free-electrons.com/source/net/ethernet/eth.c#L78
[2] https://github.com/linux-wpan/linux-wpan-next/blob/7ccf5a24a19a72b857bbcbc76342f4838dd85d6b/net/ieee802154/6lowpan/tx.c#L59
[3] https://github.com/linux-wpan/linux-wpan-next/blob/7ccf5a24a19a72b857bbcbc76342f4838dd85d6b/net/ieee802154/6lowpan/tx.c#L261

Re: 6lowpan raw socket problems

[Alexander Aring]

I forget this,

On Thu, Sep 18, 2014 at 06:30:08PM +0200, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 04:54:53PM +0100, Simon Vincent wrote:
> > It looks like in 6lowpan_iphc.c lowpan_header_compress the original ipv6
> > header is removed and the compressed header is attached.
> > 
> > These four lines are responsible.
> >     skb_pull(skb, sizeof(struct ipv6hdr));
> >     skb_reset_transport_header(skb);
> >     memcpy(skb_push(skb, hc06_ptr - head), head, hc06_ptr - head);
> >     skb_reset_network_header(skb);
> > 
> > I don't think we can do this as the skb is being used in other parts of the
> > ip stack. Hence when the ipv6 header is read elsewhere the addresses become
> > corrupt as they have been overwritten by the 6lowpan compressed header.
> > 
> > Any ideas on how to fix this?
> > 
> yes, but I don't believe that this makes trouble. <--- or only makes
> trouble by replacing data, see below.
> 
> It's called by a callback of header_ops [0].
> 
> This is for generating the mac header with address information from
> neighbor discovery cache (mainly destination address) and source
> addresse (mainly netdev->dev_addr).
> 
> Another example of this function is ethernet. [1]
> 
> On [1] you will se that the ethernet header will created there.
> 
> - Get data from skb for ethhdr (ethernet header)
>   struct ethhdr *eth = (struct ethhdr *)skb_push(skb, ETH_HLEN);
> 
> - memcpy(eth->h_source, saddr, ETH_ALEN); <-- source address
> 
> - memcpy(eth->h_dest, daddr, ETH_ALEN); <-- destination address.
> 
> 
> So they using the callback there to manipulate the skb here.
> 
> Another idea is that, maybe we can ADD data but not REPLACING existing
> data with that. I don't know right now.
> 
> 
> But I moved this handling out of the create callback of header_ops. This
> also fix the issue by running wireshark&co on a lowpan interface.
> 
> What I did is only to save the address information in the reserved room
> of skb in this callback. [2]
> 
> Then running replacing header in the xmit callback of lowpan device. It
> looks very different now! I have also splitted the lowpan implementation
> in three files "main.c, tx.c, rx.c". Please note that. This code is part
> of the rework and I want to fix the wireshark&co issue there.
> 
> I don't have a solution for this right now which applies on current
> mainline, sorry! But the xmit callback is 100% secure by replacing skb
> header data.
> 
> If you like you can try to apply it on mainline. What I said is that
> this also fix the IPv6 capturing on a lowpan interface. I want also say
> to you that I added a comment there "TODO ask david or marc if this run
> into trouble", because I am not sure if the reserved skb room can be
> overwritten sometimes, then we stuck into the same issue. (Worked on my
> side currently), but I also don't do much raw socket RPL messages. ;-)
> 
> What I know is that we can't use skb->cb here to save the information,
> this would be overwritten by traffic control.
> 

See also:

https://github.com/linux-wpan/linux-wpan-next/commit/b86faef5523ddb451e8e4784ea1a49a852901289

but don't trust the diff, I think there was some copy&paste bug.

What I do there in a small sentence "Only save address information in
"lowpan_header_create", do IPv6 -> 6LoWPAN replacing in "lowpan_xmit".

Hope it's understandable what I did there and why, because wireshark
capture after lowpan_header_create but before lowpan_xmit. And then we
seeing the 6LoWPAN header on wireshark which is wrong on a lowpan interface.

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

On 18/09/14 17:30, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 04:54:53PM +0100, Simon Vincent wrote:
>> It looks like in 6lowpan_iphc.c lowpan_header_compress the original ipv6
>> header is removed and the compressed header is attached.
>>
>> These four lines are responsible.
>>      skb_pull(skb, sizeof(struct ipv6hdr));
>>      skb_reset_transport_header(skb);
>>      memcpy(skb_push(skb, hc06_ptr - head), head, hc06_ptr - head);
>>      skb_reset_network_header(skb);
>>
>> I don't think we can do this as the skb is being used in other parts of the
>> ip stack. Hence when the ipv6 header is read elsewhere the addresses become
>> corrupt as they have been overwritten by the 6lowpan compressed header.
>>
>> Any ideas on how to fix this?
>>
> yes, but I don't believe that this makes trouble. <--- or only makes
> trouble by replacing data, see below.
>
> It's called by a callback of header_ops [0].
>
> This is for generating the mac header with address information from
> neighbor discovery cache (mainly destination address) and source
> addresse (mainly netdev->dev_addr).
>
> Another example of this function is ethernet. [1]
>
> On [1] you will se that the ethernet header will created there.
>
> - Get data from skb for ethhdr (ethernet header)
>    struct ethhdr *eth = (struct ethhdr *)skb_push(skb, ETH_HLEN);
>
> - memcpy(eth->h_source, saddr, ETH_ALEN); <-- source address
>
> - memcpy(eth->h_dest, daddr, ETH_ALEN); <-- destination address.
>
>
> So they using the callback there to manipulate the skb here.
>
> Another idea is that, maybe we can ADD data but not REPLACING existing
> data with that. I don't know right now.
I think what is done in ethernet is fine as they are adding data. But in 
the lowpan_header_compress we are doing a skb_pull to remove the 
existing ipv6 header before adding on the new 6lowpan header, so we are 
replacing the existing data.
>
> But I moved this handling out of the create callback of header_ops. This
> also fix the issue by running wireshark&co on a lowpan interface.
>
> What I did is only to save the address information in the reserved room
> of skb in this callback. [2]
>
> Then running replacing header in the xmit callback of lowpan device. It
> looks very different now! I have also splitted the lowpan implementation
> in three files "main.c, tx.c, rx.c". Please note that. This code is part
> of the rework and I want to fix the wireshark&co issue there.
>
> I don't have a solution for this right now which applies on current
> mainline, sorry! But the xmit callback is 100% secure by replacing skb
> header data.
>
> If you like you can try to apply it on mainline. What I said is that
> this also fix the IPv6 capturing on a lowpan interface. I want also say
> to you that I added a comment there "TODO ask david or marc if this run
> into trouble", because I am not sure if the reserved skb room can be
> overwritten sometimes, then we stuck into the same issue. (Worked on my
> side currently), but I also don't do much raw socket RPL messages. ;-)
>
> What I know is that we can't use skb->cb here to save the information,
> this would be overwritten by traffic control.
>
> - Alex
>
> [0] http://lxr.free-electrons.com/source/include/linux/netdevice.h#L255
> [1] http://lxr.free-electrons.com/source/net/ethernet/eth.c#L78
> [2] https://github.com/linux-wpan/linux-wpan-next/blob/7ccf5a24a19a72b857bbcbc76342f4838dd85d6b/net/ieee802154/6lowpan/tx.c#L59
> [3] https://github.com/linux-wpan/linux-wpan-next/blob/7ccf5a24a19a72b857bbcbc76342f4838dd85d6b/net/ieee802154/6lowpan/tx.c#L261

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 06:03:36PM +0100, Simon Vincent wrote:
> 
> On 18/09/14 17:30, Alexander Aring wrote:
> >On Thu, Sep 18, 2014 at 04:54:53PM +0100, Simon Vincent wrote:
> >>It looks like in 6lowpan_iphc.c lowpan_header_compress the original ipv6
> >>header is removed and the compressed header is attached.
> >>
> >>These four lines are responsible.
> >>     skb_pull(skb, sizeof(struct ipv6hdr));
> >>     skb_reset_transport_header(skb);
> >>     memcpy(skb_push(skb, hc06_ptr - head), head, hc06_ptr - head);
> >>     skb_reset_network_header(skb);
> >>
> >>I don't think we can do this as the skb is being used in other parts of the
> >>ip stack. Hence when the ipv6 header is read elsewhere the addresses become
> >>corrupt as they have been overwritten by the 6lowpan compressed header.
> >>
> >>Any ideas on how to fix this?
> >>
> >yes, but I don't believe that this makes trouble. <--- or only makes
> >trouble by replacing data, see below.
> >
> >It's called by a callback of header_ops [0].
> >
> >This is for generating the mac header with address information from
> >neighbor discovery cache (mainly destination address) and source
> >addresse (mainly netdev->dev_addr).
> >
> >Another example of this function is ethernet. [1]
> >
> >On [1] you will se that the ethernet header will created there.
> >
> >- Get data from skb for ethhdr (ethernet header)
> >   struct ethhdr *eth = (struct ethhdr *)skb_push(skb, ETH_HLEN);
> >
> >- memcpy(eth->h_source, saddr, ETH_ALEN); <-- source address
> >
> >- memcpy(eth->h_dest, daddr, ETH_ALEN); <-- destination address.
> >
> >
> >So they using the callback there to manipulate the skb here.
> >
> >Another idea is that, maybe we can ADD data but not REPLACING existing
> >data with that. I don't know right now.
> I think what is done in ethernet is fine as they are adding data. But in the
> lowpan_header_compress we are doing a skb_pull to remove the existing ipv6
> header before adding on the new 6lowpan header, so we are replacing the
> existing data.

Yep, then you could try to test what I did there. Simple save the
address information in reserved skb data and then replacing data in
lowpan_xmit callback. This also solves the capture on lowpan interfaces.

lowpan_xmit should be save to replacing the header.

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

On 18/09/14 18:09, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 06:03:36PM +0100, Simon Vincent wrote:
>> On 18/09/14 17:30, Alexander Aring wrote:
>>> On Thu, Sep 18, 2014 at 04:54:53PM +0100, Simon Vincent wrote:
>>>> It looks like in 6lowpan_iphc.c lowpan_header_compress the original ipv6
>>>> header is removed and the compressed header is attached.
>>>>
>>>> These four lines are responsible.
>>>>      skb_pull(skb, sizeof(struct ipv6hdr));
>>>>      skb_reset_transport_header(skb);
>>>>      memcpy(skb_push(skb, hc06_ptr - head), head, hc06_ptr - head);
>>>>      skb_reset_network_header(skb);
>>>>
>>>> I don't think we can do this as the skb is being used in other parts of the
>>>> ip stack. Hence when the ipv6 header is read elsewhere the addresses become
>>>> corrupt as they have been overwritten by the 6lowpan compressed header.
>>>>
>>>> Any ideas on how to fix this?
>>>>
>>> yes, but I don't believe that this makes trouble. <--- or only makes
>>> trouble by replacing data, see below.
>>>
>>> It's called by a callback of header_ops [0].
>>>
>>> This is for generating the mac header with address information from
>>> neighbor discovery cache (mainly destination address) and source
>>> addresse (mainly netdev->dev_addr).
>>>
>>> Another example of this function is ethernet. [1]
>>>
>>> On [1] you will se that the ethernet header will created there.
>>>
>>> - Get data from skb for ethhdr (ethernet header)
>>>    struct ethhdr *eth = (struct ethhdr *)skb_push(skb, ETH_HLEN);
>>>
>>> - memcpy(eth->h_source, saddr, ETH_ALEN); <-- source address
>>>
>>> - memcpy(eth->h_dest, daddr, ETH_ALEN); <-- destination address.
>>>
>>>
>>> So they using the callback there to manipulate the skb here.
>>>
>>> Another idea is that, maybe we can ADD data but not REPLACING existing
>>> data with that. I don't know right now.
>> I think what is done in ethernet is fine as they are adding data. But in the
>> lowpan_header_compress we are doing a skb_pull to remove the existing ipv6
>> header before adding on the new 6lowpan header, so we are replacing the
>> existing data.
> Yep, then you could try to test what I did there. Simple save the
> address information in reserved skb data and then replacing data in
> lowpan_xmit callback. This also solves the capture on lowpan interfaces.
>
> lowpan_xmit should be save to replacing the header.
I am not 100% sure it is safe to replace the header at any point. How 
can you guarantee that the skb has been read by the user application RAW 
socket by this point? The user might not read the socket for 
seconds/minutes/hours.

Also who should free the skb?

Is is worth clarifying this with the netdev mailing list?

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 09:27:05AM +0100, Simon Vincent wrote:
...
> >Yep, then you could try to test what I did there. Simple save the
> >address information in reserved skb data and then replacing data in
> >lowpan_xmit callback. This also solves the capture on lowpan interfaces.
> >
> >lowpan_xmit should be save to replacing the header.
> I am not 100% sure it is safe to replace the header at any point. How can

me, too. We need to check that.

> you guarantee that the skb has been read by the user application RAW socket
> by this point? The user might not read the socket for seconds/minutes/hours.
> 

no this can't happen. I need to dig into the code and running some
ftrace to check if this can work. The RAW socket should not involved here.

My theory is:

The current behaviour is maybe (Maybe, because it's an assumption).

 - IPv6 neighbor discovery magic
   - calling create callback of header ops
     This callback is only there to add data there not replacing data.
     We do REPLACING the data so we can't parse the skb IPv6 header
     anymore.
   - After calling create callback of header ops IPv6 stack parse the
     IPv6 header data which is inside of skb. We already replaced the
     IPv6 header with 6LoWPAN header, so we can't parse it anymore but
     IPv6 does it. -> WEIRD things happen.
   - Finally lowpan interface calling xmit callback, this is at the end
     of doing Layer 3 things, so we should sure that nobody parse it there.
     What we do there is to queue the skb in the lower wpan interface.


The solution would be (maybe):
 - IPv6 neighbor discovery magic
   - calling create callback of header ops
     Putting necessary address information in skb reserved data. What
     this means, see below. We don't replace the IPv6 header.
   - After calling create callback of header ops IPv6 stack parse the
     IPv6 header data which is inside of skb. We don't replace the
     IPv6 header with 6LoWPAN header. -> all things are fine.
   - Finally lowpan interface calling xmit callback, this is at the end
     of doing Layer 3 things. Here we start to replacing IPv6 header
     with 6LoWPAN header. The thing is, we need the address information
     from create callback, that's why we saved it in skb reserved data.

  

What is the skb reserved data?

We have a headroom and tailroom and then 4 pointers in the skb.
_________________________________________________________________
|  headroom     |       used skb data....      |     tailroom   |
^---------------^------------------------------^----------------^

The '^' chars are pointers (don't know the real name in sk_buff struct,
but data should be pointed at begin of "used skb data".

What I did in the rework is:
  - check if headroom is big enough to save "struct lowpan_addr"
  - place lowpan_addr information in pointer:
    (skb->data - (sizeof(struct lowpan_addr))).

So this will crash when somebody make a skb_pull and skb_push in the
middle of callback "create" of header_ops and lowpan_xmit callback.
And manipulate data in there, or he do that like above without calling
skb_foo functions.
I don't know if there exist a state that this could happen.
But I think in create callback of header_ops we have already the full
IPv6 header, so I don't see why some IPv6 stack implementation should
replacing it afterwards.

Do you follow this here?

> Also who should free the skb?
> 
> Is is worth clarifying this with the netdev mailing list?

To make a real solution for this issue, of course. I have only this one
idea. To save address information into skb reserved data. We can't use
skb->cb here, I already want to tried that. David Miller said to me that
skb->cb could be overwritten by traffic control framework.

I would first give a try if the above one fix your issue with RAW
sockets.

I cc Marc here, a co-worker of mine. Maybe he has also some other
solution for this.

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

On 19/09/14 10:33, Alexander Aring wrote:
> On Fri, Sep 19, 2014 at 09:27:05AM +0100, Simon Vincent wrote:
> ...
>>> Yep, then you could try to test what I did there. Simple save the
>>> address information in reserved skb data and then replacing data in
>>> lowpan_xmit callback. This also solves the capture on lowpan interfaces.
>>>
>>> lowpan_xmit should be save to replacing the header.
>> I am not 100% sure it is safe to replace the header at any point. How can
> me, too. We need to check that.
>
>> you guarantee that the skb has been read by the user application RAW socket
>> by this point? The user might not read the socket for seconds/minutes/hours.
>>
> no this can't happen. I need to dig into the code and running some
> ftrace to check if this can work. The RAW socket should not involved here.
>
> My theory is:
>
> The current behaviour is maybe (Maybe, because it's an assumption).
>
>   - IPv6 neighbor discovery magic
>     - calling create callback of header ops
>       This callback is only there to add data there not replacing data.
>       We do REPLACING the data so we can't parse the skb IPv6 header
>       anymore.
>     - After calling create callback of header ops IPv6 stack parse the
>       IPv6 header data which is inside of skb. We already replaced the
>       IPv6 header with 6LoWPAN header, so we can't parse it anymore but
>       IPv6 does it. -> WEIRD things happen.
>     - Finally lowpan interface calling xmit callback, this is at the end
>       of doing Layer 3 things, so we should sure that nobody parse it there.
>       What we do there is to queue the skb in the lower wpan interface.
>
>
> The solution would be (maybe):
>   - IPv6 neighbor discovery magic
>     - calling create callback of header ops
>       Putting necessary address information in skb reserved data. What
>       this means, see below. We don't replace the IPv6 header.
>     - After calling create callback of header ops IPv6 stack parse the
>       IPv6 header data which is inside of skb. We don't replace the
>       IPv6 header with 6LoWPAN header. -> all things are fine.
>     - Finally lowpan interface calling xmit callback, this is at the end
>       of doing Layer 3 things. Here we start to replacing IPv6 header
>       with 6LoWPAN header. The thing is, we need the address information
>       from create callback, that's why we saved it in skb reserved data.
>
>    

It does not look like it is safe to do it at xmit callback either. I 
have added debug printing out skb_shinfo(skb)->dataref in the compress 
function as well as the lowpan_xmit function and there are 3 refs to the 
data at both points. This makes sense as the packet would be sent to the 
802.15.4/ethernet/rawsocket as it is a multicast packet.
I think the only solution is to do a full copy of the packet in the 
6lowpan layer. Then we can modify the headers as much as we like. I know 
there is an overhead doing this but I can't see another way of avoiding 
this issue.

Simon
>
> What is the skb reserved data?
>
> We have a headroom and tailroom and then 4 pointers in the skb.
> _________________________________________________________________
> |  headroom     |       used skb data....      |     tailroom   |
> ^---------------^------------------------------^----------------^
>
> The '^' chars are pointers (don't know the real name in sk_buff struct,
> but data should be pointed at begin of "used skb data".
>
> What I did in the rework is:
>    - check if headroom is big enough to save "struct lowpan_addr"
>    - place lowpan_addr information in pointer:
>      (skb->data - (sizeof(struct lowpan_addr))).
>
> So this will crash when somebody make a skb_pull and skb_push in the
> middle of callback "create" of header_ops and lowpan_xmit callback.
> And manipulate data in there, or he do that like above without calling
> skb_foo functions.
> I don't know if there exist a state that this could happen.
> But I think in create callback of header_ops we have already the full
> IPv6 header, so I don't see why some IPv6 stack implementation should
> replacing it afterwards.
>
> Do you follow this here?
>
>> Also who should free the skb?
>>
>> Is is worth clarifying this with the netdev mailing list?
> To make a real solution for this issue, of course. I have only this one
> idea. To save address information into skb reserved data. We can't use
> skb->cb here, I already want to tried that. David Miller said to me that
> skb->cb could be overwritten by traffic control framework.
>
> I would first give a try if the above one fix your issue with RAW
> sockets.
>
> I cc Marc here, a co-worker of mine. Maybe he has also some other
> solution for this.
>
> - Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 10:57:32AM +0100, Simon Vincent wrote:
> 
> On 19/09/14 10:33, Alexander Aring wrote:
> >On Fri, Sep 19, 2014 at 09:27:05AM +0100, Simon Vincent wrote:
> >...
> >>>Yep, then you could try to test what I did there. Simple save the
> >>>address information in reserved skb data and then replacing data in
> >>>lowpan_xmit callback. This also solves the capture on lowpan interfaces.
> >>>
> >>>lowpan_xmit should be save to replacing the header.
> >>I am not 100% sure it is safe to replace the header at any point. How can
> >me, too. We need to check that.
> >
> >>you guarantee that the skb has been read by the user application RAW socket
> >>by this point? The user might not read the socket for seconds/minutes/hours.
> >>
> >no this can't happen. I need to dig into the code and running some
> >ftrace to check if this can work. The RAW socket should not involved here.
> >
> >My theory is:
> >
> >The current behaviour is maybe (Maybe, because it's an assumption).
> >
> >  - IPv6 neighbor discovery magic
> >    - calling create callback of header ops
> >      This callback is only there to add data there not replacing data.
> >      We do REPLACING the data so we can't parse the skb IPv6 header
> >      anymore.
> >    - After calling create callback of header ops IPv6 stack parse the
> >      IPv6 header data which is inside of skb. We already replaced the
> >      IPv6 header with 6LoWPAN header, so we can't parse it anymore but
> >      IPv6 does it. -> WEIRD things happen.
> >    - Finally lowpan interface calling xmit callback, this is at the end
> >      of doing Layer 3 things, so we should sure that nobody parse it there.
> >      What we do there is to queue the skb in the lower wpan interface.
> >
> >
> >The solution would be (maybe):
> >  - IPv6 neighbor discovery magic
> >    - calling create callback of header ops
> >      Putting necessary address information in skb reserved data. What
> >      this means, see below. We don't replace the IPv6 header.
> >    - After calling create callback of header ops IPv6 stack parse the
> >      IPv6 header data which is inside of skb. We don't replace the
> >      IPv6 header with 6LoWPAN header. -> all things are fine.
> >    - Finally lowpan interface calling xmit callback, this is at the end
> >      of doing Layer 3 things. Here we start to replacing IPv6 header
> >      with 6LoWPAN header. The thing is, we need the address information
> >      from create callback, that's why we saved it in skb reserved data.
> >
> 
> It does not look like it is safe to do it at xmit callback either. I have
> added debug printing out skb_shinfo(skb)->dataref in the compress function
> as well as the lowpan_xmit function and there are 3 refs to the data at both
> points. This makes sense as the packet would be sent to the
> 802.15.4/ethernet/rawsocket as it is a multicast packet.
> I think the only solution is to do a full copy of the packet in the 6lowpan
> layer. Then we can modify the headers as much as we like. I know there is an
> overhead doing this but I can't see another way of avoiding this issue.
> 

Can't follow here, I mean inside the xmit callback there is no layer 3
(IPv6) parsing, etc. anymore. So skb_network_header parsing as IPv6
header can't occur here.

Can you send a RFC about your solution? Then it's maybe more clear for
me.

- Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

Hi Simon,

On Fri, Sep 19, 2014 at 12:15:34PM +0200, Alexander Aring wrote:
...
> > 
> > It does not look like it is safe to do it at xmit callback either. I have
> > added debug printing out skb_shinfo(skb)->dataref in the compress function
> > as well as the lowpan_xmit function and there are 3 refs to the data at both
> > points. This makes sense as the packet would be sent to the
> > 802.15.4/ethernet/rawsocket as it is a multicast packet.
> > I think the only solution is to do a full copy of the packet in the 6lowpan
> > layer. Then we can modify the headers as much as we like. I know there is an
> > overhead doing this but I can't see another way of avoiding this issue.
> > 
> 

I found now the "dev_hard_header" call inside the IPv6 implementation [0]. It's
a wrapper to call the create callback of header ops. So what I understand about
the issue that, somebody assume that there is still an IPv6 header and
not 6LoWPAN header. But we replacing data there so some implementation
will parse garbage.

[0] shows three function which call dev_hard_header:

 - neigh_compat_output
 - neigh_resolve_output
 - neigh_connected_output

So these three we need to look at it and see if there is some code which
parse an IPv6 header afterwards. Right?

I will try to do some grep magic and find a case when this happens.

- Alex

[0] http://lxr.free-electrons.com/source/net/core/neighbour.c#L1294

Re: 6lowpan raw socket problems

[Alexander Aring]

On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
> > I have created a small test program that shows this problem. It looks like a
> > race condition as sometimes the addresses are not corrupt.
> > 
> 
> Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
> Don't know right now.
> 
> > It looks like if the RAW socket gets the packet before the packet hits the
> > 6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
> > before the RAW socket gets the packet then the addresses are corrupt.
> > 
> > The test program can be found here.
> > https://github.com/xsilon/sockdebug
> > 
> > I will continue debugging!
> > 
> ok, good luck.
>

I gave this now a try, how can I see the issue now?

I see on output:

recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]

this address doesn't exist in my network.

I can also upload wpan wireshark logs and lowpan wireshark logs, if you
like.

In sockdebug I changed also "const char* src_string =" to one of my
lowpan addresses. Simon are you still here to debug this issue with me?
:-)

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

On 19/09/14 12:08, Alexander Aring wrote:
> On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
>> On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
>>> I have created a small test program that shows this problem. It looks like a
>>> race condition as sometimes the addresses are not corrupt.
>>>
>> Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
>> Don't know right now.
>>
>>> It looks like if the RAW socket gets the packet before the packet hits the
>>> 6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
>>> before the RAW socket gets the packet then the addresses are corrupt.
>>>
>>> The test program can be found here.
>>> https://github.com/xsilon/sockdebug
>>>
>>> I will continue debugging!
>>>
>> ok, good luck.
>>
> I gave this now a try, how can I see the issue now?
>
> I see on output:
>
> recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
>
> this address doesn't exist in my network.
>
> I can also upload wpan wireshark logs and lowpan wireshark logs, if you
> like.
>
> In sockdebug I changed also "const char* src_string =" to one of my
> lowpan addresses. Simon are you still here to debug this issue with me?
> :-)
Yes this is the same error I am seeing. I find that sometimes the recv 
address is correct but mostly you get the corrupt address as the ipv6 
header has been overwritten by our compressed 6lowpan header.

If you comment out the 6lowpan header compression function it solves the 
problem.

I am trying to understand how the network stack handles skbs. As it is a 
multicast packet it will be sent out on  802.15.4, raw socket and any 
other interfaces you have but it looks like in this case the interfaces 
all get a skb pointing to the same data. Therefore when we replace the 
ipv6 header with a compressed version everyone else still thinks there 
is a normal ipv6 header still there and therefore gets corrupt data. 
Should each interface get a copy of the data? E.g. the ethernet, wifi, 
802.15.4 and raw socket all get a copy of the skb data not a clone?

Maybe normally each interface will get a copy of the skb so they can 
attach their own mac header but in the case of the RAW socket they don't 
bother doing a copy as they don't need to add a header for the socket. 
But then we come along and destroy the ipv6 header!!

Just a theory!

Simon

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 12:27:54PM +0100, Simon Vincent wrote:
> 
> On 19/09/14 12:08, Alexander Aring wrote:
> >On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
> >>On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
> >>>I have created a small test program that shows this problem. It looks like a
> >>>race condition as sometimes the addresses are not corrupt.
> >>>
> >>Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
> >>Don't know right now.
> >>
> >>>It looks like if the RAW socket gets the packet before the packet hits the
> >>>6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
> >>>before the RAW socket gets the packet then the addresses are corrupt.
> >>>
> >>>The test program can be found here.
> >>>https://github.com/xsilon/sockdebug
> >>>
> >>>I will continue debugging!
> >>>
> >>ok, good luck.
> >>
> >I gave this now a try, how can I see the issue now?
> >
> >I see on output:
> >
> >recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
> >
> >this address doesn't exist in my network.
> >
> >I can also upload wpan wireshark logs and lowpan wireshark logs, if you
> >like.
> >
> >In sockdebug I changed also "const char* src_string =" to one of my
> >lowpan addresses. Simon are you still here to debug this issue with me?
> >:-)
> Yes this is the same error I am seeing. I find that sometimes the recv
> address is correct but mostly you get the corrupt address as the ipv6 header
> has been overwritten by our compressed 6lowpan header.
> 
> If you comment out the 6lowpan header compression function it solves the
> problem.

okay, then I dig now into the issue why the address is garbage.

> 
> I am trying to understand how the network stack handles skbs. As it is a
> multicast packet it will be sent out on  802.15.4, raw socket and any other
> interfaces you have but it looks like in this case the interfaces all get a
> skb pointing to the same data. Therefore when we replace the ipv6 header
> with a compressed version everyone else still thinks there is a normal ipv6
> header still there and therefore gets corrupt data. Should each interface
> get a copy of the data? E.g. the ethernet, wifi, 802.15.4 and raw socket all
> get a copy of the skb data not a clone?
> 
> Maybe normally each interface will get a copy of the skb so they can attach
> their own mac header but in the case of the RAW socket they don't bother
> doing a copy as they don't need to add a header for the socket. But then we
> come along and destroy the ipv6 header!!
> 
> Just a theory!
> 

okay, there exists a lot of there. I know what you saying because the
data buffer is shared there exist race conditions because some other skb
has in the next step a 6LoWPAN header, if I understand that correctly.

- Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 01:45:49PM +0200, Alexander Aring wrote:
> On Fri, Sep 19, 2014 at 12:27:54PM +0100, Simon Vincent wrote:
> > 
> > On 19/09/14 12:08, Alexander Aring wrote:
> > >On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
> > >>On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
> > >>>I have created a small test program that shows this problem. It looks like a
> > >>>race condition as sometimes the addresses are not corrupt.
> > >>>
> > >>Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
> > >>Don't know right now.
> > >>
> > >>>It looks like if the RAW socket gets the packet before the packet hits the
> > >>>6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
> > >>>before the RAW socket gets the packet then the addresses are corrupt.
> > >>>
> > >>>The test program can be found here.
> > >>>https://github.com/xsilon/sockdebug
> > >>>
> > >>>I will continue debugging!
> > >>>
> > >>ok, good luck.
> > >>
> > >I gave this now a try, how can I see the issue now?
> > >
> > >I see on output:
> > >
> > >recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
> > >
> > >this address doesn't exist in my network.
> > >
> > >I can also upload wpan wireshark logs and lowpan wireshark logs, if you
> > >like.
> > >
> > >In sockdebug I changed also "const char* src_string =" to one of my
> > >lowpan addresses. Simon are you still here to debug this issue with me?
> > >:-)
> > Yes this is the same error I am seeing. I find that sometimes the recv
> > address is correct but mostly you get the corrupt address as the ipv6 header
> > has been overwritten by our compressed 6lowpan header.
> > 
> > If you comment out the 6lowpan header compression function it solves the
> > problem.
> 
> okay, then I dig now into the issue why the address is garbage.
> 

funny, I see a pattern:

recv_raw_icmp[fe80:0:41:c85d:cdab:ffff:bbaa:aaaa%lowpan0->?]
recv_raw_icmp[fe80:0:41:c85e:cdab:ffff:bbaa:aaaa%lowpan0->?]
recv_raw_icmp[fe80:0:41:c85f:cdab:ffff:bbaa:aaaa%lowpan0->?]
recv_raw_icmp[fe80:0:41:c860:cdab:ffff:bbaa:aaaa%lowpan0->?]
recv_raw_icmp[fe80:0:41:c861:cdab:ffff:bbaa:aaaa%lowpan0->?]
recv_raw_icmp[fe80:0:41:c862:cdab:ffff:bbaa:aaaa%lowpan0->?]
recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
                          ^^

The ^^ is looking like there is some sequence number which is increasing. :-)

- Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 01:50:05PM +0200, Alexander Aring wrote:
> On Fri, Sep 19, 2014 at 01:45:49PM +0200, Alexander Aring wrote:
> > On Fri, Sep 19, 2014 at 12:27:54PM +0100, Simon Vincent wrote:
> > > 
> > > On 19/09/14 12:08, Alexander Aring wrote:
> > > >On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
> > > >>On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
> > > >>>I have created a small test program that shows this problem. It looks like a
> > > >>>race condition as sometimes the addresses are not corrupt.
> > > >>>
> > > >>Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
> > > >>Don't know right now.
> > > >>
> > > >>>It looks like if the RAW socket gets the packet before the packet hits the
> > > >>>6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
> > > >>>before the RAW socket gets the packet then the addresses are corrupt.
> > > >>>
> > > >>>The test program can be found here.
> > > >>>https://github.com/xsilon/sockdebug
> > > >>>
> > > >>>I will continue debugging!
> > > >>>
> > > >>ok, good luck.
> > > >>
> > > >I gave this now a try, how can I see the issue now?
> > > >
> > > >I see on output:
> > > >
> > > >recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
> > > >
> > > >this address doesn't exist in my network.
> > > >
> > > >I can also upload wpan wireshark logs and lowpan wireshark logs, if you
> > > >like.
> > > >
> > > >In sockdebug I changed also "const char* src_string =" to one of my
> > > >lowpan addresses. Simon are you still here to debug this issue with me?
> > > >:-)
> > > Yes this is the same error I am seeing. I find that sometimes the recv
> > > address is correct but mostly you get the corrupt address as the ipv6 header
> > > has been overwritten by our compressed 6lowpan header.
> > > 
> > > If you comment out the 6lowpan header compression function it solves the
> > > problem.
> > 
> > okay, then I dig now into the issue why the address is garbage.
> > 
> 
> funny, I see a pattern:
> 
> recv_raw_icmp[fe80:0:41:c85d:cdab:ffff:bbaa:aaaa%lowpan0->?]
> recv_raw_icmp[fe80:0:41:c85e:cdab:ffff:bbaa:aaaa%lowpan0->?]
> recv_raw_icmp[fe80:0:41:c85f:cdab:ffff:bbaa:aaaa%lowpan0->?]
> recv_raw_icmp[fe80:0:41:c860:cdab:ffff:bbaa:aaaa%lowpan0->?]
> recv_raw_icmp[fe80:0:41:c861:cdab:ffff:bbaa:aaaa%lowpan0->?]
> recv_raw_icmp[fe80:0:41:c862:cdab:ffff:bbaa:aaaa%lowpan0->?]
> recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
>                           ^^
> 
> The ^^ is looking like there is some sequence number which is increasing. :-)
> 
now compared with wireshark dump, it looks like that at :41:c8... begins
the mac header. crazy...

We come closer to the issue.

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

On 19/09/14 12:45, Alexander Aring wrote:
> On Fri, Sep 19, 2014 at 12:27:54PM +0100, Simon Vincent wrote:
>> On 19/09/14 12:08, Alexander Aring wrote:
>>> On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
>>>> On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
>>>>> I have created a small test program that shows this problem. It looks like a
>>>>> race condition as sometimes the addresses are not corrupt.
>>>>>
>>>> Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
>>>> Don't know right now.
>>>>
>>>>> It looks like if the RAW socket gets the packet before the packet hits the
>>>>> 6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
>>>>> before the RAW socket gets the packet then the addresses are corrupt.
>>>>>
>>>>> The test program can be found here.
>>>>> https://github.com/xsilon/sockdebug
>>>>>
>>>>> I will continue debugging!
>>>>>
>>>> ok, good luck.
>>>>
>>> I gave this now a try, how can I see the issue now?
>>>
>>> I see on output:
>>>
>>> recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
>>>
>>> this address doesn't exist in my network.
>>>
>>> I can also upload wpan wireshark logs and lowpan wireshark logs, if you
>>> like.
>>>
>>> In sockdebug I changed also "const char* src_string =" to one of my
>>> lowpan addresses. Simon are you still here to debug this issue with me?
>>> :-)
>> Yes this is the same error I am seeing. I find that sometimes the recv
>> address is correct but mostly you get the corrupt address as the ipv6 header
>> has been overwritten by our compressed 6lowpan header.
>>
>> If you comment out the 6lowpan header compression function it solves the
>> problem.
> okay, then I dig now into the issue why the address is garbage.
>
>> I am trying to understand how the network stack handles skbs. As it is a
>> multicast packet it will be sent out on  802.15.4, raw socket and any other
>> interfaces you have but it looks like in this case the interfaces all get a
>> skb pointing to the same data. Therefore when we replace the ipv6 header
>> with a compressed version everyone else still thinks there is a normal ipv6
>> header still there and therefore gets corrupt data. Should each interface
>> get a copy of the data? E.g. the ethernet, wifi, 802.15.4 and raw socket all
>> get a copy of the skb data not a clone?
>>
>> Maybe normally each interface will get a copy of the skb so they can attach
>> their own mac header but in the case of the RAW socket they don't bother
>> doing a copy as they don't need to add a header for the socket. But then we
>> come along and destroy the ipv6 header!!
>>
>> Just a theory!
>>
> okay, there exists a lot of there. I know what you saying because the
> data buffer is shared there exist race conditions because some other skb
> has in the next step a 6LoWPAN header, if I understand that correctly.
>
Yes I think the problem is we are sharing the databuffer and modifying 
the contents. We should probably be given a copy of the data buffer. I 
can't find the code that decides if we get a copy or clone of the skb.

For example in ip6_finish_output2 because it is a multicast packet the 
skb is cloned (so same data buffer) and sent to the loopback interface. 
This loopback interface is going to expect the ipv6 header to be intact.

Simon

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 01:00:02PM +0100, Simon Vincent wrote:
> 
> On 19/09/14 12:45, Alexander Aring wrote:
> >On Fri, Sep 19, 2014 at 12:27:54PM +0100, Simon Vincent wrote:
> >>On 19/09/14 12:08, Alexander Aring wrote:
> >>>On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
> >>>>On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
> >>>>>I have created a small test program that shows this problem. It looks like a
> >>>>>race condition as sometimes the addresses are not corrupt.
> >>>>>
> >>>>Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
> >>>>Don't know right now.
> >>>>
> >>>>>It looks like if the RAW socket gets the packet before the packet hits the
> >>>>>6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
> >>>>>before the RAW socket gets the packet then the addresses are corrupt.
> >>>>>
> >>>>>The test program can be found here.
> >>>>>https://github.com/xsilon/sockdebug
> >>>>>
> >>>>>I will continue debugging!
> >>>>>
> >>>>ok, good luck.
> >>>>
> >>>I gave this now a try, how can I see the issue now?
> >>>
> >>>I see on output:
> >>>
> >>>recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
> >>>
> >>>this address doesn't exist in my network.
> >>>
> >>>I can also upload wpan wireshark logs and lowpan wireshark logs, if you
> >>>like.
> >>>
> >>>In sockdebug I changed also "const char* src_string =" to one of my
> >>>lowpan addresses. Simon are you still here to debug this issue with me?
> >>>:-)
> >>Yes this is the same error I am seeing. I find that sometimes the recv
> >>address is correct but mostly you get the corrupt address as the ipv6 header
> >>has been overwritten by our compressed 6lowpan header.
> >>
> >>If you comment out the 6lowpan header compression function it solves the
> >>problem.
> >okay, then I dig now into the issue why the address is garbage.
> >
> >>I am trying to understand how the network stack handles skbs. As it is a
> >>multicast packet it will be sent out on  802.15.4, raw socket and any other
> >>interfaces you have but it looks like in this case the interfaces all get a
> >>skb pointing to the same data. Therefore when we replace the ipv6 header
> >>with a compressed version everyone else still thinks there is a normal ipv6
> >>header still there and therefore gets corrupt data. Should each interface
> >>get a copy of the data? E.g. the ethernet, wifi, 802.15.4 and raw socket all
> >>get a copy of the skb data not a clone?
> >>
> >>Maybe normally each interface will get a copy of the skb so they can attach
> >>their own mac header but in the case of the RAW socket they don't bother
> >>doing a copy as they don't need to add a header for the socket. But then we
> >>come along and destroy the ipv6 header!!
> >>
> >>Just a theory!
> >>
> >okay, there exists a lot of there. I know what you saying because the
> >data buffer is shared there exist race conditions because some other skb
> >has in the next step a 6LoWPAN header, if I understand that correctly.
> >
> Yes I think the problem is we are sharing the databuffer and modifying the
> contents. We should probably be given a copy of the data buffer. I can't
> find the code that decides if we get a copy or clone of the skb.
> 

yea, IPv6 stack is complicated. :-)

And if you found it, it would be complicated to make any change on it,
we are only an adapation layer... All 6LoWPAN runtime decisions in IPv6
are bad and do some change. But the IPv6 are not evil, we can talk with
them that we have something which works on both stacks and doesn't
decrease much the performance of IPv6. If we have something like this
then we have a mainline solution.

> For example in ip6_finish_output2 because it is a multicast packet the skb
> is cloned (so same data buffer) and sent to the loopback interface. This
> loopback interface is going to expect the ipv6 header to be intact.
> 
ok I will remember and check it.


- Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 02:06:30PM +0200, Alexander Aring wrote:
> On Fri, Sep 19, 2014 at 01:00:02PM +0100, Simon Vincent wrote:
> > 
> > On 19/09/14 12:45, Alexander Aring wrote:
> > >On Fri, Sep 19, 2014 at 12:27:54PM +0100, Simon Vincent wrote:
> > >>On 19/09/14 12:08, Alexander Aring wrote:
> > >>>On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
> > >>>>On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
> > >>>>>I have created a small test program that shows this problem. It looks like a
> > >>>>>race condition as sometimes the addresses are not corrupt.
> > >>>>>
> > >>>>Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
> > >>>>Don't know right now.
> > >>>>
> > >>>>>It looks like if the RAW socket gets the packet before the packet hits the
> > >>>>>6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
> > >>>>>before the RAW socket gets the packet then the addresses are corrupt.
> > >>>>>
> > >>>>>The test program can be found here.
> > >>>>>https://github.com/xsilon/sockdebug
> > >>>>>
> > >>>>>I will continue debugging!
> > >>>>>
> > >>>>ok, good luck.
> > >>>>
> > >>>I gave this now a try, how can I see the issue now?
> > >>>
> > >>>I see on output:
> > >>>
> > >>>recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
> > >>>
> > >>>this address doesn't exist in my network.
> > >>>
> > >>>I can also upload wpan wireshark logs and lowpan wireshark logs, if you
> > >>>like.
> > >>>
> > >>>In sockdebug I changed also "const char* src_string =" to one of my
> > >>>lowpan addresses. Simon are you still here to debug this issue with me?
> > >>>:-)
> > >>Yes this is the same error I am seeing. I find that sometimes the recv
> > >>address is correct but mostly you get the corrupt address as the ipv6 header
> > >>has been overwritten by our compressed 6lowpan header.
> > >>
> > >>If you comment out the 6lowpan header compression function it solves the
> > >>problem.
> > >okay, then I dig now into the issue why the address is garbage.
> > >
> > >>I am trying to understand how the network stack handles skbs. As it is a
> > >>multicast packet it will be sent out on  802.15.4, raw socket and any other
> > >>interfaces you have but it looks like in this case the interfaces all get a
> > >>skb pointing to the same data. Therefore when we replace the ipv6 header
> > >>with a compressed version everyone else still thinks there is a normal ipv6
> > >>header still there and therefore gets corrupt data. Should each interface
> > >>get a copy of the data? E.g. the ethernet, wifi, 802.15.4 and raw socket all
> > >>get a copy of the skb data not a clone?
> > >>
> > >>Maybe normally each interface will get a copy of the skb so they can attach
> > >>their own mac header but in the case of the RAW socket they don't bother
> > >>doing a copy as they don't need to add a header for the socket. But then we
> > >>come along and destroy the ipv6 header!!
> > >>
> > >>Just a theory!
> > >>
> > >okay, there exists a lot of there. I know what you saying because the
> > >data buffer is shared there exist race conditions because some other skb
> > >has in the next step a 6LoWPAN header, if I understand that correctly.
> > >
> > Yes I think the problem is we are sharing the databuffer and modifying the
> > contents. We should probably be given a copy of the data buffer. I can't
> > find the code that decides if we get a copy or clone of the skb.
> > 
> 
> yea, IPv6 stack is complicated. :-)
> 
> And if you found it, it would be complicated to make any change on it,
> we are only an adapation layer... All 6LoWPAN runtime decisions in IPv6
> are bad and do some change. But the IPv6 are not evil, we can talk with
> them that we have something which works on both stacks and doesn't
> decrease much the performance of IPv6. If we have something like this
> then we have a mainline solution.
> 

mhh, take a look on skb_unshare - make a copy of a shared buffe [0].

Seems that we could use that to have a copy of the buffer. Don't know if
this can work, because we are inside of callback and the caller lost the
reference then.

[0] http://lxr.free-electrons.com/source/include/linux/skbuff.h#L1107

Re: 6lowpan raw socket problems

[Simon Vincent]

On 19/09/14 13:38, Alexander Aring wrote:
> On Fri, Sep 19, 2014 at 02:06:30PM +0200, Alexander Aring wrote:
>> On Fri, Sep 19, 2014 at 01:00:02PM +0100, Simon Vincent wrote:
>>> On 19/09/14 12:45, Alexander Aring wrote:
>>>> On Fri, Sep 19, 2014 at 12:27:54PM +0100, Simon Vincent wrote:
>>>>> On 19/09/14 12:08, Alexander Aring wrote:
>>>>>> On Thu, Sep 18, 2014 at 04:19:11PM +0200, Alexander Aring wrote:
>>>>>>> On Thu, Sep 18, 2014 at 03:02:17PM +0100, Simon Vincent wrote:
>>>>>>>> I have created a small test program that shows this problem. It looks like a
>>>>>>>> race condition as sometimes the addresses are not corrupt.
>>>>>>>>
>>>>>>> Mhh maybe some used after freed and then we copy somewhere garbage sometimes.
>>>>>>> Don't know right now.
>>>>>>>
>>>>>>>> It looks like if the RAW socket gets the packet before the packet hits the
>>>>>>>> 6lowpan layer the addresses are fine. If the packet hits the 6lowpan layer
>>>>>>>> before the RAW socket gets the packet then the addresses are corrupt.
>>>>>>>>
>>>>>>>> The test program can be found here.
>>>>>>>> https://github.com/xsilon/sockdebug
>>>>>>>>
>>>>>>>> I will continue debugging!
>>>>>>>>
>>>>>>> ok, good luck.
>>>>>>>
>>>>>> I gave this now a try, how can I see the issue now?
>>>>>>
>>>>>> I see on output:
>>>>>>
>>>>>> recv_raw_icmp[fe80:0:41:c863:cdab:ffff:bbaa:aaaa%lowpan0->?]
>>>>>>
>>>>>> this address doesn't exist in my network.
>>>>>>
>>>>>> I can also upload wpan wireshark logs and lowpan wireshark logs, if you
>>>>>> like.
>>>>>>
>>>>>> In sockdebug I changed also "const char* src_string =" to one of my
>>>>>> lowpan addresses. Simon are you still here to debug this issue with me?
>>>>>> :-)
>>>>> Yes this is the same error I am seeing. I find that sometimes the recv
>>>>> address is correct but mostly you get the corrupt address as the ipv6 header
>>>>> has been overwritten by our compressed 6lowpan header.
>>>>>
>>>>> If you comment out the 6lowpan header compression function it solves the
>>>>> problem.
>>>> okay, then I dig now into the issue why the address is garbage.
>>>>
>>>>> I am trying to understand how the network stack handles skbs. As it is a
>>>>> multicast packet it will be sent out on  802.15.4, raw socket and any other
>>>>> interfaces you have but it looks like in this case the interfaces all get a
>>>>> skb pointing to the same data. Therefore when we replace the ipv6 header
>>>>> with a compressed version everyone else still thinks there is a normal ipv6
>>>>> header still there and therefore gets corrupt data. Should each interface
>>>>> get a copy of the data? E.g. the ethernet, wifi, 802.15.4 and raw socket all
>>>>> get a copy of the skb data not a clone?
>>>>>
>>>>> Maybe normally each interface will get a copy of the skb so they can attach
>>>>> their own mac header but in the case of the RAW socket they don't bother
>>>>> doing a copy as they don't need to add a header for the socket. But then we
>>>>> come along and destroy the ipv6 header!!
>>>>>
>>>>> Just a theory!
>>>>>
>>>> okay, there exists a lot of there. I know what you saying because the
>>>> data buffer is shared there exist race conditions because some other skb
>>>> has in the next step a 6LoWPAN header, if I understand that correctly.
>>>>
>>> Yes I think the problem is we are sharing the databuffer and modifying the
>>> contents. We should probably be given a copy of the data buffer. I can't
>>> find the code that decides if we get a copy or clone of the skb.
>>>
>> yea, IPv6 stack is complicated. :-)
>>
>> And if you found it, it would be complicated to make any change on it,
>> we are only an adapation layer... All 6LoWPAN runtime decisions in IPv6
>> are bad and do some change. But the IPv6 are not evil, we can talk with
>> them that we have something which works on both stacks and doesn't
>> decrease much the performance of IPv6. If we have something like this
>> then we have a mainline solution.
>>
> mhh, take a look on skb_unshare - make a copy of a shared buffe [0].
>
> Seems that we could use that to have a copy of the buffer. Don't know if
> this can work, because we are inside of callback and the caller lost the
> reference then.
I tried that earlier! Didn't work as we lost the reference as you say.
I think if we do it inside lowpan_xmit like you do in your rework we 
might be ok. I will have a go.

Simon

Re: 6lowpan raw socket problems

[Alexander Aring]

Hi Simon,

On Fri, Sep 19, 2014 at 01:44:38PM +0100, Simon Vincent wrote:
...
> >mhh, take a look on skb_unshare - make a copy of a shared buffe [0].
> >
> >Seems that we could use that to have a copy of the buffer. Don't know if
> >this can work, because we are inside of callback and the caller lost the
> >reference then.
> I tried that earlier! Didn't work as we lost the reference as you say.
> I think if we do it inside lowpan_xmit like you do in your rework we might
> be ok. I will have a go.
> 

Yea, with combination of lowpan_xmit and do a:

skb = skb_unshare(skb, GFP_ATOMIC);

it's woooorrkkinnnnggg.

Output now is:

send_raw_icmp[fe80::2->ff02::1] (on if: 4): Success
recv_raw_icmp[fe80::2%lowpan0->?]
send_raw_icmp[fe80::2->ff02::1] (on if: 4): Success
recv_raw_icmp[fe80::2%lowpan0->?]
send_raw_icmp[fe80::2->ff02::1] (on if: 4): Success
recv_raw_icmp[fe80::2%lowpan0->?]
send_raw_icmp[fe80::2->ff02::1] (on if: 4): Success
recv_raw_icmp[fe80::2%lowpan0->?]
send_raw_icmp[fe80::2->ff02::1] (on if: 4): Success
recv_raw_icmp[fe80::2%lowpan0->?]
send_raw_icmp[fe80::2->ff02::1] (on if: 4): Success
recv_raw_icmp[fe80::2%lowpan0->?]
send_raw_icmp[fe80::2->ff02::1] (on if: 4): Success
recv_raw_icmp[fe80::2%lowpan0->?]
send_raw_icmp[fe80::2->ff02::1] (on if: 4): Success
recv_raw_icmp[fe80::2%lowpan0->?]


Thanks! If you like you can create a patch for the rework branch, then I will
send it when the rework is finished.


So combination what we always talked about solved this issue, crazy...

- Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 02:55:44PM +0200, Alexander Aring wrote:
> Hi Simon,
> 
> On Fri, Sep 19, 2014 at 01:44:38PM +0100, Simon Vincent wrote:
> ...
> > >mhh, take a look on skb_unshare - make a copy of a shared buffe [0].
> > >
> > >Seems that we could use that to have a copy of the buffer. Don't know if
> > >this can work, because we are inside of callback and the caller lost the
> > >reference then.
> > I tried that earlier! Didn't work as we lost the reference as you say.
> > I think if we do it inside lowpan_xmit like you do in your rework we might
> > be ok. I will have a go.
> > 
> 
> Yea, with combination of lowpan_xmit and do a:
> 
> skb = skb_unshare(skb, GFP_ATOMIC);
> 
> it's woooorrkkinnnnggg.
> 

I hope it's working now, it doesn't look like garbage anymore.

- Alex

Re: 6lowpan raw socket problems

[Simon Vincent]

I have implemented and tested the fix and it seems to work!
I used a combination of skb_unshare and pulled some of your wpan_rework 
changes in so the compression is done in lowpan_xmit.

Simon

On 19/09/14 13:57, Alexander Aring wrote:
> On Fri, Sep 19, 2014 at 02:55:44PM +0200, Alexander Aring wrote:
>> Hi Simon,
>>
>> On Fri, Sep 19, 2014 at 01:44:38PM +0100, Simon Vincent wrote:
>> ...
>>>> mhh, take a look on skb_unshare - make a copy of a shared buffe [0].
>>>>
>>>> Seems that we could use that to have a copy of the buffer. Don't know if
>>>> this can work, because we are inside of callback and the caller lost the
>>>> reference then.
>>> I tried that earlier! Didn't work as we lost the reference as you say.
>>> I think if we do it inside lowpan_xmit like you do in your rework we might
>>> be ok. I will have a go.
>>>
>> Yea, with combination of lowpan_xmit and do a:
>>
>> skb = skb_unshare(skb, GFP_ATOMIC);
>>
>> it's woooorrkkinnnnggg.
>>
> I hope it's working now, it doesn't look like garbage anymore.
>
> - Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 03:14:14PM +0100, Simon Vincent wrote:
> I have implemented and tested the fix and it seems to work!
ok.

> I used a combination of skb_unshare and pulled some of your wpan_rework
> changes in so the compression is done in lowpan_xmit.
> 

So you have a solution which based on current mainline. Do you like to
rebase it on linux-wpan and send patch?

- Alex

Re: 6lowpan raw socket problems

[Alexander Aring]

On Fri, Sep 19, 2014 at 04:21:16PM +0200, Alexander Aring wrote:
> On Fri, Sep 19, 2014 at 03:14:14PM +0100, Simon Vincent wrote:
> > I have implemented and tested the fix and it seems to work!
> ok.
> 
> > I used a combination of skb_unshare and pulled some of your wpan_rework
> > changes in so the compression is done in lowpan_xmit.
> > 
> 
> So you have a solution which based on current mainline. Do you like to
> rebase it on linux-wpan and send patch?
> 
and additional this fixes also doing wiresharl,tcpdump on lowpan
interfaces.

- Alex