Re: [PATCH v4 bluetooth] 6lowpan: fix incorrect return values in lowpan_rcv

[Martin Townsend <martin.townsend@xsilon.com>]

Hi Marcel,

Are you ok with me breaking this down into separate patches and submitting them individually to bluetooth-next?

- Martin.

On 06/10/14 08:12, Alexander Aring wrote:
> Hi,
>
> On Sun, Oct 05, 2014 at 10:00:59PM +0100, Martin Townsend wrote:
>> Hi Alex
>> On 05/10/14 18:50, Alexander Aring wrote:
>>> Hi Martin,
>>>
>>> On Wed, Oct 01, 2014 at 01:10:22PM +0100, Martin Townsend wrote:
>>>> Currently there are a number of error paths in the lowpan_rcv function that
>>>> free the skb before returning, the patch simplifies the receive path by
>>>> ensuring that the skb is only freed from this function.
>>>>
>>>> Passing the skb from 6lowpan up to the higher layers is not a
>>>> function of IPHC.  By moving it out of IPHC we also remove the
>>>> need to support error code returns with NET_RX codes.
>>> I think we should split the movement of "passing skb to higher layer"
>>> into a separate patch.
>> Wasn't it separated in v1 patch series and you asked me to combine into one
>> patch??
> yea, I know but it's hard to review and we need some "bisectable" git history.
>
>>>> It also makes the lowpan_rcv function more extendable as we
>>>> can support more compression schemes.
>>>>
>>>> With the above 2 lowpan_rcv is refactored so eliminate incorrect return values.
>>>>
>>>> Signed-off-by: Martin Townsend <martin.townsend@xsilon.com>
>>>> ---
>>>>  include/net/6lowpan.h         |  9 +++--
>>>>  net/6lowpan/iphc.c            | 92 ++++++++++++++++---------------------------
>>>>  net/bluetooth/6lowpan.c       | 52 ++++++++++++++++--------
>>>>  net/ieee802154/6lowpan_rtnl.c | 66 ++++++++++++++++++++-----------
>>>>  4 files changed, 117 insertions(+), 102 deletions(-)
>>>>
>>>> diff --git a/include/net/6lowpan.h b/include/net/6lowpan.h
>>>> index d184df1..05ff67e 100644
>>>> --- a/include/net/6lowpan.h
>>>> +++ b/include/net/6lowpan.h
>>>> @@ -374,10 +374,11 @@ lowpan_uncompress_size(const struct sk_buff *skb, u16 *dgram_offset)
>>>>  typedef int (*skb_delivery_cb)(struct sk_buff *skb, struct net_device *dev);
>>>> -int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>> -		const u8 *saddr, const u8 saddr_type, const u8 saddr_len,
>>>> -		const u8 *daddr, const u8 daddr_type, const u8 daddr_len,
>>>> -		u8 iphc0, u8 iphc1, skb_delivery_cb skb_deliver);
>>>> +int
>>>> +lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>> +		    const u8 *saddr, const u8 saddr_type, const u8 saddr_len,
>>>> +		    const u8 *daddr, const u8 daddr_type, const u8 daddr_len,
>>>> +		    u8 iphc0, u8 iphc1);
>>>>  int lowpan_header_compress(struct sk_buff *skb, struct net_device *dev,
>>>>  			unsigned short type, const void *_daddr,
>>>>  			const void *_saddr, unsigned int len);
>>>> diff --git a/net/6lowpan/iphc.c b/net/6lowpan/iphc.c
>>>> index 142eef5..3888357 100644
>>>> --- a/net/6lowpan/iphc.c
>>>> +++ b/net/6lowpan/iphc.c
>>>> @@ -171,37 +171,6 @@ static int uncompress_context_based_src_addr(struct sk_buff *skb,
>>>>  	return 0;
>>>>  }
>>>> -static int skb_deliver(struct sk_buff *skb, struct ipv6hdr *hdr,
>>>> -		       struct net_device *dev, skb_delivery_cb deliver_skb)
>>>> -{
>>>> -	struct sk_buff *new;
>>>> -	int stat;
>>>> -
>>>> -	new = skb_copy_expand(skb, sizeof(struct ipv6hdr), skb_tailroom(skb),
>>>> -			      GFP_ATOMIC);
>>>> -	kfree_skb(skb);
>>>> -
>>>> -	if (!new)
>>>> -		return -ENOMEM;
>>>> -
>>>> -	skb_push(new, sizeof(struct ipv6hdr));
>>>> -	skb_reset_network_header(new);
>>>> -	skb_copy_to_linear_data(new, hdr, sizeof(struct ipv6hdr));
>>>> -
>>>> -	new->protocol = htons(ETH_P_IPV6);
>>>> -	new->pkt_type = PACKET_HOST;
>>>> -	new->dev = dev;
>>>> -
>>>> -	raw_dump_table(__func__, "raw skb data dump before receiving",
>>>> -		       new->data, new->len);
>>>> -
>>>> -	stat = deliver_skb(new, dev);
>>>> -
>>>> -	kfree_skb(new);
>>>> -
>>>> -	return stat;
>>>> -}
>>>> -
>>>>  /* Uncompress function for multicast destination address,
>>>>   * when M bit is set.
>>>>   */
>>>> @@ -332,10 +301,11 @@ err:
>>>>  /* TTL uncompression values */
>>>>  static const u8 lowpan_ttl_values[] = { 0, 1, 64, 255 };
>>>> -int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>> -			const u8 *saddr, const u8 saddr_type, const u8 saddr_len,
>>>> -			const u8 *daddr, const u8 daddr_type, const u8 daddr_len,
>>>> -			u8 iphc0, u8 iphc1, skb_delivery_cb deliver_skb)
>>>> +int
>>>> +lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>> +		    const u8 *saddr, const u8 saddr_type, const u8 saddr_len,
>>>> +		    const u8 *daddr, const u8 daddr_type, const u8 daddr_len,
>>>> +		    u8 iphc0, u8 iphc1)
>>>>  {
>>>>  	struct ipv6hdr hdr = {};
>>>>  	u8 tmp, num_context = 0;
>>>> @@ -348,7 +318,7 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  	if (iphc1 & LOWPAN_IPHC_CID) {
>>>>  		pr_debug("CID flag is set, increase header with one\n");
>>>>  		if (lowpan_fetch_skb(skb, &num_context, sizeof(num_context)))
>>>> -			goto drop;
>>>> +			return -EINVAL;
>>>>  	}
>>>>  	hdr.version = 6;
>>>> @@ -360,7 +330,7 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  	 */
>>>>  	case 0: /* 00b */
>>>>  		if (lowpan_fetch_skb(skb, &tmp, sizeof(tmp)))
>>>> -			goto drop;
>>>> +			return -EINVAL;
>>>>  		memcpy(&hdr.flow_lbl, &skb->data[0], 3);
>>>>  		skb_pull(skb, 3);
>>>> @@ -373,7 +343,7 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  	 */
>>>>  	case 2: /* 10b */
>>>>  		if (lowpan_fetch_skb(skb, &tmp, sizeof(tmp)))
>>>> -			goto drop;
>>>> +			return -EINVAL;
>>>>  		hdr.priority = ((tmp >> 2) & 0x0f);
>>>>  		hdr.flow_lbl[0] = ((tmp << 6) & 0xC0) | ((tmp >> 2) & 0x30);
>>>> @@ -383,7 +353,7 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  	 */
>>>>  	case 1: /* 01b */
>>>>  		if (lowpan_fetch_skb(skb, &tmp, sizeof(tmp)))
>>>> -			goto drop;
>>>> +			return -EINVAL;
>>>>  		hdr.flow_lbl[0] = (skb->data[0] & 0x0F) | ((tmp >> 2) & 0x30);
>>>>  		memcpy(&hdr.flow_lbl[1], &skb->data[0], 2);
>>>> @@ -400,7 +370,7 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  	if ((iphc0 & LOWPAN_IPHC_NH_C) == 0) {
>>>>  		/* Next header is carried inline */
>>>>  		if (lowpan_fetch_skb(skb, &hdr.nexthdr, sizeof(hdr.nexthdr)))
>>>> -			goto drop;
>>>> +			return -EINVAL;
>>>>  		pr_debug("NH flag is set, next header carried inline: %02x\n",
>>>>  			 hdr.nexthdr);
>>>> @@ -412,7 +382,7 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  	} else {
>>>>  		if (lowpan_fetch_skb(skb, &hdr.hop_limit,
>>>>  				     sizeof(hdr.hop_limit)))
>>>> -			goto drop;
>>>> +			return -EINVAL;
>>>>  	}
>>>>  	/* Extract SAM to the tmp variable */
>>>> @@ -431,7 +401,7 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  	/* Check on error of previous branch */
>>>>  	if (err)
>>>> -		goto drop;
>>>> +		return -EINVAL;
>>>>  	/* Extract DAM to the tmp variable */
>>>>  	tmp = ((iphc1 & LOWPAN_IPHC_DAM_11) >> LOWPAN_IPHC_DAM_BIT) & 0x03;
>>>> @@ -446,7 +416,7 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  								tmp);
>>>>  			if (err)
>>>> -				goto drop;
>>>> +				return -EINVAL;
>>>>  		}
>>>>  	} else {
>>>>  		err = uncompress_addr(skb, &hdr.daddr, tmp, daddr,
>>>> @@ -454,28 +424,26 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  		pr_debug("dest: stateless compression mode %d dest %pI6c\n",
>>>>  			 tmp, &hdr.daddr);
>>>>  		if (err)
>>>> -			goto drop;
>>>> +			return -EINVAL;
>>>>  	}
>>>>  	/* UDP data uncompression */
>>>>  	if (iphc0 & LOWPAN_IPHC_NH_C) {
>>>>  		struct udphdr uh;
>>>> -		struct sk_buff *new;
>>>>  		if (uncompress_udp_header(skb, &uh))
>>>> -			goto drop;
>>>> +			return -EINVAL;
>>>>  		/* replace the compressed UDP head by the uncompressed UDP
>>>>  		 * header
>>>>  		 */
>>>> -		new = skb_copy_expand(skb, sizeof(struct udphdr),
>>>> -				      skb_tailroom(skb), GFP_ATOMIC);
>>>> -		kfree_skb(skb);
>>>> -
>>>> -		if (!new)
>>>> -			return -ENOMEM;
>>>> +		if (skb_headroom(skb) < sizeof(struct udphdr) + sizeof(hdr)) {
>>>> +			int n = sizeof(struct udphdr) + sizeof(hdr);
>>> For this also a separate for "check if we have enough headroom". This
>>> has nothing to do with the errno fix. All patches should follow "keep it
>>> short and simple" not doing too much in a patch, if you can separate it
>>> then separate it. Then we have a better overview while reviewing and a
>>> nice git commit history.
>>>
>>> Also this should be "if (skb_headroom(skb) < sizeof(struct udphdr))"
>>> only, without the sizeof(hdr). Why you check also for space for ipv6
>>> header here? This is part of transport header.
>> If you are decompressing UDP header then you also have to decompress the IP
>> Header below it.  This would mean that 2 pskb_expand_head calls would be
>> needed which could result in 2 data copies which seems a waste, why not just
>> do it once?
>>>> -		skb = new;
>>>> +			err = pskb_expand_head(skb, n, 0, GFP_ATOMIC);
>>>> +			if (unlikely(err))
>>>> +				return err;
>>>> +		}
>>>>  		skb_push(skb, sizeof(struct udphdr));
>>> Here we add data for udphdr only. Then check only for headroom for
>>> udphdr.
>>>
>>>>  		skb_reset_transport_header(skb);
>>>> @@ -485,6 +453,13 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  			       (u8 *)&uh, sizeof(uh));
>>>>  		hdr.nexthdr = UIP_PROTO_UDP;
>>>> +	} else {
>>> Now I see why you make "sizeof(struct udphdr) + sizeof(hdr)" Better
>>> remove this else branch and then simple make:
>>>
>>> "if (skb_headroom(skb) < sizeof(hdr))", then we don't need to check this
>>> above. This is a little bit confusing.
>> See above, maybe a comment would help things?
>>>> +		if (skb_headroom(skb) < sizeof(hdr)) {
>>>> +			err = pskb_expand_head(skb, sizeof(hdr), 0, GFP_ATOMIC);
>>>> +
>>> remove this whitespace.
>> ok.
>>>> +			if (unlikely(err))
>>>> +				return err;
>>>> +		}
>>>>  	}
>>>>  	hdr.payload_len = htons(skb->len);
>>>> @@ -499,11 +474,14 @@ int lowpan_process_data(struct sk_buff *skb, struct net_device *dev,
>>>>  	raw_dump_table(__func__, "raw header dump", (u8 *)&hdr, sizeof(hdr));
>>>> -	return skb_deliver(skb, &hdr, dev, deliver_skb);
>>>> +	skb_push(skb, sizeof(hdr));
>>> Here is the push for sizeof(hdr) above we should _always_ check for
>>> "(skb_headroom(skb) < sizeof(hdr))".
>> The intention of the code is to ensure that there is enough headroom before
>> this push.  Maybe the following pseudocode helps :)
>>
>> IF UDP Hdr Compression
>>     ensure enough room for UDP Hdr and IPv6 Hdr
>>     make room for UDP Hdr (skb_push)
>>     decompress UDP Hdr
>> ELSE
>>     ensure enough room for IPv6 Hdr
>>
>> make room for IPv6 Hdr (skb_push)
>> decompress IPv6 Hdr
>>
> Yes I know what you doing here, this saves one us one allocation.
>
> I thought about to combine always the length of skb_push with
> pskb_expand_head head so it's easier "why" we do this here. But with
> your solution we save a allocation.
>
> I will try to add a "length" field for necessary space in next header
> compression framework and do this "check and expand" transparent inside
> of next header compression framework. I think this should be fine.
>
> Then please leave this code as it is. I am always fine to remove
> allocations in hotpaths.
>>>
>>> The code is much similar for udphdr and hdr, here.... create a static
>>> function in this file, then we have a generic function for this (this is
>>> useful for other transport header when next header compression layer
>>> comes mainline.
>>>
>>> protoype looks like:
>>>
>>> int skb_check_and_expand(struct sk_buff *skb, size_t len)
>>>
>>> should contain something like:
>>>
>>> if (skb_headroom(skb) < len)
>>> 	do_expand_stuff(skb....);
>>>
>>> return 0;
>>>
>>>
>>>> +	skb_reset_network_header(skb);
>>>> +	skb_copy_to_linear_data(skb, &hdr, sizeof(hdr));
>>>> -drop:
>>>> -	kfree_skb(skb);
>>>> -	return -EINVAL;
>>>> +	raw_dump_table(__func__, "raw skb data dump before receiving",
>>>> +		       skb->data, skb->len);
>>>> +
>>>> +	return 0;
>>>>  }
>>>>  EXPORT_SYMBOL_GPL(lowpan_process_data);
>>>> diff --git a/net/bluetooth/6lowpan.c b/net/bluetooth/6lowpan.c
>>>> index 206b65c..adfd361 100644
>>>> --- a/net/bluetooth/6lowpan.c
>>>> +++ b/net/bluetooth/6lowpan.c
>>>> @@ -230,36 +230,31 @@ static int process_data(struct sk_buff *skb, struct net_device *netdev,
>>>>  	peer = peer_lookup_chan(dev, chan);
>>>>  	read_unlock_irqrestore(&devices_lock, flags);
>>>>  	if (!peer)
>>>> -		goto drop;
>>>> +		return -EINVAL;
>>>>  	saddr = peer->eui64_addr;
>>>>  	daddr = dev->netdev->dev_addr;
>>>>  	/* at least two bytes will be used for the encoding */
>>>>  	if (skb->len < 2)
>>>> -		goto drop;
>>>> +		return -EINVAL;
>>>>  	if (lowpan_fetch_skb_u8(skb, &iphc0))
>>>> -		goto drop;
>>>> +		return -EINVAL;
>>>>  	if (lowpan_fetch_skb_u8(skb, &iphc1))
>>>> -		goto drop;
>>>> +		return -EINVAL;
>>>>  	return lowpan_process_data(skb, netdev,
>>>>  				   saddr, IEEE802154_ADDR_LONG, EUI64_ADDR_LEN,
>>>>  				   daddr, IEEE802154_ADDR_LONG, EUI64_ADDR_LEN,
>>>> -				   iphc0, iphc1, give_skb_to_upper);
>>>> -
>>>> -drop:
>>>> -	kfree_skb(skb);
>>>> -	return -EINVAL;
>>>> +				   iphc0, iphc1);
>>>>  }
>>>>  static int recv_pkt(struct sk_buff *skb, struct net_device *dev,
>>>>  		    struct l2cap_chan *chan)
>>>>  {
>>>>  	struct sk_buff *local_skb;
>>>> -	int ret;
>>>>  	if (!netif_running(dev))
>>>>  		goto drop;
>>>> @@ -280,12 +275,8 @@ static int recv_pkt(struct sk_buff *skb, struct net_device *dev,
>>>>  		local_skb->protocol = htons(ETH_P_IPV6);
>>>>  		local_skb->pkt_type = PACKET_HOST;
>>>> -		skb_reset_network_header(local_skb);
>>>> -		skb_set_transport_header(local_skb, sizeof(struct ipv6hdr));
>>>> -
>>>>  		if (give_skb_to_upper(local_skb, dev) != NET_RX_SUCCESS) {
>>>> -			kfree_skb(local_skb);
>>>> -			goto drop;
>>>> +			goto drop_local_skb;
>>>>  		}
>>>>  		dev->stats.rx_bytes += skb->len;
>>>> @@ -294,15 +285,40 @@ static int recv_pkt(struct sk_buff *skb, struct net_device *dev,
>>>>  		kfree_skb(local_skb);
>>>>  		kfree_skb(skb);
>>>>  	} else {
>>>> +		int ret;
>>>> +
>>>> +		if (skb_cloned(skb)) {
>>> why is this now here? Is this necessary, we don't have such thing
>>> current mainline, if necessary make a separate patch for this or put it
>>> in the right one.
>> It's necessary as pskb_expand_head must have no references as you are
>> potentially reallocating the data part of the skb.   It's position is
>> questionable but until other compression schemes are supported it will
>> suffice.
>>>> +			struct sk_buff *new;
>>>> +			int new_headroom = sizeof(struct ipv6hdr) +
>>>> +					   sizeof(struct udphdr);
>>>> +
>>>> +			new = skb_copy_expand(skb, new_headroom,
>>>> +					      skb_tailroom(skb), GFP_ATOMIC);
>>>> +			if (!new)
>>>> +				return -ENOMEM;
>>>> +			consume_skb(skb);
>>>> +			skb = new;
>>>> +		}
>>>> +
>>>>  		switch (skb->data[0] & 0xe0) {
>>>>  		case LOWPAN_DISPATCH_IPHC:	/* ipv6 datagram */
>>>> +
>>>> +			ret = process_data(skb, dev, chan);
>>>> +			if (ret < 0)
>>>> +				goto drop;
>>>> +
>>>>  			local_skb = skb_clone(skb, GFP_ATOMIC);
>>>>  			if (!local_skb)
>>>>  				goto drop;
>>> This looks a little bit confusing, I will look at this when you sperate
>>> the patches. Again "move handling to upper layer" and the "check if
>>> expand is necessary". Then I will look at this code for correct
>>> "kfree_skb" and "consume_skb" calling. Hope this is okay, I need to
>>> decrypt this code at first. :-)
>> sorry, I didn't intend to writing confusing code, wait until you see the GHC
>> patch :)
>>
> nono, please it's only confusing me because there are too much changes.
> You didn't write confusing code.
>
>>> Sorry too much changes in one patch, we need to split this one. Also
>>> base it on bluetooth-next, should be complicated to megre this stuff for
>>> bluetooth with -next...
>>>
>>> - Alex
>> If I'm going to base it on bluetooth-next then I think the best solution is
>> to submit separate patches in stages
>>
>> 1) Checking for headroom and using pskb_expand_head for decompression:
>> reason for patch: saves on copying the whole structure and there potentially
>> might not be a copy especially if the driver allocates a larger buffer than
>> needed knowing that it will be decompressed
>>
>> 2) Move delivery out of IPHC.
>> reason: Ensures return value isn't a mish mash of error codes and NET_RX_
>> codes, allows for better scalability as we don't have a requirement for
>> decompression to pass the skb onto the next layer
>>
>> 3) Fix lowpan_rcv
>> reason: Fix those pesky return value problems
>>
> ack. You can do that.
>
>> Then I can go on to submit those 2 outstanding patches
>> 4) Refactor lowpan_rcv to handle fragmented uncompressed IPv6 packets.
>>
> I'm very excited about that.
>
>> 5) Rename process_data etc..
>>
> ok.
>
>> I think this will help Jukka as my patch currently breaks their transmit
>> code and we could see which part causes this. Also I'm really busy at the
>> moment writing a QEMU virtual device to model our transceiver so we can get
>> a virtual test bed working as well as a couple of other things so it would
>> be better for me to concentrate on one patch at a time as time is something
>> I don't have a lot of at the moment :)
>>
> So it takes some time now, or what do you mean here?
>
>> Would you be happy with this? I know it means the bug fix happens third, but
>> I think it will be better in the long run.
>>
> I am happy with this and indeed we should fix this issue fast.
>
> - Alex