sk_buff handling in packet handler

sk_buff handling in packet handler

[Mark Smith]

Hi Claude,

You could have a look at my Ethernet Configuration Testing Protocol
(ECTP) implementation. I followed what other protocols did regarding
packet/skb handling, and also tried to make it very clear what was
required, as I struggled a bit with similar issues that you are. The
routine to start with is ectp_rcv().

http://lwn.net/Articles/330797/

Regards,
Mark.

Re: sk_buff handling in packet handler

[Claude Robitaille]

[-- Attachment #1: Type: text/plain, Size: 1242 bytes --]

Thanks Mark.

very interesting

this helped me but just to get to next problem....

what I am trying to do is to receive frame using a packet handler, 
manipulate it and send a portion of it over a UDP channel. I am using 
sock_sendmsg. Things sort of work but I get some warnings while in 
net_tx_action (via the sendmsg I am guessing) about doing scheduling 
while in atomic and sometime the kernel simply completely crashes. All 
this probably means that using kernel socket functions cannot be used 
from any context.... Any suggestion on how to properly do this? 
Ultimately, I'd like to avoid copying the data so if I could simply send 
an skb to some UDP TX handler would be ideal but if not, what is the 
best approach?

Thanks

Claude

Mark Smith wrote:
> Hi Claude,
>
> You could have a look at my Ethernet Configuration Testing Protocol
> (ECTP) implementation. I followed what other protocols did regarding
> packet/skb handling, and also tried to make it very clear what was
> required, as I struggled a bit with similar issues that you are. The
> routine to start with is ectp_rcv().
>
> http://lwn.net/Articles/330797/
>
> Regards,
> Mark.
>
>   

-- 
Claude Robitaille
CTO
Accedian Networks
514-764-0119
clauder@accedian.com


[-- Attachment #2: clauder.vcf --]
[-- Type: text/x-vcard, Size: 176 bytes --]

begin:vcard
fn:Claude Robitaille
n:Robitaille;Claude
email;internet:clauder@accedian.com
tel;work:514-764-0119
tel;cell:514-796-7225
x-mozilla-html:TRUE
version:2.1
end:vcard

Re: sk_buff handling in packet handler

[Stephen Hemminger]

On Sun, 17 May 2009 13:23:49 -0400
Claude Robitaille <clauder@accedian.com> wrote:

> Thanks Mark.
> 
> very interesting
> 
> this helped me but just to get to next problem....
> 
> what I am trying to do is to receive frame using a packet handler, 
> manipulate it and send a portion of it over a UDP channel. I am using 
> sock_sendmsg. Things sort of work but I get some warnings while in 
> net_tx_action (via the sendmsg I am guessing) about doing scheduling 
> while in atomic and sometime the kernel simply completely crashes. All 
> this probably means that using kernel socket functions cannot be used 
> from any context.... Any suggestion on how to properly do this? 
> Ultimately, I'd like to avoid copying the data so if I could simply send 
> an skb to some UDP TX handler would be ideal but if not, what is the 
> best approach?
> 

Write a netfilter module, it is easier to do what you are trying to
do with iptables rules.

> 


-- 

Re: sk_buff handling in packet handler

[Claude Robitaille]

[-- Attachment #1: Type: text/plain, Size: 2215 bytes --]

Thanks Mark, very helpful

But I still  have some questions.

I see that you are using sbk_share_check, which will create a clone if 
the sk_buff is already used by someone else. In most cases the skb is 
not shared and the function simply return the same pointer without any 
modification to the original skb (in particular skb->users stays at 1). 
You  then manipulate the  sk_buff  (in particular  adding a MAC header, 
etc.) and eventually transmit the packet using dev_queue_xmit. The 
sk_buff will be ultimately kfree_skb by the NIC driver (or by you if an 
error occured somewhere before dev_queue_xmit).

My questions are:

- after you are done, net_rx_action may pass the sk_buff, modified by 
ectp_rcv, to other packet handler. How will other packet handlers know 
that, if they need to manipulate the sk_buff, they must create a clone? 
skb->users is still set to 1
- I presume that net_rx_action must also kfree the sk_buff, particularly 
if no packet handler decides to grab the packet. But it should not free 
it if it is still in the transmit queue, waiting for the HW to transmit 
it. But since nothing in the sk_buff tell net_rx_action that the packet 
is pending transmission how could this be the case? dev_queue_xmit does 
not appear to increase users (I used printk in another module to check 
this before and after calling dev_queue_xmit but it might be that my 
idle system is too fast for the printk to catch skb->users going to 2). 
In any case, I try to use sk_get (and 1 x kfree_skb) in my packet 
handler but I end up consuming all the memory in the system, presumably 
due to the fact that the sk_buff never get freed completely. What should 
be the correct way of handling this?

Claude

Mark Smith wrote:
> Hi Claude,
>
> You could have a look at my Ethernet Configuration Testing Protocol
> (ECTP) implementation. I followed what other protocols did regarding
> packet/skb handling, and also tried to make it very clear what was
> required, as I struggled a bit with similar issues that you are. The
> routine to start with is ectp_rcv().
>
> http://lwn.net/Articles/330797/
>
> Regards,
> Mark.
>
>   

-- 
Claude Robitaille
CTO
Accedian Networks
514-764-0119
clauder@accedian.com


[-- Attachment #2: clauder.vcf --]
[-- Type: text/x-vcard, Size: 176 bytes --]

begin:vcard
fn:Claude Robitaille
n:Robitaille;Claude
email;internet:clauder@accedian.com
tel;work:514-764-0119
tel;cell:514-796-7225
x-mozilla-html:TRUE
version:2.1
end:vcard

Re: sk_buff handling in packet handler

[Evgeniy Polyakov]

On Sat, May 16, 2009 at 12:31:28PM -0400, Claude Robitaille (clauder@accedian.com) wrote:
> I suspect that the handler needs to do something with the sk_buff but I 
> am not sure what. I printed the content of the sk_buff and found that 
> the skb_buff is not cloned (looking on the web I had found that the 
> sk_buff should have been cloned) and that he number of users is 1.
> 
> Should I simply do a dev_kfree_skb? What are the rules governing the use 
> of the sk_buff in a packet handler? Should I clone the sk_buff?
> 
> Also, a related question, the packet handler is expected to return an 
> int. What should be returned?

Use packet socket code as the best documentation, in particular
packet_rcv, but 0 is ok, although in theory it should match NET_RX_*
constants.

-- 
	Evgeniy Polyakov

sk_buff handling in packet handler

[Claude Robitaille]

[-- Attachment #1: Type: text/plain, Size: 1235 bytes --]

Hi all,

My question is a development question so I hope this is the correct list 
to post it. If not just tell me and I will find another list...

I am developing a packet handler that I register via dev_add_pack. All 
is fine except that after a while the kernel crashes. I suspect my 
handler not doing something correct...

The handler is registered with the ETH_P_ALL list, listening on a 
specific interface.

After many attempt at reducing the scope of what my handler is doing I 
now simply do a return in my handler. When I send lots of frames to 
interface that I am listening to after a while I start getting kernel 
messages about low memory and eventually even get exception dumps.

I suspect that the handler needs to do something with the sk_buff but I 
am not sure what. I printed the content of the sk_buff and found that 
the skb_buff is not cloned (looking on the web I had found that the 
sk_buff should have been cloned) and that he number of users is 1.

Should I simply do a dev_kfree_skb? What are the rules governing the use 
of the sk_buff in a packet handler? Should I clone the sk_buff?

Also, a related question, the packet handler is expected to return an 
int. What should be returned?

Thanks

Claude
 

[-- Attachment #2: clauder.vcf --]
[-- Type: text/x-vcard, Size: 176 bytes --]

begin:vcard
fn:Claude Robitaille
n:Robitaille;Claude
email;internet:clauder@accedian.com
tel;work:514-764-0119
tel;cell:514-796-7225
x-mozilla-html:TRUE
version:2.1
end:vcard