question on incoming packets and scheduler

question on incoming packets and scheduler

[Chris Friesen]

I have a long-running cpu hog background task, and a high-priority 
critical task that waits on a socket for network traffic.  When a packet 
comes in, I'd like the cpu hog to be swapped out ASAP, rather than 
waiting for the end of the timeslice.  Is there any way to make this happen?

The code paths that I managed to trace didn't seem to be calling the 
scheduler to force the context switch.  Hopefully I missed something.

Thanks,

Chris

-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: question on incoming packets and scheduler

[Chris Friesen]

Friesen, Christopher [CAR:7Q28:EXCH] wrote:
> 
> I have a long-running cpu hog background task, and a high-priority 
> critical task that waits on a socket for network traffic.  When a packet 
> comes in, I'd like the cpu hog to be swapped out ASAP, rather than 
> waiting for the end of the timeslice.  Is there any way to make this 
> happen?
> 
> The code paths that I managed to trace didn't seem to be calling the 
> scheduler to force the context switch.  Hopefully I missed something.

I should add that I'm on 2.4.22.

Chris



-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: question on incoming packets and scheduler

[Davide Libenzi]

On Tue, 14 Oct 2003, Chris Friesen wrote:

>
> I have a long-running cpu hog background task, and a high-priority
> critical task that waits on a socket for network traffic.  When a packet
> comes in, I'd like the cpu hog to be swapped out ASAP, rather than
> waiting for the end of the timeslice.  Is there any way to make this happen?
>
> The code paths that I managed to trace didn't seem to be calling the
> scheduler to force the context switch.  Hopefully I missed something.

What do you mean for high priority? Is it an RT task? The wakeup (AKA
inserion in the run queue) happen soon :
IRQ->do_IRQ->softirq->net_rx_action->ip_rcv->...
but if your task is not RT there no guarantee that it'll preempt the
current running.



- Davide

Re: question on incoming packets and scheduler

[Chris Friesen]

Davide Libenzi wrote:
> On Tue, 14 Oct 2003, Chris Friesen wrote:

>>I have a long-running cpu hog background task, and a high-priority
>>critical task that waits on a socket for network traffic.  When a packet
>>comes in, I'd like the cpu hog to be swapped out ASAP, rather than
>>waiting for the end of the timeslice.  Is there any way to make this happen?


> What do you mean for high priority? Is it an RT task? The wakeup (AKA
> inserion in the run queue) happen soon :
> IRQ->do_IRQ->softirq->net_rx_action->ip_rcv->...
> but if your task is not RT there no guarantee that it'll preempt the
> current running.

Yes, it was an RT task.

It appears that 2.4.20 fixes this issue, but there is another one 
remaining that the latency appears to be dependent on the number of 
incoming packets.  See thread "incoming packet latency in 2.4.[18-20]" 
for details.  This behaviour doesn't show up in 2.6, and I'm about to 
test 2.4.22.

Chris




-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: question on incoming packets and scheduler

[Davide Libenzi]

On Wed, 15 Oct 2003, Chris Friesen wrote:

> Davide Libenzi wrote:
> > On Tue, 14 Oct 2003, Chris Friesen wrote:
>
> >>I have a long-running cpu hog background task, and a high-priority
> >>critical task that waits on a socket for network traffic.  When a packet
> >>comes in, I'd like the cpu hog to be swapped out ASAP, rather than
> >>waiting for the end of the timeslice.  Is there any way to make this happen?
>
>
> > What do you mean for high priority? Is it an RT task? The wakeup (AKA
> > inserion in the run queue) happen soon :
> > IRQ->do_IRQ->softirq->net_rx_action->ip_rcv->...
> > but if your task is not RT there no guarantee that it'll preempt the
> > current running.
>
> Yes, it was an RT task.
>
> It appears that 2.4.20 fixes this issue, but there is another one
> remaining that the latency appears to be dependent on the number of
> incoming packets.  See thread "incoming packet latency in 2.4.[18-20]"
> for details.  This behaviour doesn't show up in 2.6, and I'm about to
> test 2.4.22.

Are you sure it's not a livelock issue during the burst?


- Davide

Re: question on incoming packets and scheduler

[Chris Friesen]

Davide Libenzi wrote:
> On Wed, 15 Oct 2003, Chris Friesen wrote:

>>It appears that 2.4.20 fixes this issue, but there is another one
>>remaining that the latency appears to be dependent on the number of
>>incoming packets.  See thread "incoming packet latency in 2.4.[18-20]"
>>for details.  This behaviour doesn't show up in 2.6, and I'm about to
>>test 2.4.22.

> Are you sure it's not a livelock issue during the burst?

I dunno, you tell me.

The test app simply sits in select() until a packet comes in, then it 
spins on recvmsg() until there are no more messages.  It uses 
SO_TIMESTAMP to find out when the packet got to the kernel, and does a 
gettimeofday() right after the recvmsg(), then calculates the delta for 
each packet and the overall average.

With 2.4.[18-20], the overall average goes up when the number of packets 
goes up.  For 2.6.0-test6, it stays constant.

Chris




-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com

Re: question on incoming packets and scheduler

[Andrew R. Reiter]

On Wed, 15 Oct 2003, Chris Friesen wrote:

:Davide Libenzi wrote:
:> On Wed, 15 Oct 2003, Chris Friesen wrote:
:
:>>It appears that 2.4.20 fixes this issue, but there is another one
:>>remaining that the latency appears to be dependent on the number of
:>>incoming packets.  See thread "incoming packet latency in 2.4.[18-20]"
:>>for details.  This behaviour doesn't show up in 2.6, and I'm about to
:>>test 2.4.22.
:
:> Are you sure it's not a livelock issue during the burst?
:
:I dunno, you tell me.
:
:The test app simply sits in select() until a packet comes in, then it
:spins on recvmsg() until there are no more messages.  It uses
:SO_TIMESTAMP to find out when the packet got to the kernel, and does a
:gettimeofday() right after the recvmsg(), then calculates the delta for
:each packet and the overall average.
:
:With 2.4.[18-20], the overall average goes up when the number of packets
:goes up.  For 2.6.0-test6, it stays constant.

I would be interested in seeing the test be run with epoll too :)  I
ealize .18 doesn't support this, but how about it for .20?  If not,
perhaps you could shoot over the code and I could test?

Cheers,
Andrew

--
Andrew R. Reiter
arr@watson.org
arr@FreeBSD.org

Re: question on incoming packets and scheduler

[Chris Friesen]

Friesen, Christopher [CAR:7Q28:EXCH] wrote:


> With 2.4.[18-20], the overall average goes up when the number of packets 
> goes up.  For 2.6.0-test6, it stays constant.

Okay, its official.  My brain isn't working.

The different kernels were tested on two different machines.  The one 
running 2.4 is *just* enough slower that the packets start to pile up 
and end up waiting in the socket buffer.  By slowing down the packet 
sending I get results that are invariant with the number of packets.

Sorry for the false alarm guys....

Chris




-- 
Chris Friesen                    | MailStop: 043/33/F10
Nortel Networks                  | work: (613) 765-0557
3500 Carling Avenue              | fax:  (613) 765-2986
Nepean, ON K2H 8E9 Canada        | email: cfriesen@nortelnetworks.com