rtl8187: kernel oops when leds enabled

rtl8187: kernel oops when leds enabled

[Richard Farina]

Using kernel 2.6.31 and compat-wireless stable 2.6.32_rc5 I get this 
kernel oops nearly 100% of the time when I unplug the device.  I have 
tried with the device down, up, and while transmitting, here are the 
three oopses.  Sorry it took me so long for this report, it is an odd 
bug.  When I was running older kernel/compat-wireless this oops was 
about 4-5 screens longs so I couldn't take a picture.  Additionally I 
took the advice of a few of the list members to try the crashkernel 
feature (which is AWESOME  btw) but it doesn't work for this oops.  Let 
me be more specific, I can trigger an oops and the crash kernel kicks 
in, but with this oops the crash kernel never kicks in so I am guessing 
this is bad news.

I'm honestly a little surprised that no one has bothered to reproduce 
this in the month since I reported it but maybe it is more unique to my 
setup than it seems.

Hope these pictures are readable enough to be useful.

http://picpaste.com/pics/cimg1470.1256404028.jpg
http://picpaste.com/pics/cimg1474.1256404126.jpg
http://picpaste.com/pics/cimg1477.1256404170.jpg

I can fairly well generate this error at will and I'm available to test 
reproducing the bug in any possible situation which is suggested and/or 
test patches to fix.

Thanks,
Rick

Re: rtl8187: kernel oops when leds enabled

[Larry Finger]

On 10/26/2009 10:17 AM, Richard Farina wrote:

> I'm honestly a little surprised that no one has bothered to reproduce
> this in the month since I reported it but maybe it is more unique to my
> setup than it seems.

Your machine seems to be unique in that it happens most of the time.
It took more than 10 tries, but I finally got my copy of the
wireless-testing kernel to oops.

I will be working on a patch.

Larry

Re: rtl8187: kernel oops when leds enabled

[Larry Finger]

Richard Farina wrote:
> Larry Finger wrote:
>> On 10/26/2009 10:17 AM, Richard Farina wrote:
>>
>>  
>>> I'm honestly a little surprised that no one has bothered to reproduce
>>> this in the month since I reported it but maybe it is more unique to my
>>> setup than it seems.
>>>     
>>
>> Your machine seems to be unique in that it happens most of the time.
>> It took more than 10 tries, but I finally got my copy of the
>> wireless-testing kernel to oops.
>>
>> I will be working on a patch.
>>
>>   
> Excellent! Thanks for looking into it.  I'll be as responsive as
> possible in testing any patches.

I finally got a full dump on my own machine. It is a kernel BUG from
"scheduling while atomic". That makes me think that the bug is not in
rtl8187, but that unplugging the RTL8187 module triggers a bug in the
kernel.

I'm currently building the latest git pull of Linus's tree to see if
the problem is still there. If so, I will post on LKML and make a
kernel bugzilla entry.

Larry

Possible fix for rtl8187: kernel oops when leds enabled

[Larry Finger]

Rick,

Does this patch fix your problem? I tried to bisect this problem as it
looked like a regression; however, the problem was sometimes a little
flakey. On one boot of a given kernel, it might run for 3-400 cycles
without failing, then fail immediately after rebooting. I suspect that
the random contents of some memory location would control that.

This patch was inspired by the code in p54usb, which does not have the
problem.


Index: wireless-testing/drivers/net/wireless/rtl818x/rtl8187_leds.c
===================================================================
--- wireless-testing.orig/drivers/net/wireless/rtl818x/rtl8187_leds.c
+++ wireless-testing/drivers/net/wireless/rtl818x/rtl8187_leds.c
@@ -210,10 +210,10 @@ void rtl8187_leds_exit(struct ieee80211_

 	/* turn the LED off before exiting */
 	ieee80211_queue_delayed_work(dev, &priv->led_off, 0);
-	cancel_delayed_work_sync(&priv->led_off);
-	cancel_delayed_work_sync(&priv->led_on);
 	rtl8187_unregister_led(&priv->led_rx);
 	rtl8187_unregister_led(&priv->led_tx);
+	cancel_delayed_work_sync(&priv->led_off);
+	cancel_delayed_work_sync(&priv->led_on);
 }
 #endif /* def CONFIG_RTL8187_LED */

Thanks for testing,

Larry

Re: Possible fix for rtl8187: kernel oops when leds enabled

[Richard Farina]

Larry Finger wrote:
> Rick,
>
> Does this patch fix your problem? I tried to bisect this problem as it
> looked like a regression; however, the problem was sometimes a little
> flakey. On one boot of a given kernel, it might run for 3-400 cycles
> without failing, then fail immediately after rebooting. I suspect that
> the random contents of some memory location would control that.
>
>   

Larry,
As you mentioned, this bug was not 100% reliable to reproduce in the 
first place for either of us, I merely had a much much higher chance of 
panick for some unknown reason.  That said, I have plugged and unplugged 
the device a few dozen times while bringing the interface up and down 
and making the led blink and I'm thrilled to say it no longer kernel 
panicks on my system.  I'm left with a usable wifi card with a cool 
flashing led, many thanks for fixing this.

John,

imho this patch should go upstream asap, next dot release if at all 
possible.  This fixes a significant issue in that the kernel panicked on 
my system nearly 100% of the time on unplug if the led was enabled.  
Either way it is up to the maintainers but consider this my vote of 
confidence.

Acked-By: Rick Farina
Tested-By: Rick Farina

Thanks,
Rick Farina

> This patch was inspired by the code in p54usb, which does not have the
> problem.
>
>
> Index: wireless-testing/drivers/net/wireless/rtl818x/rtl8187_leds.c
> ===================================================================
> --- wireless-testing.orig/drivers/net/wireless/rtl818x/rtl8187_leds.c
> +++ wireless-testing/drivers/net/wireless/rtl818x/rtl8187_leds.c
> @@ -210,10 +210,10 @@ void rtl8187_leds_exit(struct ieee80211_
>
>  	/* turn the LED off before exiting */
>  	ieee80211_queue_delayed_work(dev, &priv->led_off, 0);
> -	cancel_delayed_work_sync(&priv->led_off);
> -	cancel_delayed_work_sync(&priv->led_on);
>  	rtl8187_unregister_led(&priv->led_rx);
>  	rtl8187_unregister_led(&priv->led_tx);
> +	cancel_delayed_work_sync(&priv->led_off);
> +	cancel_delayed_work_sync(&priv->led_on);
>  }
>  #endif /* def CONFIG_RTL8187_LED */
>
> Thanks for testing,
>
> Larry
>
>
>
>
>
>
>
>   

Re: rtl8187: kernel oops when leds enabled

[Hin-Tak Leung]

On Mon, Oct 26, 2009 at 3:17 PM, Richard Farina <sidhayn@gmail.com> wrote:

> I'm honestly a little surprised that no one has bothered to reproduce this
> in the month since I reported it but maybe it is more unique to my setup
> than it seems.

Yes, quite possibly. Mine is built-in and hooked up directly to the
laptop's motherboard (and cannot be unplugged!) so I am no use. Larry
is on it, I think. We appreciate the persistence...

Hin-Tak

Re: rtl8187: kernel oops when leds enabled

[Larry Finger]

Richard Farina wrote:
> Using kernel 2.6.31 and compat-wireless stable 2.6.32_rc5 I get this
> kernel oops nearly 100% of the time when I unplug the device.  I have
> tried with the device down, up, and while transmitting, here are the
> three oopses.  Sorry it took me so long for this report, it is an odd
> bug.  When I was running older kernel/compat-wireless this oops was
> about 4-5 screens longs so I couldn't take a picture.  Additionally I
> took the advice of a few of the list members to try the crashkernel
> feature (which is AWESOME  btw) but it doesn't work for this oops.  Let
> me be more specific, I can trigger an oops and the crash kernel kicks
> in, but with this oops the crash kernel never kicks in so I am guessing
> this is bad news.

I have a bit more information about this crash. It shows up as either
a "BUG: Scheduling while atomic" or "Kernel panic - not synching:
Fatal exception in interrupt". I could also trigger this oops by
running a rmmod/insmod loop, which makes testing easier in that I can
walk away and let the machine do the testing.

As you noted, the problem does not appear when the LED code is not
enabled. I tried to find a problem in the rtl8187 LED code without
success, then discovered that the problem is present in mainline
2.6.32-rc5, but not in 2.6.31. One does not need compat-wireless. Note
that the rtl8187 LED code did not change in that time. It seems likely
that there is a bug in some other part of the system that rtl8187 is
triggering. In any case, I now have a starting point for bisection,
which is my next step.

This problem is clearly a regression between 2.6.31 and 2.6.32-rc5. I
will file a Bugzilla on it once I know the commit that broke the system.

Larry