* kmemleak report on isp1763 and sierra MC8705
@ 2012-10-26 21:57 Richard Retanubun
2012-10-26 23:35 ` Greg KH
0 siblings, 1 reply; 11+ messages in thread
From: Richard Retanubun @ 2012-10-26 21:57 UTC (permalink / raw)
To: linux-kernel
Cc: catalin.marinas, Lennart Sorensen, Tang Nguyen, m.grzeschik,
Arvid Brodin, linux-usb mailing list, bigeasy
Hi Guys,
I am debugging a reported kmemleak involving a sierra wireless MC8705 connected
through isp1763 on powerpc linux-3.0.22
We are still isolating the exact trigger, but this is a pretty good one so far
send "at!reset" to the modem control tty, wait until it finishes rebooting
then try to bring up a PPP link that will fail (non existent ISP).
After some time, we got the report (included at the end) from kmemleak.
There seems to be two variants of trace that is prevalent:
something like this:
unreferenced object 0xd58e58c8 (size 8):
comm "khubd", pid 1034, jiffies 74467293 (age 2380.122s)
hex dump (first 8 bytes):
4d 43 38 37 30 35 00 00 MC8705..
backtrace:
[<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
[<e30e77bc>] usb_enumerate_device+0x44/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
and something like this:
unreferenced object 0xd5893e00 (size 512):
comm "khubd", pid 1034, jiffies 74467270 (age 2378.786s)
hex dump (first 32 bytes):
09 02 a8 00 06 01 01 e0 00 00 00 00 d5 87 d6 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<e30f1740>] usb_get_configuration+0x5c/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
Some questions:
1. Have you guys seen anything like this before?
2. The report does not point to sierra or isp1763, so our current understanding
is that the memory is allocated outside these drivers and it is supposed
to mark it done for someone to free it. We think this way because if
we rigged a driver to leak a memory it allocates, kmemleak will trace
right into it. Is this understanding correct?
3. Any ideas on how to deepen the probe to get more understanding of what happens?
4. Michael, is this similar to the problem you reported here?
http://marc.info/?l=linux-usb&m=133432571801643&w=4
From reading your report (serial device hanging), It doesn't look like it...
5. Our current hypothesis is this:
we open the /dev/ttyUSB to send "at!reset", then a race begins
between closing the file handle and freeing the driver resources
and the modem hardware actually resetting, which then caused the leak.
Can this be it? and if so, any ideas on how to solve it?
To test this we are power cycling the modem using a gpio
(without opening /dev/ttyUSB) to see if this is the culprit.
6. There is a worrisome line in our (old version) of isp1763 inherited from isp1760:
isp1760_endpoint_disable()
...
qh_destroy(qh);
ep->hcpriv = NULL;
/* remove requests and leak them.
* ATL are pretty fast done, INT could take a while...
* The latter shoule be removed
*/
What is leaking here? qh_destroy release the memory already.
Thanks for everyone's time!
-- Richard Retanubun
--------------------------------------------------------------------------------
unreferenced object 0xd5922c00 (size 1024):
comm "khubd", pid 1034, jiffies 74467113 (age 2378.943s)
hex dump (first 32 bytes):
ff ff ff ff 31 2e 32 00 00 00 00 00 00 00 00 00 ....1.2.........
00 00 00 00 00 00 00 02 00 00 00 00 00 00 00 03 ................
backtrace:
[<e30e4718>] usb_alloc_dev+0x48/0x290 [usbcore]
[<e30e91ec>] hub_thread+0x654/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd58e52b0 (size 8):
comm "khubd", pid 1034, jiffies 74467113 (age 2378.943s)
hex dump (first 8 bytes):
32 2d 31 2e 32 00 04 00 2-1.2...
backtrace:
[<c018a9ec>] kvasprintf+0x58/0x88
[<c0180910>] kobject_set_name_vargs+0x34/0x84
[<c01b3d20>] dev_set_name+0x50/0x60
[<e30e4860>] usb_alloc_dev+0x190/0x290 [usbcore]
[<e30e91ec>] hub_thread+0x654/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5893e00 (size 512):
comm "khubd", pid 1034, jiffies 74467270 (age 2378.786s)
hex dump (first 32 bytes):
09 02 a8 00 06 01 01 e0 00 00 00 00 d5 87 d6 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<e30f1740>] usb_get_configuration+0x5c/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd58e5930 (size 8):
comm "khubd", pid 1034, jiffies 74467270 (age 2378.786s)
hex dump (first 8 bytes):
d5 8a dc c0 00 00 00 00 ........
backtrace:
[<e30f1760>] usb_get_configuration+0x7c/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd58adcc0 (size 192):
comm "khubd", pid 1034, jiffies 74467271 (age 2378.786s)
hex dump (first 32 bytes):
09 02 a8 00 06 01 01 e0 00 09 04 00 00 02 ff ff ................
ff 00 07 05 81 02 00 02 20 07 05 01 02 00 02 20 ........ ......
backtrace:
[<e30f1804>] usb_get_configuration+0x120/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd59555c0 (size 64):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.766s)
hex dump (first 32 bytes):
00 00 00 01 00 00 00 01 09 04 00 00 02 ff ff ff ................
00 00 00 00 d5 92 7a e0 00 00 00 00 d5 8a dc d2 ......z.........
backtrace:
[<e30f1cb0>] usb_get_configuration+0x5cc/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5955400 (size 64):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.766s)
hex dump (first 32 bytes):
00 00 00 01 00 00 00 01 09 04 01 00 02 ff ff ff ................
00 00 00 00 d5 92 7a 20 00 00 00 00 d5 8a dc e9 ......z ........
backtrace:
[<e30f1cb0>] usb_get_configuration+0x5cc/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5955280 (size 64):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.779s)
hex dump (first 32 bytes):
00 00 00 01 00 00 00 01 09 04 02 00 02 ff ff ff ................
00 00 00 00 d5 92 70 00 00 00 00 00 d5 8a dd 00 ......p.........
backtrace:
[<e30f1cb0>] usb_get_configuration+0x5cc/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd59554c0 (size 64):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.779s)
hex dump (first 32 bytes):
00 00 00 01 00 00 00 01 09 04 03 00 03 ff ff ff ................
00 00 00 00 d5 8a d5 40 00 00 00 00 d5 8a dd 17 .......@........
backtrace:
[<e30f1cb0>] usb_get_configuration+0x5cc/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5955580 (size 64):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.779s)
hex dump (first 32 bytes):
00 00 00 01 00 00 00 01 09 04 04 00 03 ff ff ff ................
00 00 00 00 d5 8a d2 40 00 00 00 00 d5 8a dd 35 .......@.......5
backtrace:
[<e30f1cb0>] usb_get_configuration+0x5cc/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5955300 (size 64):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.779s)
hex dump (first 32 bytes):
00 00 00 01 00 00 00 01 09 04 07 00 03 ff ff ff ................
00 00 00 00 d5 8a d3 00 00 00 00 00 d5 8a dd 53 ...............S
backtrace:
[<e30f1cb0>] usb_get_configuration+0x5cc/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5927ae0 (size 96):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.779s)
hex dump (first 32 bytes):
07 05 81 02 00 02 20 00 00 00 00 00 00 00 00 00 ...... .........
d5 92 7a f0 d5 92 7a f0 00 00 00 00 00 00 00 00 ..z...z.........
backtrace:
[<e30f2514>] usb_get_configuration+0xe30/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5927a20 (size 96):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.779s)
hex dump (first 32 bytes):
07 05 82 02 00 02 20 00 00 00 00 00 00 00 00 00 ...... .........
d5 92 7a 30 d5 92 7a 30 00 00 00 00 00 00 00 00 ..z0..z0........
backtrace:
[<e30f2514>] usb_get_configuration+0xe30/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5927000 (size 96):
comm "khubd", pid 1034, jiffies 74467291 (age 2378.780s)
hex dump (first 32 bytes):
07 05 83 02 00 02 20 00 00 00 00 00 00 00 00 00 ...... .........
d5 92 70 10 d5 92 70 10 00 00 00 00 00 00 00 00 ..p...p.........
backtrace:
[<e30f2514>] usb_get_configuration+0xe30/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd58ad540 (size 192):
comm "khubd", pid 1034, jiffies 74467291 (age 2380.124s)
hex dump (first 32 bytes):
07 05 84 03 40 00 05 00 00 00 00 00 00 00 00 00 ....@...........
d5 8a d5 50 d5 8a d5 50 00 00 00 00 00 00 00 00 ...P...P........
backtrace:
[<e30f2514>] usb_get_configuration+0xe30/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd58ad240 (size 192):
comm "khubd", pid 1034, jiffies 74467291 (age 2380.124s)
hex dump (first 32 bytes):
07 05 86 03 40 00 05 00 00 00 00 00 00 00 00 00 ....@...........
d5 8a d2 50 d5 8a d2 50 00 00 00 00 00 00 00 00 ...P...P........
backtrace:
[<e30f2514>] usb_get_configuration+0xe30/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd58ad300 (size 192):
comm "khubd", pid 1034, jiffies 74467291 (age 2380.124s)
hex dump (first 32 bytes):
07 05 88 03 40 00 05 00 00 00 00 00 00 00 00 00 ....@...........
d5 8a d3 10 d5 8a d3 10 00 00 00 00 00 00 00 00 ................
backtrace:
[<e30f2514>] usb_get_configuration+0xe30/0x13a8 [usbcore]
[<e30e7850>] usb_enumerate_device+0xd8/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd58e58c8 (size 8):
comm "khubd", pid 1034, jiffies 74467293 (age 2380.122s)
hex dump (first 8 bytes):
4d 43 38 37 30 35 00 00 MC8705..
backtrace:
[<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
[<e30e77bc>] usb_enumerate_device+0x44/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd587d3c0 (size 32):
comm "khubd", pid 1034, jiffies 74467293 (age 2380.122s)
hex dump (first 32 bytes):
53 69 65 72 72 61 20 57 69 72 65 6c 65 73 73 2c Sierra Wireless,
20 49 6e 63 6f 72 70 6f 72 61 74 65 64 00 64 00 Incorporated.d.
backtrace:
[<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
[<e30e77cc>] usb_enumerate_device+0x54/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd670a1a0 (size 16):
comm "khubd", pid 1034, jiffies 74467294 (age 2380.122s)
hex dump (first 16 bytes):
33 35 33 35 36 37 30 34 30 31 31 31 37 39 32 00 353567040111792.
backtrace:
[<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
[<e30e77dc>] usb_enumerate_device+0x64/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd5927480 (size 96):
comm "khubd", pid 1034, jiffies 74467294 (age 2380.122s)
hex dump (first 32 bytes):
d5 92 74 80 d5 92 74 80 c0 1b 35 4c c0 1b 36 a8 ..t...t...5L..6.
00 00 00 00 00 10 01 00 00 20 02 00 00 00 00 00 ......... ......
backtrace:
[<c01b46c4>] device_private_init+0x34/0x8c
[<c01b4f28>] device_add+0x27c/0x6a8
[<e30e7b00>] usb_new_device+0x9c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
unreferenced object 0xd587d600 (size 32):
comm "khubd", pid 1034, jiffies 74467300 (age 2380.792s)
hex dump (first 32 bytes):
53 69 65 72 72 61 20 43 6f 6e 66 69 67 75 72 61 Sierra Configura
74 69 6f 6e 00 2f 52 00 df 82 09 60 df 40 64 00 tion./R....`.@d.
backtrace:
[<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
[<e30f0264>] usb_set_configuration+0x4b8/0x60c [usbcore]
[<e30f8850>] generic_probe+0x48/0xb8 [usbcore]
[<e30f0b00>] usb_probe_device+0x38/0x70 [usbcore]
[<c01b79e8>] driver_probe_device+0xc0/0x2a8
[<c01b6be4>] bus_for_each_drv+0x70/0xac
[<c01b7df4>] device_attach+0xb4/0xd8
[<c01b6340>] bus_probe_device+0x2c/0x44
[<c01b51b8>] device_add+0x50c/0x6a8
[<e30e7b00>] usb_new_device+0x9c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-10-26 21:57 kmemleak report on isp1763 and sierra MC8705 Richard Retanubun
@ 2012-10-26 23:35 ` Greg KH
2012-10-29 20:47 ` Richard Retanubun
0 siblings, 1 reply; 11+ messages in thread
From: Greg KH @ 2012-10-26 23:35 UTC (permalink / raw)
To: Richard Retanubun
Cc: linux-kernel, catalin.marinas, Lennart Sorensen, Tang Nguyen,
m.grzeschik, Arvid Brodin, linux-usb mailing list, bigeasy
On Fri, Oct 26, 2012 at 05:57:23PM -0400, Richard Retanubun wrote:
> Hi Guys,
>
> I am debugging a reported kmemleak involving a sierra wireless MC8705 connected
> through isp1763 on powerpc linux-3.0.22
Does this also happen on 3.6.3?
thanks,
greg k-h
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-10-26 23:35 ` Greg KH
@ 2012-10-29 20:47 ` Richard Retanubun
2012-10-29 21:11 ` Greg KH
2012-10-29 22:14 ` Alan Stern
0 siblings, 2 replies; 11+ messages in thread
From: Richard Retanubun @ 2012-10-29 20:47 UTC (permalink / raw)
To: Greg KH
Cc: linux-kernel, catalin.marinas, Lennart Sorensen, Tang Nguyen,
m.grzeschik, Arvid Brodin, linux-usb mailing list, bigeasy
On 26/10/12 07:35 PM, Greg KH wrote:
> On Fri, Oct 26, 2012 at 05:57:23PM -0400, Richard Retanubun wrote:
>> Hi Guys,
>>
>> I am debugging a reported kmemleak involving a sierra wireless MC8705 connected
>> through isp1763 on powerpc linux-3.0.22
>
> Does this also happen on 3.6.3?
>
> thanks,
>
> greg k-h
Hi Greg,
Unfortunately, it is not trivial for us to update the kernel on this platform,
Is there a specific experiment/patch I should look at for 3.0.22?
I will be attempting to use 3.6.3 on another platform, but this may take some time.
I am thinking there may be an action that can be done at /sysfs or /procfs to
do the disconnect without actually removing the power to the device.
I tried "echo 1 > /sys/bus/usb/drivers/usb/2-1.2/remove" and then take down the power
but this produced the same leak signature even before I take down the power.
Update on trigger to problem
============================
This will happen as the modem is powered down and /dev/ttyUSB from sierra is teared down
either by powering it off/removing it, or sending at!reset.
It does not happen when the same thing is done using a simple usb to serial converter (pl2303)
Focusing down on one of the dumps:
unreferenced object 0xd3849740 (size 8):
comm "khubd", pid 1026, jiffies 232553037 (age 506.597s)
hex dump (first 8 bytes):
4d 43 38 37 30 35 00 00 MC8705..
backtrace:
[<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
[<e30e77bc>] usb_enumerate_device+0x44/0xf8 [usbcore]
[<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
[<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
[<c0043aa8>] kthread+0x7c/0x80
[<c000ed48>] kernel_thread+0x4c/0x68
I have a small question. How does the memory kmalloc-ed() in usb_cache_string is supposed to be released?
(during usb_serial_disconnect()?) Is the sierra driver is supposed to participate
in the tear down process (in sierra_release() maybe) and not doing something that is expected?
I am still missing the link between the actions done by the hub_thread() for the caching the stings
and the sierra driver code.
Thanks a lot for your time.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-10-29 20:47 ` Richard Retanubun
@ 2012-10-29 21:11 ` Greg KH
2012-10-29 22:14 ` Alan Stern
1 sibling, 0 replies; 11+ messages in thread
From: Greg KH @ 2012-10-29 21:11 UTC (permalink / raw)
To: Richard Retanubun
Cc: linux-kernel, catalin.marinas, Lennart Sorensen, Tang Nguyen,
m.grzeschik, Arvid Brodin, linux-usb mailing list, bigeasy
On Mon, Oct 29, 2012 at 04:47:04PM -0400, Richard Retanubun wrote:
> On 26/10/12 07:35 PM, Greg KH wrote:
> >On Fri, Oct 26, 2012 at 05:57:23PM -0400, Richard Retanubun wrote:
> >>Hi Guys,
> >>
> >>I am debugging a reported kmemleak involving a sierra wireless MC8705 connected
> >>through isp1763 on powerpc linux-3.0.22
> >
> >Does this also happen on 3.6.3?
> >
> >thanks,
> >
> >greg k-h
>
> Hi Greg,
>
> Unfortunately, it is not trivial for us to update the kernel on this platform,
> Is there a specific experiment/patch I should look at for 3.0.22?
I have no idea, given that 3.0.22 is based on 18 month old kernel, tens
of thousands of patches ago.
You are on your own here, sorry.
Best of luck,
greg k-h
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-10-29 20:47 ` Richard Retanubun
2012-10-29 21:11 ` Greg KH
@ 2012-10-29 22:14 ` Alan Stern
2012-11-09 22:14 ` Richard Retanubun
1 sibling, 1 reply; 11+ messages in thread
From: Alan Stern @ 2012-10-29 22:14 UTC (permalink / raw)
To: Richard Retanubun
Cc: Greg KH, linux-kernel, catalin.marinas, Lennart Sorensen,
Tang Nguyen, m.grzeschik, Arvid Brodin, linux-usb mailing list,
bigeasy
On Mon, 29 Oct 2012, Richard Retanubun wrote:
> do the disconnect without actually removing the power to the device.
> I tried "echo 1 > /sys/bus/usb/drivers/usb/2-1.2/remove" and then take down the power
> but this produced the same leak signature even before I take down the power.
>
> Update on trigger to problem
> ============================
> This will happen as the modem is powered down and /dev/ttyUSB from sierra is teared down
> either by powering it off/removing it, or sending at!reset.
>
> It does not happen when the same thing is done using a simple usb to serial converter (pl2303)
>
> Focusing down on one of the dumps:
>
> unreferenced object 0xd3849740 (size 8):
> comm "khubd", pid 1026, jiffies 232553037 (age 506.597s)
> hex dump (first 8 bytes):
> 4d 43 38 37 30 35 00 00 MC8705..
> backtrace:
> [<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
> [<e30e77bc>] usb_enumerate_device+0x44/0xf8 [usbcore]
> [<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
> [<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
> [<c0043aa8>] kthread+0x7c/0x80
> [<c000ed48>] kernel_thread+0x4c/0x68
>
> I have a small question. How does the memory kmalloc-ed() in usb_cache_string is supposed to be released?
> (during usb_serial_disconnect()?)
It doesn't get released during usb_serial_disconnect(). It gets
released during usb_release_dev() in drivers/usb/core/usb.c.
> Is the sierra driver is supposed to participate
> in the tear down process (in sierra_release() maybe) and not doing something that is expected?
Probably not.
> I am still missing the link between the actions done by the hub_thread() for the caching the stings
> and the sierra driver code.
They aren't all that closely related.
usb_release_dev() won't be called until all references to the USB
device have been dropped. Maybe there's an extra reference hanging
around.
Alan Stern
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-10-29 22:14 ` Alan Stern
@ 2012-11-09 22:14 ` Richard Retanubun
2012-11-10 14:30 ` Johan Hovold
0 siblings, 1 reply; 11+ messages in thread
From: Richard Retanubun @ 2012-11-09 22:14 UTC (permalink / raw)
To: linux-kernel
Cc: Alan Stern, Greg KH, Lennart Sorensen, Tang Nguyen,
linux-usb mailing list
On 29/10/12 06:14 PM, Alan Stern wrote:
> On Mon, 29 Oct 2012, Richard Retanubun wrote:
>> Focusing down on one of the dumps:
>>
>> unreferenced object 0xd3849740 (size 8):
>> comm "khubd", pid 1026, jiffies 232553037 (age 506.597s)
>> hex dump (first 8 bytes):
>> 4d 43 38 37 30 35 00 00 MC8705..
>> backtrace:
>> [<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
>> [<e30e77bc>] usb_enumerate_device+0x44/0xf8 [usbcore]
>> [<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
>> [<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
>> [<c0043aa8>] kthread+0x7c/0x80
>> [<c000ed48>] kernel_thread+0x4c/0x68
>>
>> I have a small question. How does the memory kmalloc-ed() in usb_cache_string is supposed to be released?
>> (during usb_serial_disconnect()?)
>
> It doesn't get released during usb_serial_disconnect(). It gets
> released during usb_release_dev() in drivers/usb/core/usb.c.
>
>> Is the sierra driver is supposed to participate
>> in the tear down process (in sierra_release() maybe) and not doing something that is expected?
>
> Probably not.
>
>> I am still missing the link between the actions done by the hub_thread() for the caching the stings
>> and the sierra driver code.
>
> They aren't all that closely related.
>
> usb_release_dev() won't be called until all references to the USB
> device have been dropped. Maybe there's an extra reference hanging
> around.
>
> Alan Stern
>
Thanks a lot for the hint Alan.
I added a dev_dbg print in usb_release_dev() and saw that in the builds where there is a leak, this was simply never called!
the last line printed in a trace with all dev_dbg on is this "usb_disable_device nuking all URBs"
When the sierra modem is unplugged, the cleanup sequence never calls usb_release_dev() (on PL2303 it always calls usb_release_dev()
This is the current state of versions from linux-stable
3.0.y (3.0.51) - Have the issue
3.2.y (3.2.33) - Have the issue
3.4.y (3.4.18) - Have the issue
3.5.y (3.5.7) - Does not have the issue (but leaks because the portdata patches is not backported yet)
3.6.y (3.6.6) - Does not have the issue
So a diff between 3.4.y and 3.5.y ought to narrow it down further.
I am posting just in case someone recalls a particular patch I should be trying out first...
-- RR --
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-11-09 22:14 ` Richard Retanubun
@ 2012-11-10 14:30 ` Johan Hovold
2012-11-14 17:12 ` Richard Retanubun
0 siblings, 1 reply; 11+ messages in thread
From: Johan Hovold @ 2012-11-10 14:30 UTC (permalink / raw)
To: Richard Retanubun, Greg Kroah-Hartman
Cc: linux-kernel, Alan Stern, Lennart Sorensen, Tang Nguyen,
linux-usb mailing list, Johan Hovold
On Fri, Nov 9, 2012 at 11:14 PM, Richard Retanubun
<richardretanubun@ruggedcom.com> wrote:
> On 29/10/12 06:14 PM, Alan Stern wrote:
>>
>> On Mon, 29 Oct 2012, Richard Retanubun wrote:
>>>
>>> Focusing down on one of the dumps:
>>>
>>> unreferenced object 0xd3849740 (size 8):
>>> comm "khubd", pid 1026, jiffies 232553037 (age 506.597s)
>>> hex dump (first 8 bytes):
>>> 4d 43 38 37 30 35 00 00 MC8705..
>>> backtrace:
>>> [<e30efd74>] usb_cache_string+0x74/0xac [usbcore]
>>> [<e30e77bc>] usb_enumerate_device+0x44/0xf8 [usbcore]
>>> [<e30e7aa0>] usb_new_device+0x3c/0x13c [usbcore]
>>> [<e30e9824>] hub_thread+0xc8c/0x1544 [usbcore]
>>> [<c0043aa8>] kthread+0x7c/0x80
>>> [<c000ed48>] kernel_thread+0x4c/0x68
>>>
>>> I have a small question. How does the memory kmalloc-ed() in
>>> usb_cache_string is supposed to be released?
>>> (during usb_serial_disconnect()?)
>>
>>
>> It doesn't get released during usb_serial_disconnect(). It gets
>> released during usb_release_dev() in drivers/usb/core/usb.c.
>>
>>> Is the sierra driver is supposed to participate
>>> in the tear down process (in sierra_release() maybe) and not doing
>>> something that is expected?
>>
>>
>> Probably not.
>>
>>> I am still missing the link between the actions done by the hub_thread()
>>> for the caching the stings
>>> and the sierra driver code.
>>
>>
>> They aren't all that closely related.
>>
>> usb_release_dev() won't be called until all references to the USB
>> device have been dropped. Maybe there's an extra reference hanging
>> around.
>>
>> Alan Stern
>>
> Thanks a lot for the hint Alan.
>
> I added a dev_dbg print in usb_release_dev() and saw that in the builds
> where there is a leak, this was simply never called!
> the last line printed in a trace with all dev_dbg on is this
> "usb_disable_device nuking all URBs"
> When the sierra modem is unplugged, the cleanup sequence never calls
> usb_release_dev() (on PL2303 it always calls usb_release_dev()
>
> This is the current state of versions from linux-stable
>
> 3.0.y (3.0.51) - Have the issue
> 3.2.y (3.2.33) - Have the issue
> 3.4.y (3.4.18) - Have the issue
> 3.5.y (3.5.7) - Does not have the issue (but leaks because the portdata
> patches is not backported yet)
> 3.6.y (3.6.6) - Does not have the issue
>
> So a diff between 3.4.y and 3.5.y ought to narrow it down further.
>
> I am posting just in case someone recalls a particular patch I should be
> trying out first...
There was a reference-count fix for the probe error path that went in to
v3.5. Haven't read all the details on how you trigger your leak, but at
the face of it, it could be related.
Have a look at 0658a3366db7e27fa ("usb: use usb_serial_put in
usb_serial_probe errors). If related, you should be seeing "Ignoring
blacklisted interface #n" messages when you enable debug (e.g. #define
DEBUG) in the sierra driver.
Greg, it seems to me that the fix referred to above should be backported
to the earlier stable trees either way.
Thanks,
Johan
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-11-10 14:30 ` Johan Hovold
@ 2012-11-14 17:12 ` Richard Retanubun
2012-11-14 17:52 ` Johan Hovold
0 siblings, 1 reply; 11+ messages in thread
From: Richard Retanubun @ 2012-11-14 17:12 UTC (permalink / raw)
To: Johan Hovold
Cc: Greg Kroah-Hartman, linux-kernel, Alan Stern, Lennart Sorensen,
Tang Nguyen, linux-usb mailing list
On 10/11/12 09:30 AM, Johan Hovold wrote:
Hi Johan,
> There was a reference-count fix for the probe error path that went in to
> v3.5. Haven't read all the details on how you trigger your leak, but at
> the face of it, it could be related.
>
> Have a look at 0658a3366db7e27fa ("usb: use usb_serial_put in
> usb_serial_probe errors). If related, you should be seeing "Ignoring
> blacklisted interface #n" messages when you enable debug (e.g. #define
> DEBUG) in the sierra driver.
That was it! Thanks so much for the research.
I can apply it cleanly to 3.0.22 and see usb_release_dev() being called and thus no more kmemleak.
>
> Greg, it seems to me that the fix referred to above should be backported
> to the earlier stable trees either way.
I would vote "yes" for this also.
While my setup circumstances may be a corner case, (modem kept resetting to re-establish PPP connection)
it was leaking 1192 bytes per occurrence.
Thanks for everyone's time.
-- Richard Retanubun.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-11-14 17:12 ` Richard Retanubun
@ 2012-11-14 17:52 ` Johan Hovold
2012-11-21 1:15 ` Greg Kroah-Hartman
0 siblings, 1 reply; 11+ messages in thread
From: Johan Hovold @ 2012-11-14 17:52 UTC (permalink / raw)
To: Richard Retanubun
Cc: Johan Hovold, Greg Kroah-Hartman, linux-kernel, Alan Stern,
Lennart Sorensen, Tang Nguyen, linux-usb mailing list, stable,
Ben Hutchings
On Wed, Nov 14, 2012 at 12:12:01PM -0500, Richard Retanubun wrote:
> On 10/11/12 09:30 AM, Johan Hovold wrote:
> Hi Johan,
>
> > There was a reference-count fix for the probe error path that went in to
> > v3.5. Haven't read all the details on how you trigger your leak, but at
> > the face of it, it could be related.
> >
> > Have a look at 0658a3366db7e27fa ("usb: use usb_serial_put in
> > usb_serial_probe errors). If related, you should be seeing "Ignoring
> > blacklisted interface #n" messages when you enable debug (e.g. #define
> > DEBUG) in the sierra driver.
>
> That was it! Thanks so much for the research.
> I can apply it cleanly to 3.0.22 and see usb_release_dev() being
> called and thus no more kmemleak.
>
> >
> > Greg, it seems to me that the fix referred to above should be backported
> > to the earlier stable trees either way.
> I would vote "yes" for this also.
>
> While my setup circumstances may be a corner case, (modem kept
> resetting to re-establish PPP connection) it was leaking 1192 bytes
> per occurrence.
The leak affects every failed probe, for example due to blacklisted
interfaces which is quite common, so commit 0658a3366db7 ("usb: use
usb_serial_put in usb_serial_probe errors) should be backported to the
<= 3.4 stable trees.
Thanks for reporting,
Johan
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-11-14 17:52 ` Johan Hovold
@ 2012-11-21 1:15 ` Greg Kroah-Hartman
2012-11-25 14:24 ` Ben Hutchings
0 siblings, 1 reply; 11+ messages in thread
From: Greg Kroah-Hartman @ 2012-11-21 1:15 UTC (permalink / raw)
To: Johan Hovold
Cc: Richard Retanubun, linux-kernel, Alan Stern, Lennart Sorensen,
Tang Nguyen, linux-usb mailing list, stable, Ben Hutchings
On Wed, Nov 14, 2012 at 06:52:18PM +0100, Johan Hovold wrote:
> On Wed, Nov 14, 2012 at 12:12:01PM -0500, Richard Retanubun wrote:
> > On 10/11/12 09:30 AM, Johan Hovold wrote:
> > Hi Johan,
> >
> > > There was a reference-count fix for the probe error path that went in to
> > > v3.5. Haven't read all the details on how you trigger your leak, but at
> > > the face of it, it could be related.
> > >
> > > Have a look at 0658a3366db7e27fa ("usb: use usb_serial_put in
> > > usb_serial_probe errors). If related, you should be seeing "Ignoring
> > > blacklisted interface #n" messages when you enable debug (e.g. #define
> > > DEBUG) in the sierra driver.
> >
> > That was it! Thanks so much for the research.
> > I can apply it cleanly to 3.0.22 and see usb_release_dev() being
> > called and thus no more kmemleak.
> >
> > >
> > > Greg, it seems to me that the fix referred to above should be backported
> > > to the earlier stable trees either way.
> > I would vote "yes" for this also.
> >
> > While my setup circumstances may be a corner case, (modem kept
> > resetting to re-establish PPP connection) it was leaking 1192 bytes
> > per occurrence.
>
> The leak affects every failed probe, for example due to blacklisted
> interfaces which is quite common, so commit 0658a3366db7 ("usb: use
> usb_serial_put in usb_serial_probe errors) should be backported to the
> <= 3.4 stable trees.
Thanks, now applied.
greg k-h
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: kmemleak report on isp1763 and sierra MC8705
2012-11-21 1:15 ` Greg Kroah-Hartman
@ 2012-11-25 14:24 ` Ben Hutchings
0 siblings, 0 replies; 11+ messages in thread
From: Ben Hutchings @ 2012-11-25 14:24 UTC (permalink / raw)
To: Richard Retanubun, Johan Hovold
Cc: linux-kernel, Alan Stern, Lennart Sorensen, Tang Nguyen,
linux-usb mailing list, stable, Greg Kroah-Hartman
[-- Attachment #1: Type: text/plain, Size: 1714 bytes --]
On Tue, 2012-11-20 at 17:15 -0800, Greg Kroah-Hartman wrote:
> On Wed, Nov 14, 2012 at 06:52:18PM +0100, Johan Hovold wrote:
> > On Wed, Nov 14, 2012 at 12:12:01PM -0500, Richard Retanubun wrote:
> > > On 10/11/12 09:30 AM, Johan Hovold wrote:
> > > Hi Johan,
> > >
> > > > There was a reference-count fix for the probe error path that went in to
> > > > v3.5. Haven't read all the details on how you trigger your leak, but at
> > > > the face of it, it could be related.
> > > >
> > > > Have a look at 0658a3366db7e27fa ("usb: use usb_serial_put in
> > > > usb_serial_probe errors). If related, you should be seeing "Ignoring
> > > > blacklisted interface #n" messages when you enable debug (e.g. #define
> > > > DEBUG) in the sierra driver.
> > >
> > > That was it! Thanks so much for the research.
> > > I can apply it cleanly to 3.0.22 and see usb_release_dev() being
> > > called and thus no more kmemleak.
> > >
> > > >
> > > > Greg, it seems to me that the fix referred to above should be backported
> > > > to the earlier stable trees either way.
> > > I would vote "yes" for this also.
> > >
> > > While my setup circumstances may be a corner case, (modem kept
> > > resetting to re-establish PPP connection) it was leaking 1192 bytes
> > > per occurrence.
> >
> > The leak affects every failed probe, for example due to blacklisted
> > interfaces which is quite common, so commit 0658a3366db7 ("usb: use
> > usb_serial_put in usb_serial_probe errors) should be backported to the
> > <= 3.4 stable trees.
>
> Thanks, now applied.
Also queued up for 3.2.
Ben.
--
Ben Hutchings
Never attribute to conspiracy what can adequately be explained by stupidity.
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 828 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2012-11-25 14:25 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-10-26 21:57 kmemleak report on isp1763 and sierra MC8705 Richard Retanubun
2012-10-26 23:35 ` Greg KH
2012-10-29 20:47 ` Richard Retanubun
2012-10-29 21:11 ` Greg KH
2012-10-29 22:14 ` Alan Stern
2012-11-09 22:14 ` Richard Retanubun
2012-11-10 14:30 ` Johan Hovold
2012-11-14 17:12 ` Richard Retanubun
2012-11-14 17:52 ` Johan Hovold
2012-11-21 1:15 ` Greg Kroah-Hartman
2012-11-25 14:24 ` Ben Hutchings
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).