* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds [not found] ` <29bad771-843c-1dee-906c-6e9475aed7d8@gmail.com> @ 2019-03-12 15:07 ` Gregor Jasny 2019-03-15 22:34 ` Sean Young 0 siblings, 1 reply; 19+ messages in thread From: Gregor Jasny @ 2019-03-12 15:07 UTC (permalink / raw) To: Linux Media Mailing List; +Cc: CHEMLA Samuel, Mauro Carvalho Chehab Hello Mauro, below you find a bug report about an use-after-free in dvbv5-zap. On 12.03.19 13:37, CHEMLA Samuel wrote: > please find a bug report that seems to concern ARMHF builds of dvbv5-zap > (dvb-tool package) : https://bugs.launchpad.net/raspbian/+bug/1819650 > I filed it against raspbian because I thought it was a raspbian problem, > but don't think they re-build their own package, but use debian ones > instead... Thanks, Gregor ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-12 15:07 ` [Bug report] dvbv5-zap crash dvb-tool ARMHF builds Gregor Jasny @ 2019-03-15 22:34 ` Sean Young 2019-03-17 9:52 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 19+ messages in thread From: Sean Young @ 2019-03-15 22:34 UTC (permalink / raw) To: Gregor Jasny Cc: Linux Media Mailing List, CHEMLA Samuel, Mauro Carvalho Chehab Hi, On Tue, Mar 12, 2019 at 04:07:23PM +0100, Gregor Jasny wrote: > Hello Mauro, > > below you find a bug report about an use-after-free in dvbv5-zap. > > On 12.03.19 13:37, CHEMLA Samuel wrote: > > please find a bug report that seems to concern ARMHF builds of dvbv5-zap > > (dvb-tool package) : https://bugs.launchpad.net/raspbian/+bug/1819650 > > I filed it against raspbian because I thought it was a raspbian problem, > > but don't think they re-build their own package, but use debian ones > > instead... > So I can reproduce the issue with v4l-utils 1.12.3 but not with current v4l-utils (or dvbv5-zap). It looks exactly like the issue fixed in commit 6e21f6f34c1d7c3a7a059062e1ddd9705c984e2c (but I did not cherry-pick and test that on top of 1.12.3 to test that theory). Sean ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-15 22:34 ` Sean Young @ 2019-03-17 9:52 ` Mauro Carvalho Chehab 2019-03-19 19:45 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 19+ messages in thread From: Mauro Carvalho Chehab @ 2019-03-17 9:52 UTC (permalink / raw) To: Sean Young, CHEMLA Samuel Cc: Gregor Jasny, Linux Media Mailing List, Mauro Carvalho Chehab Em Fri, 15 Mar 2019 22:34:25 +0000 Sean Young <sean@mess.org> escreveu: > Hi, > > On Tue, Mar 12, 2019 at 04:07:23PM +0100, Gregor Jasny wrote: > > Hello Mauro, > > > > below you find a bug report about an use-after-free in dvbv5-zap. > > > > On 12.03.19 13:37, CHEMLA Samuel wrote: > > > please find a bug report that seems to concern ARMHF builds of dvbv5-zap > > > (dvb-tool package) : https://bugs.launchpad.net/raspbian/+bug/1819650 > > > I filed it against raspbian because I thought it was a raspbian problem, > > > but don't think they re-build their own package, but use debian ones > > > instead... > > > > So I can reproduce the issue with v4l-utils 1.12.3 but not with current > v4l-utils (or dvbv5-zap). It looks exactly like the issue fixed in > commit 6e21f6f34c1d7c3a7a059062e1ddd9705c984e2c (but I did not cherry-pick > and test that on top of 1.12.3 to test that theory). I added it to stable/1.12 and another patch fixing the initialization of the parameters struct, with Samuel reported to fix the issue upstream (He pinged me in priv too, and I'm helping him to track it). Samuel, Could you please check if the 1.12 stable branch is OK now? Regards, Mauro Thanks, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-17 9:52 ` Mauro Carvalho Chehab @ 2019-03-19 19:45 ` Mauro Carvalho Chehab 2019-03-20 19:38 ` Gregor Jasny 0 siblings, 1 reply; 19+ messages in thread From: Mauro Carvalho Chehab @ 2019-03-19 19:45 UTC (permalink / raw) To: Gregor Jasny Cc: Sean Young, CHEMLA Samuel, Linux Media Mailing List, Mauro Carvalho Chehab Hi Gregor, Samuel reported in priv that the issues he had with user after free were solved by the patchsets merged at 1.12 and 1.16 stable branches. Could you please generate a new staging release for them? Thanks! Mauro Em Sun, 17 Mar 2019 06:52:42 -0300 Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu: > Em Fri, 15 Mar 2019 22:34:25 +0000 > Sean Young <sean@mess.org> escreveu: > > > Hi, > > > > On Tue, Mar 12, 2019 at 04:07:23PM +0100, Gregor Jasny wrote: > > > Hello Mauro, > > > > > > below you find a bug report about an use-after-free in dvbv5-zap. > > > > > > On 12.03.19 13:37, CHEMLA Samuel wrote: > > > > please find a bug report that seems to concern ARMHF builds of dvbv5-zap > > > > (dvb-tool package) : https://bugs.launchpad.net/raspbian/+bug/1819650 > > > > I filed it against raspbian because I thought it was a raspbian problem, > > > > but don't think they re-build their own package, but use debian ones > > > > instead... > > > > > > > So I can reproduce the issue with v4l-utils 1.12.3 but not with current > > v4l-utils (or dvbv5-zap). It looks exactly like the issue fixed in > > commit 6e21f6f34c1d7c3a7a059062e1ddd9705c984e2c (but I did not cherry-pick > > and test that on top of 1.12.3 to test that theory). > > I added it to stable/1.12 and another patch fixing the initialization of > the parameters struct, with Samuel reported to fix the issue upstream > (He pinged me in priv too, and I'm helping him to track it). > > Samuel, > > Could you please check if the 1.12 stable branch is OK now? > > Regards, > Mauro > Thanks, > Mauro Thanks, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-19 19:45 ` Mauro Carvalho Chehab @ 2019-03-20 19:38 ` Gregor Jasny 2019-03-21 9:41 ` Sean Young 0 siblings, 1 reply; 19+ messages in thread From: Gregor Jasny @ 2019-03-20 19:38 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Sean Young, CHEMLA Samuel, Linux Media Mailing List, Mauro Carvalho Chehab Hello Mauro, On 19.03.19 20:45, Mauro Carvalho Chehab wrote: > Hi Gregor, > > Samuel reported in priv that the issues he had with user after free were > solved by the patchsets merged at 1.12 and 1.16 stable branches. > > Could you please generate a new staging release for them? Sure, I can create a new 1.12 and 1.16 stable release. But when reviewing the patches for approval by debian release managers I noticed an additional double-free that Sean addressed with the following patch: > https://git.linuxtv.org/v4l-utils.git/commit/?id=ebd890019ba7383b8b486d829f6683c8f49fdbda Could you please give that patch a thorough review, some testing, and cherry-pick it to stable-1.12 and -1.16 as well? Thanks, Gregor ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-20 19:38 ` Gregor Jasny @ 2019-03-21 9:41 ` Sean Young 2019-03-21 11:30 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 19+ messages in thread From: Sean Young @ 2019-03-21 9:41 UTC (permalink / raw) To: Gregor Jasny Cc: Mauro Carvalho Chehab, CHEMLA Samuel, Linux Media Mailing List, Mauro Carvalho Chehab On Wed, Mar 20, 2019 at 08:38:52PM +0100, Gregor Jasny wrote: > Hello Mauro, > > On 19.03.19 20:45, Mauro Carvalho Chehab wrote: > > Hi Gregor, > > > > Samuel reported in priv that the issues he had with user after free were > > solved by the patchsets merged at 1.12 and 1.16 stable branches. > > > > Could you please generate a new staging release for them? > > Sure, I can create a new 1.12 and 1.16 stable release. But when reviewing > the patches for approval by debian release managers I noticed an additional > double-free that Sean addressed with the following patch: > > > https://git.linuxtv.org/v4l-utils.git/commit/?id=ebd890019ba7383b8b486d829f6683c8f49fdbda > > Could you please give that patch a thorough review, some testing, and > cherry-pick it to stable-1.12 and -1.16 as well? I did test it myself (and also under valgrind). The bad paths are hard to hit though. I'd say just go ahead with merging and releasing, the patch isn't that controversial (I hope!). Sean ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-21 9:41 ` Sean Young @ 2019-03-21 11:30 ` Mauro Carvalho Chehab 2019-03-21 19:59 ` Gregor Jasny 0 siblings, 1 reply; 19+ messages in thread From: Mauro Carvalho Chehab @ 2019-03-21 11:30 UTC (permalink / raw) To: Sean Young Cc: Gregor Jasny, CHEMLA Samuel, Linux Media Mailing List, Mauro Carvalho Chehab Em Thu, 21 Mar 2019 09:41:28 +0000 Sean Young <sean@mess.org> escreveu: > On Wed, Mar 20, 2019 at 08:38:52PM +0100, Gregor Jasny wrote: > > Hello Mauro, > > > > On 19.03.19 20:45, Mauro Carvalho Chehab wrote: > > > Hi Gregor, > > > > > > Samuel reported in priv that the issues he had with user after free were > > > solved by the patchsets merged at 1.12 and 1.16 stable branches. > > > > > > Could you please generate a new staging release for them? > > > > Sure, I can create a new 1.12 and 1.16 stable release. But when reviewing > > the patches for approval by debian release managers I noticed an additional > > double-free that Sean addressed with the following patch: > > > > > https://git.linuxtv.org/v4l-utils.git/commit/?id=ebd890019ba7383b8b486d829f6683c8f49fdbda > > > > Could you please give that patch a thorough review, some testing, and > > cherry-pick it to stable-1.12 and -1.16 as well? > > I did test it myself (and also under valgrind). The bad paths are hard > to hit though. I'd say just go ahead with merging and releasing, the patch > isn't that controversial (I hope!). I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, we can release a new minor version for all those stable branches. After the patches, on my tests, I didn't get any memory leaks or double-free issues. It should be noticed that I had to add a new patch at -1.14, due to the usage of minor() and major() macros, as one of the files there were not including sys/sysmacros.h. Without that, I was getting compilation errors. Funny enough, this header was already included on two other places within -1.14. As the major() and minor() macros were added at glibc 2.3.3[1], released in 2004 [2], it should be save to include sys/sysmacros.h unconditionally at stable-1.14. [1] according with "man 3 makedev", as pointed by: https://stackoverflow.com/questions/22240973/major-and-minor-macros-defined-in-sys-sysmacros-h-pulled-in-by-iterator [2] https://ftp.gnu.org/gnu/libc/'s glibc tarball is from 2004-08-03. Thanks, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-21 11:30 ` Mauro Carvalho Chehab @ 2019-03-21 19:59 ` Gregor Jasny [not found] ` <CANJnhGfRtEwAony5Z4rFMPcu58aF2k0G+9NSkMKsq_PhfmSNqw@mail.gmail.com> 0 siblings, 1 reply; 19+ messages in thread From: Gregor Jasny @ 2019-03-21 19:59 UTC (permalink / raw) To: Mauro Carvalho Chehab, Sean Young Cc: CHEMLA Samuel, Linux Media Mailing List, Mauro Carvalho Chehab Hello, On 21.03.19 12:30, Mauro Carvalho Chehab wrote: > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, we can > release a new minor version for all those stable branches. > > After the patches, on my tests, I didn't get any memory leaks or > double-free issues. I issues a new 1.12, 1.14, and 1.16 release. Thanks, Gregor ^ permalink raw reply [flat|nested] 19+ messages in thread
[parent not found: <CANJnhGfRtEwAony5Z4rFMPcu58aF2k0G+9NSkMKsq_PhfmSNqw@mail.gmail.com>]
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds [not found] ` <CANJnhGfRtEwAony5Z4rFMPcu58aF2k0G+9NSkMKsq_PhfmSNqw@mail.gmail.com> @ 2019-03-25 17:08 ` Mauro Carvalho Chehab 2019-03-26 7:35 ` Samuel CHEMLA 0 siblings, 1 reply; 19+ messages in thread From: Mauro Carvalho Chehab @ 2019-03-25 17:08 UTC (permalink / raw) To: Samuel CHEMLA Cc: Gregor Jasny, Sean Young, Linux Media Mailing List, Mauro Carvalho Chehab Em Mon, 25 Mar 2019 17:33:30 +0100 Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > Hi guys, > > I'm afraid I'm coming with sad news. > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 > ). > I previously test both branches on a raspberry zero and issues were gone > (same raspbian version). > There may be more memory issues somewhere... Could you test it with valgrind? > > Sam > > Le jeu. 21 mars 2019 à 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : > > > Hello, > > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote: > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and > > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, > > we can > > > release a new minor version for all those stable branches. > > > > > > After the patches, on my tests, I didn't get any memory leaks or > > > double-free issues. > > > > I issues a new 1.12, 1.14, and 1.16 release. > > > > Thanks, > > Gregor > > > > Thanks, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-25 17:08 ` Mauro Carvalho Chehab @ 2019-03-26 7:35 ` Samuel CHEMLA 2019-03-26 13:26 ` Sean Young 0 siblings, 1 reply; 19+ messages in thread From: Samuel CHEMLA @ 2019-03-26 7:35 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Gregor Jasny, Sean Young, Linux Media Mailing List, Mauro Carvalho Chehab Hi, I am struggling with valgrind because it always complain with either : ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find any core dump or, if I link statically libasan with -static-libasan: Shadow memory range interleaves with an existing memory mapping. ASan cannot proceed correctly. ABORTING. ASan shadow was supposed to be located in the [0x00007fff7000-0x10007fff7fff] range. I retested again on my raspberry zero W, and I confirm i cannot reproduce the hang. Your fix did work on that device. I am testing with same OS (raspbian with latest updates, same kernel), same configure options, same USB dongle... :-( The only differences are CPU architecture (armv6 vs armv7), memory constraints, and I was not using the same channels.conf, I'll fix that today and re-check Sam On 25/03/2019 18:08, Mauro Carvalho Chehab wrote: Em Mon, 25 Mar 2019 17:33:30 +0100 Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: Hi guys, I'm afraid I'm coming with sad news. I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 ). I previously test both branches on a raspberry zero and issues were gone (same raspbian version). There may be more memory issues somewhere... Could you test it with valgrind? Sam Le jeu. 21 mars 2019 à 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : Hello, On 21.03.19 12:30, Mauro Carvalho Chehab wrote: I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, we can release a new minor version for all those stable branches. After the patches, on my tests, I didn't get any memory leaks or double-free issues. I issues a new 1.12, 1.14, and 1.16 release. Thanks, Gregor Thanks, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-26 7:35 ` Samuel CHEMLA @ 2019-03-26 13:26 ` Sean Young 2019-03-26 15:10 ` Samuel CHEMLA 0 siblings, 1 reply; 19+ messages in thread From: Sean Young @ 2019-03-26 13:26 UTC (permalink / raw) To: Samuel CHEMLA Cc: Mauro Carvalho Chehab, Gregor Jasny, Linux Media Mailing List, Mauro Carvalho Chehab Hi Sam, On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote: > Hi, > > > I am struggling with valgrind because it always complain with either : > ASan runtime does not come first in initial library list; you > should either link runtime to your application or manually preload it > with LD_PRELOAD > -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find > any core dump > > or, if I link statically libasan with -static-libasan: > Shadow memory range interleaves with an existing memory mapping. > ASan cannot proceed correctly. ABORTING. > ASan shadow was supposed to be located in the > [0x00007fff7000-0x10007fff7fff] range. > > > I retested again on my raspberry zero W, and I confirm i cannot > reproduce the hang. > Your fix did work on that device. > I am testing with same OS (raspbian with latest updates, same kernel), > same configure options, same USB dongle... :-( > The only differences are CPU architecture (armv6 vs armv7), memory > constraints, and I was not using the same channels.conf, I'll fix that > today and re-check Earlier you said "random hangs are back". When this happens, does the whole device become unresponsive or just dvbv5-zap? Since this issue is "back", I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. It would be useful to see the output from dmesg (best thing would be after the issue occurs). Also what dvb hardware are you using? Thanks, san > > > Sam > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote: > > Em Mon, 25 Mar 2019 17:33:30 +0100 > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > Hi guys, > > I'm afraid I'm coming with sad news. > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 > ). > I previously test both branches on a raspberry zero and issues were gone > (same raspbian version). > There may be more memory issues somewhere... > > Could you test it with valgrind? > > Sam > > Le jeu. 21 mars 2019 à 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : > > Hello, > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote: > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, > > we can > > release a new minor version for all those stable branches. > > After the patches, on my tests, I didn't get any memory leaks or > double-free issues. > > I issues a new 1.12, 1.14, and 1.16 release. > > Thanks, > Gregor > > > > Thanks, > Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-26 13:26 ` Sean Young @ 2019-03-26 15:10 ` Samuel CHEMLA 2019-03-26 15:31 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 19+ messages in thread From: Samuel CHEMLA @ 2019-03-26 15:10 UTC (permalink / raw) To: Sean Young Cc: Mauro Carvalho Chehab, Gregor Jasny, Linux Media Mailing List, Mauro Carvalho Chehab Hi, > Earlier you said "random hangs are back". When this happens, does the whole > device become unresponsive or just dvbv5-zap? The device completely freeze, you can't even switch numlock on/off. I said "the issue is back", it is because I **thought** it was gone. To be more precise: - on raspberry zero W, the issue is gone since Mauro's patch (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89) - on raspberry 2, the issue, it is still there and the patch has no effect (the issue was and is still there) > Since this issue is "back", > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. The issue doesn't appear anymore on raspberry zero W since Mauro's commit. So it did improve on that platform. > It would be useful to see the output from dmesg (best thing would be after > the issue occurs). You can't, device is frozen. Logs are not flushed to disk, and journalctl -f freeze before showing anything > Also what dvb hardware are you using? I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM and a TerraTec Cinergy T Stick+ You can found all the details here: https://bugs.launchpad.net/raspbian/+bug/1819650 Sam Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit : > > Hi Sam, > > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote: > > Hi, > > > > > > I am struggling with valgrind because it always complain with either : > > ASan runtime does not come first in initial library list; you > > should either link runtime to your application or manually preload it > > with LD_PRELOAD > > -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find > > any core dump > > > > or, if I link statically libasan with -static-libasan: > > Shadow memory range interleaves with an existing memory mapping. > > ASan cannot proceed correctly. ABORTING. > > ASan shadow was supposed to be located in the > > [0x00007fff7000-0x10007fff7fff] range. > > > > > > I retested again on my raspberry zero W, and I confirm i cannot > > reproduce the hang. > > Your fix did work on that device. > > I am testing with same OS (raspbian with latest updates, same kernel), > > same configure options, same USB dongle... :-( > > The only differences are CPU architecture (armv6 vs armv7), memory > > constraints, and I was not using the same channels.conf, I'll fix that > > today and re-check > > Earlier you said "random hangs are back". When this happens, does the whole > device become unresponsive or just dvbv5-zap? Since this issue is "back", > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > > It would be useful to see the output from dmesg (best thing would be after > the issue occurs). > > Also what dvb hardware are you using? > > Thanks, > > san > > > > > > > Sam > > > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote: > > > > Em Mon, 25 Mar 2019 17:33:30 +0100 > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > > Hi guys, > > > > I'm afraid I'm coming with sad news. > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 > > ). > > I previously test both branches on a raspberry zero and issues were gone > > (same raspbian version). > > There may be more memory issues somewhere... > > > > Could you test it with valgrind? > > > > Sam > > > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : > > > > Hello, > > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote: > > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, > > > > we can > > > > release a new minor version for all those stable branches. > > > > After the patches, on my tests, I didn't get any memory leaks or > > double-free issues. > > > > I issues a new 1.12, 1.14, and 1.16 release. > > > > Thanks, > > Gregor > > > > > > > > Thanks, > > Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-26 15:10 ` Samuel CHEMLA @ 2019-03-26 15:31 ` Mauro Carvalho Chehab 2019-03-28 18:48 ` Samuel CHEMLA 0 siblings, 1 reply; 19+ messages in thread From: Mauro Carvalho Chehab @ 2019-03-26 15:31 UTC (permalink / raw) To: Samuel CHEMLA Cc: Sean Young, Gregor Jasny, Linux Media Mailing List, Mauro Carvalho Chehab Em Tue, 26 Mar 2019 16:10:33 +0100 Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > Hi, > > > > Earlier you said "random hangs are back". When this happens, does the whole > > device become unresponsive or just dvbv5-zap? > The device completely freeze, you can't even switch numlock on/off. dvbv5-tools can't hang the machine. this is very likely happening due to a Kernel crash. > I said "the issue is back", it is because I **thought** it was gone. > To be more precise: > - on raspberry zero W, the issue is gone since Mauro's patch > (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89) > - on raspberry 2, the issue, it is still there and the patch has no > effect (the issue was and is still there) RPi2 has a serious issue with their USB ports: on devices that require more than a few mW to work, it causes several device disconnection and re-connection, as it cannot sustain the required 5V. Depending on how fast this happens, it could be triggering some Kernel bug. That affects most V4L and DVB devices. You should either use a powered USB 2.0 hub (with may be problematic, as the USB host driver on RPi has issues - and may cause data loss on high sustained ISOC traffic, specially when used with hubs) or a device that has its own power supply, like DVBSky T680C or S960. > > Since this issue is "back", > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > The issue doesn't appear anymore on raspberry zero W since Mauro's commit. > So it did improve on that platform. > > > It would be useful to see the output from dmesg (best thing would be after > > the issue occurs). > You can't, device is frozen. > Logs are not flushed to disk, and journalctl -f freeze before showing anything You can use a serial port in order to get the logs. On a serial console, use something like: # dmesg -n 8 In order to make sure it will display all Kernel messages at console. > > > Also what dvb hardware are you using? > I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM > and a TerraTec Cinergy T Stick+ None of them supports an external power supply. > You can found all the details here: > https://bugs.launchpad.net/raspbian/+bug/1819650 > > > Sam > > > Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit : > > > > Hi Sam, > > > > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote: > > > Hi, > > > > > > > > > I am struggling with valgrind because it always complain with either : > > > ASan runtime does not come first in initial library list; you > > > should either link runtime to your application or manually preload it > > > with LD_PRELOAD > > > -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find > > > any core dump > > > > > > or, if I link statically libasan with -static-libasan: > > > Shadow memory range interleaves with an existing memory mapping. > > > ASan cannot proceed correctly. ABORTING. > > > ASan shadow was supposed to be located in the > > > [0x00007fff7000-0x10007fff7fff] range. > > > > > > > > > I retested again on my raspberry zero W, and I confirm i cannot > > > reproduce the hang. > > > Your fix did work on that device. > > > I am testing with same OS (raspbian with latest updates, same kernel), > > > same configure options, same USB dongle... :-( > > > The only differences are CPU architecture (armv6 vs armv7), memory > > > constraints, and I was not using the same channels.conf, I'll fix that > > > today and re-check > > > > Earlier you said "random hangs are back". When this happens, does the whole > > device become unresponsive or just dvbv5-zap? Since this issue is "back", > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > > > > It would be useful to see the output from dmesg (best thing would be after > > the issue occurs). > > > > Also what dvb hardware are you using? > > > > Thanks, > > > > san > > > > > > > > > > > Sam > > > > > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote: > > > > > > Em Mon, 25 Mar 2019 17:33:30 +0100 > > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > > > > Hi guys, > > > > > > I'm afraid I'm coming with sad news. > > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and > > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 > > > ). > > > I previously test both branches on a raspberry zero and issues were gone > > > (same raspbian version). > > > There may be more memory issues somewhere... > > > > > > Could you test it with valgrind? > > > > > > Sam > > > > > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : > > > > > > Hello, > > > > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote: > > > > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and > > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, > > > > > > we can > > > > > > release a new minor version for all those stable branches. > > > > > > After the patches, on my tests, I didn't get any memory leaks or > > > double-free issues. > > > > > > I issues a new 1.12, 1.14, and 1.16 release. > > > > > > Thanks, > > > Gregor > > > > > > > > > > > > Thanks, > > > Mauro Thanks, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-26 15:31 ` Mauro Carvalho Chehab @ 2019-03-28 18:48 ` Samuel CHEMLA 2019-03-28 19:17 ` Mauro Carvalho Chehab 0 siblings, 1 reply; 19+ messages in thread From: Samuel CHEMLA @ 2019-03-28 18:48 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Sean Young, Gregor Jasny, Linux Media Mailing List, Mauro Carvalho Chehab Hi, 1) I did reproduce the bug with a serial console, but the serial console was also frozen, and there was no message before freeze. 2) I also tried a powered USB HUB but it didn't change anything. 3) I tried DVBJet, from https://github.com/lightful/DVBdirect , it is a tuner that directly does ioctl on linux kernel DVB device. I couldn't reproduce the issue with DVBJet. It's been running for almost 48Hrs now, without a crash. I collect all PIDs on the multiplex, just like dvbv5-zap, so the USB port is solicited at it's maximum. That makes me think it is not a hardware issue. I'm pretty sure dvbv5-zap can still improve, and it already did when you fixed: struct arguments args = {}; Regards Le mar. 26 mars 2019 à 16:31, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> a écrit : > > Em Tue, 26 Mar 2019 16:10:33 +0100 > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > Hi, > > > > > > > Earlier you said "random hangs are back". When this happens, does the whole > > > device become unresponsive or just dvbv5-zap? > > The device completely freeze, you can't even switch numlock on/off. > > dvbv5-tools can't hang the machine. this is very likely happening due to > a Kernel crash. > > > I said "the issue is back", it is because I **thought** it was gone. > > To be more precise: > > - on raspberry zero W, the issue is gone since Mauro's patch > > (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89) > > - on raspberry 2, the issue, it is still there and the patch has no > > effect (the issue was and is still there) > > RPi2 has a serious issue with their USB ports: on devices that require > more than a few mW to work, it causes several device disconnection and > re-connection, as it cannot sustain the required 5V. > > Depending on how fast this happens, it could be triggering some Kernel > bug. > > That affects most V4L and DVB devices. You should either use a powered > USB 2.0 hub (with may be problematic, as the USB host driver on RPi > has issues - and may cause data loss on high sustained ISOC traffic, > specially when used with hubs) or a device that has its own power > supply, like DVBSky T680C or S960. > > > > Since this issue is "back", > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > > The issue doesn't appear anymore on raspberry zero W since Mauro's commit. > > So it did improve on that platform. > > > > > It would be useful to see the output from dmesg (best thing would be after > > > the issue occurs). > > You can't, device is frozen. > > Logs are not flushed to disk, and journalctl -f freeze before showing anything > > You can use a serial port in order to get the logs. On a serial console, > use something like: > > # dmesg -n 8 > > In order to make sure it will display all Kernel messages at console. > > > > > > Also what dvb hardware are you using? > > I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM > > and a TerraTec Cinergy T Stick+ > > None of them supports an external power supply. > > > You can found all the details here: > > https://bugs.launchpad.net/raspbian/+bug/1819650 > > > > > > Sam > > > > > > Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit : > > > > > > Hi Sam, > > > > > > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote: > > > > Hi, > > > > > > > > > > > > I am struggling with valgrind because it always complain with either : > > > > ASan runtime does not come first in initial library list; you > > > > should either link runtime to your application or manually preload it > > > > with LD_PRELOAD > > > > -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find > > > > any core dump > > > > > > > > or, if I link statically libasan with -static-libasan: > > > > Shadow memory range interleaves with an existing memory mapping. > > > > ASan cannot proceed correctly. ABORTING. > > > > ASan shadow was supposed to be located in the > > > > [0x00007fff7000-0x10007fff7fff] range. > > > > > > > > > > > > I retested again on my raspberry zero W, and I confirm i cannot > > > > reproduce the hang. > > > > Your fix did work on that device. > > > > I am testing with same OS (raspbian with latest updates, same kernel), > > > > same configure options, same USB dongle... :-( > > > > The only differences are CPU architecture (armv6 vs armv7), memory > > > > constraints, and I was not using the same channels.conf, I'll fix that > > > > today and re-check > > > > > > Earlier you said "random hangs are back". When this happens, does the whole > > > device become unresponsive or just dvbv5-zap? Since this issue is "back", > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > > > > > > It would be useful to see the output from dmesg (best thing would be after > > > the issue occurs). > > > > > > Also what dvb hardware are you using? > > > > > > Thanks, > > > > > > san > > > > > > > > > > > > > > > Sam > > > > > > > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote: > > > > > > > > Em Mon, 25 Mar 2019 17:33:30 +0100 > > > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > > > > > > Hi guys, > > > > > > > > I'm afraid I'm coming with sad news. > > > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and > > > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 > > > > ). > > > > I previously test both branches on a raspberry zero and issues were gone > > > > (same raspbian version). > > > > There may be more memory issues somewhere... > > > > > > > > Could you test it with valgrind? > > > > > > > > Sam > > > > > > > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : > > > > > > > > Hello, > > > > > > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote: > > > > > > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and > > > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, > > > > > > > > we can > > > > > > > > release a new minor version for all those stable branches. > > > > > > > > After the patches, on my tests, I didn't get any memory leaks or > > > > double-free issues. > > > > > > > > I issues a new 1.12, 1.14, and 1.16 release. > > > > > > > > Thanks, > > > > Gregor > > > > > > > > > > > > > > > > Thanks, > > > > Mauro > > > > Thanks, > Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-28 18:48 ` Samuel CHEMLA @ 2019-03-28 19:17 ` Mauro Carvalho Chehab 2019-03-28 19:48 ` Gregor Jasny 2019-04-01 18:58 ` Samuel CHEMLA 0 siblings, 2 replies; 19+ messages in thread From: Mauro Carvalho Chehab @ 2019-03-28 19:17 UTC (permalink / raw) To: Samuel CHEMLA Cc: Sean Young, Gregor Jasny, Linux Media Mailing List, Mauro Carvalho Chehab Em Thu, 28 Mar 2019 19:48:35 +0100 Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > Hi, Please, don't top post. > > > 1) I did reproduce the bug with a serial console, but the serial > console was also frozen, and there was no message before freeze. The only way for a machine to freeze even on serial console is due to a very serious Kernel or hardware bug, like a bad lock/semaphore. > 2) I also tried a powered USB HUB but it didn't change anything. Ok. > 3) I tried DVBJet, from https://github.com/lightful/DVBdirect , it is > a tuner that directly does ioctl on linux kernel DVB device. > I couldn't reproduce the issue with DVBJet. I don't know DVBJet. If it uses the same Kernel drivers, maybe it could then be a race issue: send commands on a slower way to the Kernel would be solving the issue. > It's been running for almost 48Hrs now, without a crash. > I collect all PIDs on the multiplex, just like dvbv5-zap, so the USB > port is solicited at it's maximum. > > That makes me think it is not a hardware issue. > I'm pretty sure dvbv5-zap can still improve, and it already did when > you fixed: struct arguments args = {}; Everything can be improved, but the thing is that we need first to discover the root cause :-) Can you post on pastebin (or equivalent) the dmesg with the DVBJet running? Maybe it could be printing some Kernel messages that might help to discover what's going wrong. Btw, could you also apply this patch: https://patchwork.linuxtv.org/patch/55274/ It is probably unrelated, but this is the kind of bug that could cause such issues. > > > Regards > > Le mar. 26 mars 2019 à 16:31, Mauro Carvalho Chehab > <mchehab+samsung@kernel.org> a écrit : > > > > Em Tue, 26 Mar 2019 16:10:33 +0100 > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > > > Hi, > > > > > > > > > > Earlier you said "random hangs are back". When this happens, does the whole > > > > device become unresponsive or just dvbv5-zap? > > > The device completely freeze, you can't even switch numlock on/off. > > > > dvbv5-tools can't hang the machine. this is very likely happening due to > > a Kernel crash. > > > > > I said "the issue is back", it is because I **thought** it was gone. > > > To be more precise: > > > - on raspberry zero W, the issue is gone since Mauro's patch > > > (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89) > > > - on raspberry 2, the issue, it is still there and the patch has no > > > effect (the issue was and is still there) > > > > RPi2 has a serious issue with their USB ports: on devices that require > > more than a few mW to work, it causes several device disconnection and > > re-connection, as it cannot sustain the required 5V. > > > > Depending on how fast this happens, it could be triggering some Kernel > > bug. > > > > That affects most V4L and DVB devices. You should either use a powered > > USB 2.0 hub (with may be problematic, as the USB host driver on RPi > > has issues - and may cause data loss on high sustained ISOC traffic, > > specially when used with hubs) or a device that has its own power > > supply, like DVBSky T680C or S960. > > > > > > Since this issue is "back", > > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > > > The issue doesn't appear anymore on raspberry zero W since Mauro's commit. > > > So it did improve on that platform. > > > > > > > It would be useful to see the output from dmesg (best thing would be after > > > > the issue occurs). > > > You can't, device is frozen. > > > Logs are not flushed to disk, and journalctl -f freeze before showing anything > > > > You can use a serial port in order to get the logs. On a serial console, > > use something like: > > > > # dmesg -n 8 > > > > In order to make sure it will display all Kernel messages at console. > > > > > > > > > Also what dvb hardware are you using? > > > I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM > > > and a TerraTec Cinergy T Stick+ > > > > None of them supports an external power supply. > > > > > You can found all the details here: > > > https://bugs.launchpad.net/raspbian/+bug/1819650 > > > > > > > > > Sam > > > > > > > > > Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit : > > > > > > > > Hi Sam, > > > > > > > > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote: > > > > > Hi, > > > > > > > > > > > > > > > I am struggling with valgrind because it always complain with either : > > > > > ASan runtime does not come first in initial library list; you > > > > > should either link runtime to your application or manually preload it > > > > > with LD_PRELOAD > > > > > -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find > > > > > any core dump > > > > > > > > > > or, if I link statically libasan with -static-libasan: > > > > > Shadow memory range interleaves with an existing memory mapping. > > > > > ASan cannot proceed correctly. ABORTING. > > > > > ASan shadow was supposed to be located in the > > > > > [0x00007fff7000-0x10007fff7fff] range. > > > > > > > > > > > > > > > I retested again on my raspberry zero W, and I confirm i cannot > > > > > reproduce the hang. > > > > > Your fix did work on that device. > > > > > I am testing with same OS (raspbian with latest updates, same kernel), > > > > > same configure options, same USB dongle... :-( > > > > > The only differences are CPU architecture (armv6 vs armv7), memory > > > > > constraints, and I was not using the same channels.conf, I'll fix that > > > > > today and re-check > > > > > > > > Earlier you said "random hangs are back". When this happens, does the whole > > > > device become unresponsive or just dvbv5-zap? Since this issue is "back", > > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > > > > > > > > It would be useful to see the output from dmesg (best thing would be after > > > > the issue occurs). > > > > > > > > Also what dvb hardware are you using? > > > > > > > > Thanks, > > > > > > > > san > > > > > > > > > > > > > > > > > > > Sam > > > > > > > > > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote: > > > > > > > > > > Em Mon, 25 Mar 2019 17:33:30 +0100 > > > > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > > > > > > > > Hi guys, > > > > > > > > > > I'm afraid I'm coming with sad news. > > > > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and > > > > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 > > > > > ). > > > > > I previously test both branches on a raspberry zero and issues were gone > > > > > (same raspbian version). > > > > > There may be more memory issues somewhere... > > > > > > > > > > Could you test it with valgrind? > > > > > > > > > > Sam > > > > > > > > > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : > > > > > > > > > > Hello, > > > > > > > > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote: > > > > > > > > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and > > > > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, > > > > > > > > > > we can > > > > > > > > > > release a new minor version for all those stable branches. > > > > > > > > > > After the patches, on my tests, I didn't get any memory leaks or > > > > > double-free issues. > > > > > > > > > > I issues a new 1.12, 1.14, and 1.16 release. > > > > > > > > > > Thanks, > > > > > Gregor > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > Mauro > > > > > > > > Thanks, > > Mauro Thanks, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-28 19:17 ` Mauro Carvalho Chehab @ 2019-03-28 19:48 ` Gregor Jasny 2019-03-28 20:04 ` Mauro Carvalho Chehab 2019-04-01 18:58 ` Samuel CHEMLA 1 sibling, 1 reply; 19+ messages in thread From: Gregor Jasny @ 2019-03-28 19:48 UTC (permalink / raw) To: Mauro Carvalho Chehab, Samuel CHEMLA Cc: Sean Young, Linux Media Mailing List, Mauro Carvalho Chehab Hello, On 28.03.19 20:17, Mauro Carvalho Chehab wrote: > Em Thu, 28 Mar 2019 19:48:35 +0100 > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: >> 1) I did reproduce the bug with a serial console, but the serial >> console was also frozen, and there was no message before freeze. > > The only way for a machine to freeze even on serial console is due to > a very serious Kernel or hardware bug, like a bad lock/semaphore. Does anything speak against upgrading Debian to latest 1.16.5 release? Thanks, Gregor ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-28 19:48 ` Gregor Jasny @ 2019-03-28 20:04 ` Mauro Carvalho Chehab 0 siblings, 0 replies; 19+ messages in thread From: Mauro Carvalho Chehab @ 2019-03-28 20:04 UTC (permalink / raw) To: Gregor Jasny Cc: Samuel CHEMLA, Sean Young, Linux Media Mailing List, Mauro Carvalho Chehab Em Thu, 28 Mar 2019 20:48:34 +0100 Gregor Jasny <gjasny@googlemail.com> escreveu: > Hello, > > On 28.03.19 20:17, Mauro Carvalho Chehab wrote: > > Em Thu, 28 Mar 2019 19:48:35 +0100 > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > >> 1) I did reproduce the bug with a serial console, but the serial > >> console was also frozen, and there was no message before freeze. > > > > The only way for a machine to freeze even on serial console is due to > > a very serious Kernel or hardware bug, like a bad lock/semaphore. > > Does anything speak against upgrading Debian to latest 1.16.5 release? I don't think so. The changes we did were just at memory allocation/free with standard glibc malloc()/calloc()/free() functions. The kind of errors Samuel are experiencing are machine freezes. dvbv5-utils can't cause such kind of errors, as it doesn't run in privileged mode, nor do anything potentially harmful. So, even the most serious issue there won't cause a machine crash. For a machine to crash, it has to be triggering a Kernel bug or some memory corruption outside its memory space (e.g. triggering some glibc bug - with also seems very unlikely). Thanks, Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-03-28 19:17 ` Mauro Carvalho Chehab 2019-03-28 19:48 ` Gregor Jasny @ 2019-04-01 18:58 ` Samuel CHEMLA 2019-04-08 6:32 ` Samuel CHEMLA 1 sibling, 1 reply; 19+ messages in thread From: Samuel CHEMLA @ 2019-04-01 18:58 UTC (permalink / raw) To: Mauro Carvalho Chehab Cc: Sean Young, Gregor Jasny, Linux Media Mailing List, Mauro Carvalho Chehab Le jeu. 28 mars 2019 à 20:18, Mauro Carvalho Chehab <mchehab+samsung@kernel.org> a écrit : > > Em Thu, 28 Mar 2019 19:48:35 +0100 > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > Hi, > > Please, don't top post. > > > > > > 1) I did reproduce the bug with a serial console, but the serial > > console was also frozen, and there was no message before freeze. > > The only way for a machine to freeze even on serial console is due to > a very serious Kernel or hardware bug, like a bad lock/semaphore. > > > 2) I also tried a powered USB HUB but it didn't change anything. > > Ok. > > > 3) I tried DVBJet, from https://github.com/lightful/DVBdirect , it is > > a tuner that directly does ioctl on linux kernel DVB device. > > I couldn't reproduce the issue with DVBJet. > > I don't know DVBJet. If it uses the same Kernel drivers, maybe > it could then be a race issue: send commands on a slower way to > the Kernel would be solving the issue. > > > It's been running for almost 48Hrs now, without a crash. > > I collect all PIDs on the multiplex, just like dvbv5-zap, so the USB > > port is solicited at it's maximum. > > > > That makes me think it is not a hardware issue. > > I'm pretty sure dvbv5-zap can still improve, and it already did when > > you fixed: struct arguments args = {}; > > Everything can be improved, but the thing is that we need first to > discover the root cause :-) > > Can you post on pastebin (or equivalent) the dmesg with the > DVBJet running? Maybe it could be printing some Kernel messages > that might help to discover what's going wrong. Here is the dmesg: https://pastebin.com/3XRim4XL Just look at the code, it's quite straightforward, just a few ioctl. > > Btw, could you also apply this patch: > https://patchwork.linuxtv.org/patch/55274/ OK, I'll give it a try and provide feedback > > It is probably unrelated, but this is the kind of bug that could cause > such issues. > > > > > > > Regards > > > > Le mar. 26 mars 2019 à 16:31, Mauro Carvalho Chehab > > <mchehab+samsung@kernel.org> a écrit : > > > > > > Em Tue, 26 Mar 2019 16:10:33 +0100 > > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > > > > > Hi, > > > > > > > > > > > > > Earlier you said "random hangs are back". When this happens, does the whole > > > > > device become unresponsive or just dvbv5-zap? > > > > The device completely freeze, you can't even switch numlock on/off. > > > > > > dvbv5-tools can't hang the machine. this is very likely happening due to > > > a Kernel crash. > > > > > > > I said "the issue is back", it is because I **thought** it was gone. > > > > To be more precise: > > > > - on raspberry zero W, the issue is gone since Mauro's patch > > > > (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89) > > > > - on raspberry 2, the issue, it is still there and the patch has no > > > > effect (the issue was and is still there) > > > > > > RPi2 has a serious issue with their USB ports: on devices that require > > > more than a few mW to work, it causes several device disconnection and > > > re-connection, as it cannot sustain the required 5V. > > > > > > Depending on how fast this happens, it could be triggering some Kernel > > > bug. > > > > > > That affects most V4L and DVB devices. You should either use a powered > > > USB 2.0 hub (with may be problematic, as the USB host driver on RPi > > > has issues - and may cause data loss on high sustained ISOC traffic, > > > specially when used with hubs) or a device that has its own power > > > supply, like DVBSky T680C or S960. > > > > > > > > Since this issue is "back", > > > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > > > > The issue doesn't appear anymore on raspberry zero W since Mauro's commit. > > > > So it did improve on that platform. > > > > > > > > > It would be useful to see the output from dmesg (best thing would be after > > > > > the issue occurs). > > > > You can't, device is frozen. > > > > Logs are not flushed to disk, and journalctl -f freeze before showing anything > > > > > > You can use a serial port in order to get the logs. On a serial console, > > > use something like: > > > > > > # dmesg -n 8 > > > > > > In order to make sure it will display all Kernel messages at console. > > > > > > > > > > > > Also what dvb hardware are you using? > > > > I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM > > > > and a TerraTec Cinergy T Stick+ > > > > > > None of them supports an external power supply. > > > > > > > You can found all the details here: > > > > https://bugs.launchpad.net/raspbian/+bug/1819650 > > > > > > > > > > > > Sam > > > > > > > > > > > > Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit : > > > > > > > > > > Hi Sam, > > > > > > > > > > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote: > > > > > > Hi, > > > > > > > > > > > > > > > > > > I am struggling with valgrind because it always complain with either : > > > > > > ASan runtime does not come first in initial library list; you > > > > > > should either link runtime to your application or manually preload it > > > > > > with LD_PRELOAD > > > > > > -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find > > > > > > any core dump > > > > > > > > > > > > or, if I link statically libasan with -static-libasan: > > > > > > Shadow memory range interleaves with an existing memory mapping. > > > > > > ASan cannot proceed correctly. ABORTING. > > > > > > ASan shadow was supposed to be located in the > > > > > > [0x00007fff7000-0x10007fff7fff] range. > > > > > > > > > > > > > > > > > > I retested again on my raspberry zero W, and I confirm i cannot > > > > > > reproduce the hang. > > > > > > Your fix did work on that device. > > > > > > I am testing with same OS (raspbian with latest updates, same kernel), > > > > > > same configure options, same USB dongle... :-( > > > > > > The only differences are CPU architecture (armv6 vs armv7), memory > > > > > > constraints, and I was not using the same channels.conf, I'll fix that > > > > > > today and re-check > > > > > > > > > > Earlier you said "random hangs are back". When this happens, does the whole > > > > > device become unresponsive or just dvbv5-zap? Since this issue is "back", > > > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. > > > > > > > > > > It would be useful to see the output from dmesg (best thing would be after > > > > > the issue occurs). > > > > > > > > > > Also what dvb hardware are you using? > > > > > > > > > > Thanks, > > > > > > > > > > san > > > > > > > > > > > > > > > > > > > > > > > Sam > > > > > > > > > > > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote: > > > > > > > > > > > > Em Mon, 25 Mar 2019 17:33:30 +0100 > > > > > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: > > > > > > > > > > > > Hi guys, > > > > > > > > > > > > I'm afraid I'm coming with sad news. > > > > > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and > > > > > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 > > > > > > ). > > > > > > I previously test both branches on a raspberry zero and issues were gone > > > > > > (same raspbian version). > > > > > > There may be more memory issues somewhere... > > > > > > > > > > > > Could you test it with valgrind? > > > > > > > > > > > > Sam > > > > > > > > > > > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : > > > > > > > > > > > > Hello, > > > > > > > > > > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote: > > > > > > > > > > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and > > > > > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, > > > > > > > > > > > > we can > > > > > > > > > > > > release a new minor version for all those stable branches. > > > > > > > > > > > > After the patches, on my tests, I didn't get any memory leaks or > > > > > > double-free issues. > > > > > > > > > > > > I issues a new 1.12, 1.14, and 1.16 release. > > > > > > > > > > > > Thanks, > > > > > > Gregor > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > Mauro > > > > > > > > > > > > Thanks, > > > Mauro > > > > Thanks, > Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds 2019-04-01 18:58 ` Samuel CHEMLA @ 2019-04-08 6:32 ` Samuel CHEMLA 0 siblings, 0 replies; 19+ messages in thread From: Samuel CHEMLA @ 2019-04-08 6:32 UTC (permalink / raw) To: Mauro Carvalho Chehab; +Cc: Sean Young, Gregor Jasny, Linux Media Mailing List On 01/04/2019 20:58, Samuel CHEMLA wrote: > Le jeu. 28 mars 2019 à 20:18, Mauro Carvalho Chehab > <mchehab+samsung@kernel.org> a écrit : >> Em Thu, 28 Mar 2019 19:48:35 +0100 >> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: >> >>> Hi, >> Please, don't top post. >>> >>> 1) I did reproduce the bug with a serial console, but the serial >>> console was also frozen, and there was no message before freeze. >> The only way for a machine to freeze even on serial console is due to >> a very serious Kernel or hardware bug, like a bad lock/semaphore. >> >>> 2) I also tried a powered USB HUB but it didn't change anything. >> Ok. >> >>> 3) I tried DVBJet, from https://github.com/lightful/DVBdirect , it is >>> a tuner that directly does ioctl on linux kernel DVB device. >>> I couldn't reproduce the issue with DVBJet. >> I don't know DVBJet. If it uses the same Kernel drivers, maybe >> it could then be a race issue: send commands on a slower way to >> the Kernel would be solving the issue. >> >>> It's been running for almost 48Hrs now, without a crash. >>> I collect all PIDs on the multiplex, just like dvbv5-zap, so the USB >>> port is solicited at it's maximum. >>> >>> That makes me think it is not a hardware issue. >>> I'm pretty sure dvbv5-zap can still improve, and it already did when >>> you fixed: struct arguments args = {}; >> Everything can be improved, but the thing is that we need first to >> discover the root cause :-) >> >> Can you post on pastebin (or equivalent) the dmesg with the >> DVBJet running? Maybe it could be printing some Kernel messages >> that might help to discover what's going wrong. > Here is the dmesg: https://pastebin.com/3XRim4XL > Just look at the code, it's quite straightforward, just a few ioctl. > >> Btw, could you also apply this patch: >> https://patchwork.linuxtv.org/patch/55274/ > OK, I'll give it a try and provide feedback I tested you patch against 4.14.98 which is the current raspbian kernel, but it didn't change anything... > >> It is probably unrelated, but this is the kind of bug that could cause >> such issues. >> >>> >>> Regards >>> >>> Le mar. 26 mars 2019 à 16:31, Mauro Carvalho Chehab >>> <mchehab+samsung@kernel.org> a écrit : >>>> Em Tue, 26 Mar 2019 16:10:33 +0100 >>>> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: >>>> >>>>> Hi, >>>>> >>>>> >>>>>> Earlier you said "random hangs are back". When this happens, does the whole >>>>>> device become unresponsive or just dvbv5-zap? >>>>> The device completely freeze, you can't even switch numlock on/off. >>>> dvbv5-tools can't hang the machine. this is very likely happening due to >>>> a Kernel crash. >>>> >>>>> I said "the issue is back", it is because I **thought** it was gone. >>>>> To be more precise: >>>>> - on raspberry zero W, the issue is gone since Mauro's patch >>>>> (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89) >>>>> - on raspberry 2, the issue, it is still there and the patch has no >>>>> effect (the issue was and is still there) >>>> RPi2 has a serious issue with their USB ports: on devices that require >>>> more than a few mW to work, it causes several device disconnection and >>>> re-connection, as it cannot sustain the required 5V. >>>> >>>> Depending on how fast this happens, it could be triggering some Kernel >>>> bug. >>>> >>>> That affects most V4L and DVB devices. You should either use a powered >>>> USB 2.0 hub (with may be problematic, as the USB host driver on RPi >>>> has issues - and may cause data loss on high sustained ISOC traffic, >>>> specially when used with hubs) or a device that has its own power >>>> supply, like DVBSky T680C or S960. >>>> >>>>>> Since this issue is "back", >>>>>> I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. >>>>> The issue doesn't appear anymore on raspberry zero W since Mauro's commit. >>>>> So it did improve on that platform. >>>>> >>>>>> It would be useful to see the output from dmesg (best thing would be after >>>>>> the issue occurs). >>>>> You can't, device is frozen. >>>>> Logs are not flushed to disk, and journalctl -f freeze before showing anything >>>> You can use a serial port in order to get the logs. On a serial console, >>>> use something like: >>>> >>>> # dmesg -n 8 >>>> >>>> In order to make sure it will display all Kernel messages at console. >>>> >>>>>> Also what dvb hardware are you using? >>>>> I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM >>>>> and a TerraTec Cinergy T Stick+ >>>> None of them supports an external power supply. >>>> >>>>> You can found all the details here: >>>>> https://bugs.launchpad.net/raspbian/+bug/1819650 >>>>> >>>>> >>>>> Sam >>>>> >>>>> >>>>> Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit : >>>>>> Hi Sam, >>>>>> >>>>>> On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote: >>>>>>> Hi, >>>>>>> >>>>>>> >>>>>>> I am struggling with valgrind because it always complain with either : >>>>>>> ASan runtime does not come first in initial library list; you >>>>>>> should either link runtime to your application or manually preload it >>>>>>> with LD_PRELOAD >>>>>>> -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find >>>>>>> any core dump >>>>>>> >>>>>>> or, if I link statically libasan with -static-libasan: >>>>>>> Shadow memory range interleaves with an existing memory mapping. >>>>>>> ASan cannot proceed correctly. ABORTING. >>>>>>> ASan shadow was supposed to be located in the >>>>>>> [0x00007fff7000-0x10007fff7fff] range. >>>>>>> >>>>>>> >>>>>>> I retested again on my raspberry zero W, and I confirm i cannot >>>>>>> reproduce the hang. >>>>>>> Your fix did work on that device. >>>>>>> I am testing with same OS (raspbian with latest updates, same kernel), >>>>>>> same configure options, same USB dongle... :-( >>>>>>> The only differences are CPU architecture (armv6 vs armv7), memory >>>>>>> constraints, and I was not using the same channels.conf, I'll fix that >>>>>>> today and re-check >>>>>> Earlier you said "random hangs are back". When this happens, does the whole >>>>>> device become unresponsive or just dvbv5-zap? Since this issue is "back", >>>>>> I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4. >>>>>> >>>>>> It would be useful to see the output from dmesg (best thing would be after >>>>>> the issue occurs). >>>>>> >>>>>> Also what dvb hardware are you using? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> san >>>>>> >>>>>>> >>>>>>> Sam >>>>>>> >>>>>>> On 25/03/2019 18:08, Mauro Carvalho Chehab wrote: >>>>>>> >>>>>>> Em Mon, 25 Mar 2019 17:33:30 +0100 >>>>>>> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu: >>>>>>> >>>>>>> Hi guys, >>>>>>> >>>>>>> I'm afraid I'm coming with sad news. >>>>>>> I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and >>>>>>> random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650 >>>>>>> ). >>>>>>> I previously test both branches on a raspberry zero and issues were gone >>>>>>> (same raspbian version). >>>>>>> There may be more memory issues somewhere... >>>>>>> >>>>>>> Could you test it with valgrind? >>>>>>> >>>>>>> Sam >>>>>>> >>>>>>> Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit : >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> On 21.03.19 12:30, Mauro Carvalho Chehab wrote: >>>>>>> >>>>>>> I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and >>>>>>> -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, >>>>>>> >>>>>>> we can >>>>>>> >>>>>>> release a new minor version for all those stable branches. >>>>>>> >>>>>>> After the patches, on my tests, I didn't get any memory leaks or >>>>>>> double-free issues. >>>>>>> >>>>>>> I issues a new 1.12, 1.14, and 1.16 release. >>>>>>> >>>>>>> Thanks, >>>>>>> Gregor >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Mauro >>>> >>>> >>>> Thanks, >>>> Mauro >> >> >> Thanks, >> Mauro ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2019-04-08 6:32 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <f4b69417-06c3-f9ab-2973-ae23d76088b8@gmail.com> [not found] ` <29bad771-843c-1dee-906c-6e9475aed7d8@gmail.com> 2019-03-12 15:07 ` [Bug report] dvbv5-zap crash dvb-tool ARMHF builds Gregor Jasny 2019-03-15 22:34 ` Sean Young 2019-03-17 9:52 ` Mauro Carvalho Chehab 2019-03-19 19:45 ` Mauro Carvalho Chehab 2019-03-20 19:38 ` Gregor Jasny 2019-03-21 9:41 ` Sean Young 2019-03-21 11:30 ` Mauro Carvalho Chehab 2019-03-21 19:59 ` Gregor Jasny [not found] ` <CANJnhGfRtEwAony5Z4rFMPcu58aF2k0G+9NSkMKsq_PhfmSNqw@mail.gmail.com> 2019-03-25 17:08 ` Mauro Carvalho Chehab 2019-03-26 7:35 ` Samuel CHEMLA 2019-03-26 13:26 ` Sean Young 2019-03-26 15:10 ` Samuel CHEMLA 2019-03-26 15:31 ` Mauro Carvalho Chehab 2019-03-28 18:48 ` Samuel CHEMLA 2019-03-28 19:17 ` Mauro Carvalho Chehab 2019-03-28 19:48 ` Gregor Jasny 2019-03-28 20:04 ` Mauro Carvalho Chehab 2019-04-01 18:58 ` Samuel CHEMLA 2019-04-08 6:32 ` Samuel CHEMLA
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).