Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Gregor Jasny]

Hello Mauro,

below you find a bug report about an use-after-free in dvbv5-zap.

On 12.03.19 13:37, CHEMLA Samuel wrote:
> please find a bug report that seems to concern ARMHF builds of dvbv5-zap 
> (dvb-tool package) : https://bugs.launchpad.net/raspbian/+bug/1819650
> I filed it against raspbian because I thought it was a raspbian problem, 
> but don't think they re-build their own package, but use debian ones 
> instead...

Thanks,
Gregor

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Sean Young]

Hi,

On Tue, Mar 12, 2019 at 04:07:23PM +0100, Gregor Jasny wrote:
> Hello Mauro,
> 
> below you find a bug report about an use-after-free in dvbv5-zap.
> 
> On 12.03.19 13:37, CHEMLA Samuel wrote:
> > please find a bug report that seems to concern ARMHF builds of dvbv5-zap
> > (dvb-tool package) : https://bugs.launchpad.net/raspbian/+bug/1819650
> > I filed it against raspbian because I thought it was a raspbian problem,
> > but don't think they re-build their own package, but use debian ones
> > instead...
> 

So I can reproduce the issue with v4l-utils 1.12.3 but not with current
v4l-utils (or dvbv5-zap). It looks exactly like the issue fixed in
commit 6e21f6f34c1d7c3a7a059062e1ddd9705c984e2c (but I did not cherry-pick
and test that on top of 1.12.3 to test that theory).


Sean

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Mauro Carvalho Chehab]

Em Fri, 15 Mar 2019 22:34:25 +0000
Sean Young <sean@mess.org> escreveu:

> Hi,
> 
> On Tue, Mar 12, 2019 at 04:07:23PM +0100, Gregor Jasny wrote:
> > Hello Mauro,
> > 
> > below you find a bug report about an use-after-free in dvbv5-zap.
> > 
> > On 12.03.19 13:37, CHEMLA Samuel wrote:  
> > > please find a bug report that seems to concern ARMHF builds of dvbv5-zap
> > > (dvb-tool package) : https://bugs.launchpad.net/raspbian/+bug/1819650
> > > I filed it against raspbian because I thought it was a raspbian problem,
> > > but don't think they re-build their own package, but use debian ones
> > > instead...  
> >   
> 
> So I can reproduce the issue with v4l-utils 1.12.3 but not with current
> v4l-utils (or dvbv5-zap). It looks exactly like the issue fixed in
> commit 6e21f6f34c1d7c3a7a059062e1ddd9705c984e2c (but I did not cherry-pick
> and test that on top of 1.12.3 to test that theory).

I added it to stable/1.12 and another patch fixing the initialization of 
the parameters struct, with Samuel reported to fix the issue upstream
(He pinged me in priv too, and I'm helping him to track it).

Samuel,

Could you please check if the 1.12 stable branch is OK now?

Regards,
Mauro
Thanks,
Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Mauro Carvalho Chehab]

Hi Gregor,

Samuel reported in priv that the issues he had with user after free were
solved by the patchsets merged at 1.12 and 1.16 stable branches.

Could you please generate a new staging release for them?

Thanks!
Mauro

Em Sun, 17 Mar 2019 06:52:42 -0300
Mauro Carvalho Chehab <mchehab+samsung@kernel.org> escreveu:

> Em Fri, 15 Mar 2019 22:34:25 +0000
> Sean Young <sean@mess.org> escreveu:
> 
> > Hi,
> > 
> > On Tue, Mar 12, 2019 at 04:07:23PM +0100, Gregor Jasny wrote:  
> > > Hello Mauro,
> > > 
> > > below you find a bug report about an use-after-free in dvbv5-zap.
> > > 
> > > On 12.03.19 13:37, CHEMLA Samuel wrote:    
> > > > please find a bug report that seems to concern ARMHF builds of dvbv5-zap
> > > > (dvb-tool package) : https://bugs.launchpad.net/raspbian/+bug/1819650
> > > > I filed it against raspbian because I thought it was a raspbian problem,
> > > > but don't think they re-build their own package, but use debian ones
> > > > instead...    
> > >     
> > 
> > So I can reproduce the issue with v4l-utils 1.12.3 but not with current
> > v4l-utils (or dvbv5-zap). It looks exactly like the issue fixed in
> > commit 6e21f6f34c1d7c3a7a059062e1ddd9705c984e2c (but I did not cherry-pick
> > and test that on top of 1.12.3 to test that theory).  
> 
> I added it to stable/1.12 and another patch fixing the initialization of 
> the parameters struct, with Samuel reported to fix the issue upstream
> (He pinged me in priv too, and I'm helping him to track it).
> 
> Samuel,
> 
> Could you please check if the 1.12 stable branch is OK now?
> 
> Regards,
> Mauro
> Thanks,
> Mauro



Thanks,
Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Gregor Jasny]

Hello Mauro,

On 19.03.19 20:45, Mauro Carvalho Chehab wrote:
> Hi Gregor,
> 
> Samuel reported in priv that the issues he had with user after free were
> solved by the patchsets merged at 1.12 and 1.16 stable branches.
> 
> Could you please generate a new staging release for them?

Sure, I can create a new 1.12 and 1.16 stable release. But when 
reviewing the patches for approval by debian release managers I noticed 
an additional double-free that Sean addressed with the following patch:

> https://git.linuxtv.org/v4l-utils.git/commit/?id=ebd890019ba7383b8b486d829f6683c8f49fdbda

Could you please give that patch a thorough review, some testing, and 
cherry-pick it to stable-1.12 and -1.16 as well?

Thanks,
Gregor

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Sean Young]

On Wed, Mar 20, 2019 at 08:38:52PM +0100, Gregor Jasny wrote:
> Hello Mauro,
> 
> On 19.03.19 20:45, Mauro Carvalho Chehab wrote:
> > Hi Gregor,
> > 
> > Samuel reported in priv that the issues he had with user after free were
> > solved by the patchsets merged at 1.12 and 1.16 stable branches.
> > 
> > Could you please generate a new staging release for them?
> 
> Sure, I can create a new 1.12 and 1.16 stable release. But when reviewing
> the patches for approval by debian release managers I noticed an additional
> double-free that Sean addressed with the following patch:
> 
> > https://git.linuxtv.org/v4l-utils.git/commit/?id=ebd890019ba7383b8b486d829f6683c8f49fdbda
> 
> Could you please give that patch a thorough review, some testing, and
> cherry-pick it to stable-1.12 and -1.16 as well?

I did test it myself (and also under valgrind). The bad paths are hard
to hit though. I'd say just go ahead with merging and releasing, the patch
isn't that controversial (I hope!).


Sean

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Mauro Carvalho Chehab]

Em Thu, 21 Mar 2019 09:41:28 +0000
Sean Young <sean@mess.org> escreveu:

> On Wed, Mar 20, 2019 at 08:38:52PM +0100, Gregor Jasny wrote:
> > Hello Mauro,
> > 
> > On 19.03.19 20:45, Mauro Carvalho Chehab wrote:
> > > Hi Gregor,
> > > 
> > > Samuel reported in priv that the issues he had with user after free were
> > > solved by the patchsets merged at 1.12 and 1.16 stable branches.
> > > 
> > > Could you please generate a new staging release for them?
> > 
> > Sure, I can create a new 1.12 and 1.16 stable release. But when reviewing
> > the patches for approval by debian release managers I noticed an additional
> > double-free that Sean addressed with the following patch:
> > 
> > > https://git.linuxtv.org/v4l-utils.git/commit/?id=ebd890019ba7383b8b486d829f6683c8f49fdbda
> > 
> > Could you please give that patch a thorough review, some testing, and
> > cherry-pick it to stable-1.12 and -1.16 as well?
> 
> I did test it myself (and also under valgrind). The bad paths are hard
> to hit though. I'd say just go ahead with merging and releasing, the patch
> isn't that controversial (I hope!).

I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
-1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, we can
release a new minor version for all those stable branches.

After the patches, on my tests, I didn't get any memory leaks or
double-free issues.

It should be noticed that I had to add a new patch at -1.14, due to
the usage of minor() and major() macros, as one of the files there
were not including sys/sysmacros.h. Without that, I was getting
compilation errors.

Funny enough, this header was already included on two other places
within -1.14.

As the major() and minor() macros were added at glibc 2.3.3[1], released
in 2004 [2], it should be save to include sys/sysmacros.h
unconditionally at stable-1.14.

[1] according with "man 3 makedev", as pointed by:
https://stackoverflow.com/questions/22240973/major-and-minor-macros-defined-in-sys-sysmacros-h-pulled-in-by-iterator

[2] https://ftp.gnu.org/gnu/libc/'s glibc tarball is from 2004-08-03.

Thanks,
Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Gregor Jasny]

Hello,

On 21.03.19 12:30, Mauro Carvalho Chehab wrote:
> I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
> -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So, we can
> release a new minor version for all those stable branches.
> 
> After the patches, on my tests, I didn't get any memory leaks or
> double-free issues.

I issues a new 1.12, 1.14, and 1.16 release.

Thanks,
Gregor

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Mauro Carvalho Chehab]

Em Mon, 25 Mar 2019 17:33:30 +0100
Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:

> Hi guys,
> 
> I'm afraid I'm coming with sad news.
> I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
> random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
> ).
> I previously test both branches on a raspberry zero and issues were gone
> (same raspbian version).
> There may be more memory issues somewhere...

Could you test it with valgrind? 

> 
> Sam
> 
> Le jeu. 21 mars 2019 à 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :
> 
> > Hello,
> >
> > On 21.03.19 12:30, Mauro Carvalho Chehab wrote:  
> > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
> > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,  
> > we can  
> > > release a new minor version for all those stable branches.
> > >
> > > After the patches, on my tests, I didn't get any memory leaks or
> > > double-free issues.  
> >
> > I issues a new 1.12, 1.14, and 1.16 release.
> >
> > Thanks,
> > Gregor
> >
> >  



Thanks,
Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Samuel CHEMLA]

Hi,


I am struggling with valgrind because it always complain with either :
    ASan runtime does not come first in initial library list; you
should either link runtime to your application or manually preload it
with LD_PRELOAD
    -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find
any core dump

or, if I link statically libasan with -static-libasan:
    Shadow memory range interleaves with an existing memory mapping.
ASan cannot proceed correctly. ABORTING.
    ASan shadow was supposed to be located in the
[0x00007fff7000-0x10007fff7fff] range.


I retested again on my raspberry zero W, and I confirm i cannot
reproduce the hang.
Your fix did work on that device.
I am testing with same OS (raspbian with latest updates, same kernel),
same configure options, same USB dongle... :-(
The only differences are CPU architecture (armv6 vs armv7), memory
constraints, and I was not using the same channels.conf, I'll fix that
today and re-check


Sam

On 25/03/2019 18:08, Mauro Carvalho Chehab wrote:

Em Mon, 25 Mar 2019 17:33:30 +0100
Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:

Hi guys,

I'm afraid I'm coming with sad news.
I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
).
I previously test both branches on a raspberry zero and issues were gone
(same raspbian version).
There may be more memory issues somewhere...

Could you test it with valgrind?

Sam

Le jeu. 21 mars 2019 à 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :

Hello,

On 21.03.19 12:30, Mauro Carvalho Chehab wrote:

I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
-1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,

we can

release a new minor version for all those stable branches.

After the patches, on my tests, I didn't get any memory leaks or
double-free issues.

I issues a new 1.12, 1.14, and 1.16 release.

Thanks,
Gregor



Thanks,
Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Sean Young]

Hi Sam,

On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote:
> Hi,
> 
> 
> I am struggling with valgrind because it always complain with either :
>     ASan runtime does not come first in initial library list; you
> should either link runtime to your application or manually preload it
> with LD_PRELOAD
>     -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find
> any core dump
> 
> or, if I link statically libasan with -static-libasan:
>     Shadow memory range interleaves with an existing memory mapping.
> ASan cannot proceed correctly. ABORTING.
>     ASan shadow was supposed to be located in the
> [0x00007fff7000-0x10007fff7fff] range.
> 
> 
> I retested again on my raspberry zero W, and I confirm i cannot
> reproduce the hang.
> Your fix did work on that device.
> I am testing with same OS (raspbian with latest updates, same kernel),
> same configure options, same USB dongle... :-(
> The only differences are CPU architecture (armv6 vs armv7), memory
> constraints, and I was not using the same channels.conf, I'll fix that
> today and re-check

Earlier you said "random hangs are back". When this happens, does the whole
device become unresponsive or just dvbv5-zap? Since this issue is "back",
I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.

It would be useful to see the output from dmesg (best thing would be after
the issue occurs).

Also what dvb hardware are you using?

Thanks,

san

> 
> 
> Sam
> 
> On 25/03/2019 18:08, Mauro Carvalho Chehab wrote:
> 
> Em Mon, 25 Mar 2019 17:33:30 +0100
> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
> 
> Hi guys,
> 
> I'm afraid I'm coming with sad news.
> I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
> random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
> ).
> I previously test both branches on a raspberry zero and issues were gone
> (same raspbian version).
> There may be more memory issues somewhere...
> 
> Could you test it with valgrind?
> 
> Sam
> 
> Le jeu. 21 mars 2019 à 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :
> 
> Hello,
> 
> On 21.03.19 12:30, Mauro Carvalho Chehab wrote:
> 
> I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
> -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,
> 
> we can
> 
> release a new minor version for all those stable branches.
> 
> After the patches, on my tests, I didn't get any memory leaks or
> double-free issues.
> 
> I issues a new 1.12, 1.14, and 1.16 release.
> 
> Thanks,
> Gregor
> 
> 
> 
> Thanks,
> Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Samuel CHEMLA]

Hi,


> Earlier you said "random hangs are back". When this happens, does the whole
> device become unresponsive or just dvbv5-zap?
The device completely freeze, you can't even switch numlock on/off.
I said "the issue is back", it is because I **thought** it was gone.
To be more precise:
- on raspberry zero W, the issue is gone since Mauro's patch
(https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89)
- on raspberry 2, the issue, it is still there and the patch has no
effect (the issue was and is still there)

> Since this issue is "back",
> I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
The issue doesn't appear anymore on raspberry zero W since Mauro's commit.
So it did improve on that platform.

> It would be useful to see the output from dmesg (best thing would be after
> the issue occurs).
You can't, device is frozen.
Logs are not flushed to disk, and journalctl -f freeze before showing anything

> Also what dvb hardware are you using?
I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM
and a TerraTec Cinergy T Stick+


You can found all the details here:
https://bugs.launchpad.net/raspbian/+bug/1819650


Sam


Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit :
>
> Hi Sam,
>
> On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote:
> > Hi,
> >
> >
> > I am struggling with valgrind because it always complain with either :
> >     ASan runtime does not come first in initial library list; you
> > should either link runtime to your application or manually preload it
> > with LD_PRELOAD
> >     -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find
> > any core dump
> >
> > or, if I link statically libasan with -static-libasan:
> >     Shadow memory range interleaves with an existing memory mapping.
> > ASan cannot proceed correctly. ABORTING.
> >     ASan shadow was supposed to be located in the
> > [0x00007fff7000-0x10007fff7fff] range.
> >
> >
> > I retested again on my raspberry zero W, and I confirm i cannot
> > reproduce the hang.
> > Your fix did work on that device.
> > I am testing with same OS (raspbian with latest updates, same kernel),
> > same configure options, same USB dongle... :-(
> > The only differences are CPU architecture (armv6 vs armv7), memory
> > constraints, and I was not using the same channels.conf, I'll fix that
> > today and re-check
>
> Earlier you said "random hangs are back". When this happens, does the whole
> device become unresponsive or just dvbv5-zap? Since this issue is "back",
> I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
>
> It would be useful to see the output from dmesg (best thing would be after
> the issue occurs).
>
> Also what dvb hardware are you using?
>
> Thanks,
>
> san
>
> >
> >
> > Sam
> >
> > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote:
> >
> > Em Mon, 25 Mar 2019 17:33:30 +0100
> > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
> >
> > Hi guys,
> >
> > I'm afraid I'm coming with sad news.
> > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
> > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
> > ).
> > I previously test both branches on a raspberry zero and issues were gone
> > (same raspbian version).
> > There may be more memory issues somewhere...
> >
> > Could you test it with valgrind?
> >
> > Sam
> >
> > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :
> >
> > Hello,
> >
> > On 21.03.19 12:30, Mauro Carvalho Chehab wrote:
> >
> > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
> > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,
> >
> > we can
> >
> > release a new minor version for all those stable branches.
> >
> > After the patches, on my tests, I didn't get any memory leaks or
> > double-free issues.
> >
> > I issues a new 1.12, 1.14, and 1.16 release.
> >
> > Thanks,
> > Gregor
> >
> >
> >
> > Thanks,
> > Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Mauro Carvalho Chehab]

Em Tue, 26 Mar 2019 16:10:33 +0100
Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:

> Hi,
> 
> 
> > Earlier you said "random hangs are back". When this happens, does the whole
> > device become unresponsive or just dvbv5-zap?  
> The device completely freeze, you can't even switch numlock on/off.

dvbv5-tools can't hang the machine. this is very likely happening due to
a Kernel crash.

> I said "the issue is back", it is because I **thought** it was gone.
> To be more precise:
> - on raspberry zero W, the issue is gone since Mauro's patch
> (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89)
> - on raspberry 2, the issue, it is still there and the patch has no
> effect (the issue was and is still there)

RPi2 has a serious issue with their USB ports: on devices that require
more than a few mW to work, it causes several device disconnection and
re-connection, as it cannot sustain the required 5V.

Depending on how fast this happens, it could be triggering some Kernel
bug.

That affects most V4L and DVB devices. You should either use a powered
USB 2.0 hub (with may be problematic, as the USB host driver on RPi
has issues - and may cause data loss on high sustained ISOC traffic,
specially when used with hubs) or a device that has its own power
supply, like DVBSky T680C or S960. 

> > Since this issue is "back",
> > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.  
> The issue doesn't appear anymore on raspberry zero W since Mauro's commit.
> So it did improve on that platform.
> 
> > It would be useful to see the output from dmesg (best thing would be after
> > the issue occurs).  
> You can't, device is frozen.
> Logs are not flushed to disk, and journalctl -f freeze before showing anything

You can use a serial port in order to get the logs. On a serial console,
use something like:

	# dmesg -n 8

In order to make sure it will display all Kernel messages at console.

> 
> > Also what dvb hardware are you using?  
> I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM
> and a TerraTec Cinergy T Stick+

None of them supports an external power supply.

> You can found all the details here:
> https://bugs.launchpad.net/raspbian/+bug/1819650
> 
> 
> Sam
> 
> 
> Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit :
> >
> > Hi Sam,
> >
> > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote:  
> > > Hi,
> > >
> > >
> > > I am struggling with valgrind because it always complain with either :
> > >     ASan runtime does not come first in initial library list; you
> > > should either link runtime to your application or manually preload it
> > > with LD_PRELOAD  
> > >     -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find  
> > > any core dump
> > >
> > > or, if I link statically libasan with -static-libasan:
> > >     Shadow memory range interleaves with an existing memory mapping.
> > > ASan cannot proceed correctly. ABORTING.
> > >     ASan shadow was supposed to be located in the
> > > [0x00007fff7000-0x10007fff7fff] range.
> > >
> > >
> > > I retested again on my raspberry zero W, and I confirm i cannot
> > > reproduce the hang.
> > > Your fix did work on that device.
> > > I am testing with same OS (raspbian with latest updates, same kernel),
> > > same configure options, same USB dongle... :-(
> > > The only differences are CPU architecture (armv6 vs armv7), memory
> > > constraints, and I was not using the same channels.conf, I'll fix that
> > > today and re-check  
> >
> > Earlier you said "random hangs are back". When this happens, does the whole
> > device become unresponsive or just dvbv5-zap? Since this issue is "back",
> > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
> >
> > It would be useful to see the output from dmesg (best thing would be after
> > the issue occurs).
> >
> > Also what dvb hardware are you using?
> >
> > Thanks,
> >
> > san
> >  
> > >
> > >
> > > Sam
> > >
> > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote:
> > >
> > > Em Mon, 25 Mar 2019 17:33:30 +0100
> > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
> > >
> > > Hi guys,
> > >
> > > I'm afraid I'm coming with sad news.
> > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
> > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
> > > ).
> > > I previously test both branches on a raspberry zero and issues were gone
> > > (same raspbian version).
> > > There may be more memory issues somewhere...
> > >
> > > Could you test it with valgrind?
> > >
> > > Sam
> > >
> > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :
> > >
> > > Hello,
> > >
> > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote:
> > >
> > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
> > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,
> > >
> > > we can
> > >
> > > release a new minor version for all those stable branches.
> > >
> > > After the patches, on my tests, I didn't get any memory leaks or
> > > double-free issues.
> > >
> > > I issues a new 1.12, 1.14, and 1.16 release.
> > >
> > > Thanks,
> > > Gregor
> > >
> > >
> > >
> > > Thanks,
> > > Mauro  



Thanks,
Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Samuel CHEMLA]

Hi,


1) I did reproduce the bug with a serial console, but the serial
console was also frozen, and there was no message before freeze.
2) I also tried a powered USB HUB but it didn't change anything.
3) I tried DVBJet, from https://github.com/lightful/DVBdirect , it is
a tuner that directly does ioctl on linux kernel DVB device.
I couldn't reproduce the issue with DVBJet.
It's been running for almost 48Hrs now, without a crash.
I collect all PIDs on the multiplex, just like dvbv5-zap, so the USB
port is solicited at it's maximum.

That makes me think it is not a hardware issue.
I'm pretty sure dvbv5-zap can still improve, and it already did when
you fixed: struct arguments args = {};


Regards

Le mar. 26 mars 2019 à 16:31, Mauro Carvalho Chehab
<mchehab+samsung@kernel.org> a écrit :
>
> Em Tue, 26 Mar 2019 16:10:33 +0100
> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
>
> > Hi,
> >
> >
> > > Earlier you said "random hangs are back". When this happens, does the whole
> > > device become unresponsive or just dvbv5-zap?
> > The device completely freeze, you can't even switch numlock on/off.
>
> dvbv5-tools can't hang the machine. this is very likely happening due to
> a Kernel crash.
>
> > I said "the issue is back", it is because I **thought** it was gone.
> > To be more precise:
> > - on raspberry zero W, the issue is gone since Mauro's patch
> > (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89)
> > - on raspberry 2, the issue, it is still there and the patch has no
> > effect (the issue was and is still there)
>
> RPi2 has a serious issue with their USB ports: on devices that require
> more than a few mW to work, it causes several device disconnection and
> re-connection, as it cannot sustain the required 5V.
>
> Depending on how fast this happens, it could be triggering some Kernel
> bug.
>
> That affects most V4L and DVB devices. You should either use a powered
> USB 2.0 hub (with may be problematic, as the USB host driver on RPi
> has issues - and may cause data loss on high sustained ISOC traffic,
> specially when used with hubs) or a device that has its own power
> supply, like DVBSky T680C or S960.
>
> > > Since this issue is "back",
> > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
> > The issue doesn't appear anymore on raspberry zero W since Mauro's commit.
> > So it did improve on that platform.
> >
> > > It would be useful to see the output from dmesg (best thing would be after
> > > the issue occurs).
> > You can't, device is frozen.
> > Logs are not flushed to disk, and journalctl -f freeze before showing anything
>
> You can use a serial port in order to get the logs. On a serial console,
> use something like:
>
>         # dmesg -n 8
>
> In order to make sure it will display all Kernel messages at console.
>
> >
> > > Also what dvb hardware are you using?
> > I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM
> > and a TerraTec Cinergy T Stick+
>
> None of them supports an external power supply.
>
> > You can found all the details here:
> > https://bugs.launchpad.net/raspbian/+bug/1819650
> >
> >
> > Sam
> >
> >
> > Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit :
> > >
> > > Hi Sam,
> > >
> > > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote:
> > > > Hi,
> > > >
> > > >
> > > > I am struggling with valgrind because it always complain with either :
> > > >     ASan runtime does not come first in initial library list; you
> > > > should either link runtime to your application or manually preload it
> > > > with LD_PRELOAD
> > > >     -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find
> > > > any core dump
> > > >
> > > > or, if I link statically libasan with -static-libasan:
> > > >     Shadow memory range interleaves with an existing memory mapping.
> > > > ASan cannot proceed correctly. ABORTING.
> > > >     ASan shadow was supposed to be located in the
> > > > [0x00007fff7000-0x10007fff7fff] range.
> > > >
> > > >
> > > > I retested again on my raspberry zero W, and I confirm i cannot
> > > > reproduce the hang.
> > > > Your fix did work on that device.
> > > > I am testing with same OS (raspbian with latest updates, same kernel),
> > > > same configure options, same USB dongle... :-(
> > > > The only differences are CPU architecture (armv6 vs armv7), memory
> > > > constraints, and I was not using the same channels.conf, I'll fix that
> > > > today and re-check
> > >
> > > Earlier you said "random hangs are back". When this happens, does the whole
> > > device become unresponsive or just dvbv5-zap? Since this issue is "back",
> > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
> > >
> > > It would be useful to see the output from dmesg (best thing would be after
> > > the issue occurs).
> > >
> > > Also what dvb hardware are you using?
> > >
> > > Thanks,
> > >
> > > san
> > >
> > > >
> > > >
> > > > Sam
> > > >
> > > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote:
> > > >
> > > > Em Mon, 25 Mar 2019 17:33:30 +0100
> > > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
> > > >
> > > > Hi guys,
> > > >
> > > > I'm afraid I'm coming with sad news.
> > > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
> > > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
> > > > ).
> > > > I previously test both branches on a raspberry zero and issues were gone
> > > > (same raspbian version).
> > > > There may be more memory issues somewhere...
> > > >
> > > > Could you test it with valgrind?
> > > >
> > > > Sam
> > > >
> > > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :
> > > >
> > > > Hello,
> > > >
> > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote:
> > > >
> > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
> > > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,
> > > >
> > > > we can
> > > >
> > > > release a new minor version for all those stable branches.
> > > >
> > > > After the patches, on my tests, I didn't get any memory leaks or
> > > > double-free issues.
> > > >
> > > > I issues a new 1.12, 1.14, and 1.16 release.
> > > >
> > > > Thanks,
> > > > Gregor
> > > >
> > > >
> > > >
> > > > Thanks,
> > > > Mauro
>
>
>
> Thanks,
> Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Mauro Carvalho Chehab]

Em Thu, 28 Mar 2019 19:48:35 +0100
Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:

> Hi,

Please, don't top post.
> 
> 
> 1) I did reproduce the bug with a serial console, but the serial
> console was also frozen, and there was no message before freeze.

The only way for a machine to freeze even on serial console is due to
a very serious Kernel or hardware bug, like a bad lock/semaphore.

> 2) I also tried a powered USB HUB but it didn't change anything.

Ok.

> 3) I tried DVBJet, from https://github.com/lightful/DVBdirect , it is
> a tuner that directly does ioctl on linux kernel DVB device.
> I couldn't reproduce the issue with DVBJet.

I don't know DVBJet. If it uses the same Kernel drivers, maybe
it could then be a race issue: send commands on a slower way to
the Kernel would be solving the issue.

> It's been running for almost 48Hrs now, without a crash.
> I collect all PIDs on the multiplex, just like dvbv5-zap, so the USB
> port is solicited at it's maximum.
> 
> That makes me think it is not a hardware issue.
> I'm pretty sure dvbv5-zap can still improve, and it already did when
> you fixed: struct arguments args = {};

Everything can be improved, but the thing is that we need first to
discover the root cause :-)

Can you post on pastebin (or equivalent) the dmesg with the
DVBJet running? Maybe it could be printing some Kernel messages
that might help to discover what's going wrong.

Btw, could you also apply this patch:
	https://patchwork.linuxtv.org/patch/55274/

It is probably unrelated, but this is the kind of bug that could cause
such issues.

> 
> 
> Regards
> 
> Le mar. 26 mars 2019 à 16:31, Mauro Carvalho Chehab
> <mchehab+samsung@kernel.org> a écrit :
> >
> > Em Tue, 26 Mar 2019 16:10:33 +0100
> > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
> >  
> > > Hi,
> > >
> > >  
> > > > Earlier you said "random hangs are back". When this happens, does the whole
> > > > device become unresponsive or just dvbv5-zap?  
> > > The device completely freeze, you can't even switch numlock on/off.  
> >
> > dvbv5-tools can't hang the machine. this is very likely happening due to
> > a Kernel crash.
> >  
> > > I said "the issue is back", it is because I **thought** it was gone.
> > > To be more precise:
> > > - on raspberry zero W, the issue is gone since Mauro's patch
> > > (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89)
> > > - on raspberry 2, the issue, it is still there and the patch has no
> > > effect (the issue was and is still there)  
> >
> > RPi2 has a serious issue with their USB ports: on devices that require
> > more than a few mW to work, it causes several device disconnection and
> > re-connection, as it cannot sustain the required 5V.
> >
> > Depending on how fast this happens, it could be triggering some Kernel
> > bug.
> >
> > That affects most V4L and DVB devices. You should either use a powered
> > USB 2.0 hub (with may be problematic, as the USB host driver on RPi
> > has issues - and may cause data loss on high sustained ISOC traffic,
> > specially when used with hubs) or a device that has its own power
> > supply, like DVBSky T680C or S960.
> >  
> > > > Since this issue is "back",
> > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.  
> > > The issue doesn't appear anymore on raspberry zero W since Mauro's commit.
> > > So it did improve on that platform.
> > >  
> > > > It would be useful to see the output from dmesg (best thing would be after
> > > > the issue occurs).  
> > > You can't, device is frozen.
> > > Logs are not flushed to disk, and journalctl -f freeze before showing anything  
> >
> > You can use a serial port in order to get the logs. On a serial console,
> > use something like:
> >
> >         # dmesg -n 8
> >
> > In order to make sure it will display all Kernel messages at console.
> >  
> > >  
> > > > Also what dvb hardware are you using?  
> > > I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM
> > > and a TerraTec Cinergy T Stick+  
> >
> > None of them supports an external power supply.
> >  
> > > You can found all the details here:
> > > https://bugs.launchpad.net/raspbian/+bug/1819650
> > >
> > >
> > > Sam
> > >
> > >
> > > Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit :  
> > > >
> > > > Hi Sam,
> > > >
> > > > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote:  
> > > > > Hi,
> > > > >
> > > > >
> > > > > I am struggling with valgrind because it always complain with either :
> > > > >     ASan runtime does not come first in initial library list; you
> > > > > should either link runtime to your application or manually preload it
> > > > > with LD_PRELOAD  
> > > > >     -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find  
> > > > > any core dump
> > > > >
> > > > > or, if I link statically libasan with -static-libasan:
> > > > >     Shadow memory range interleaves with an existing memory mapping.
> > > > > ASan cannot proceed correctly. ABORTING.
> > > > >     ASan shadow was supposed to be located in the
> > > > > [0x00007fff7000-0x10007fff7fff] range.
> > > > >
> > > > >
> > > > > I retested again on my raspberry zero W, and I confirm i cannot
> > > > > reproduce the hang.
> > > > > Your fix did work on that device.
> > > > > I am testing with same OS (raspbian with latest updates, same kernel),
> > > > > same configure options, same USB dongle... :-(
> > > > > The only differences are CPU architecture (armv6 vs armv7), memory
> > > > > constraints, and I was not using the same channels.conf, I'll fix that
> > > > > today and re-check  
> > > >
> > > > Earlier you said "random hangs are back". When this happens, does the whole
> > > > device become unresponsive or just dvbv5-zap? Since this issue is "back",
> > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
> > > >
> > > > It would be useful to see the output from dmesg (best thing would be after
> > > > the issue occurs).
> > > >
> > > > Also what dvb hardware are you using?
> > > >
> > > > Thanks,
> > > >
> > > > san
> > > >  
> > > > >
> > > > >
> > > > > Sam
> > > > >
> > > > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote:
> > > > >
> > > > > Em Mon, 25 Mar 2019 17:33:30 +0100
> > > > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
> > > > >
> > > > > Hi guys,
> > > > >
> > > > > I'm afraid I'm coming with sad news.
> > > > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
> > > > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
> > > > > ).
> > > > > I previously test both branches on a raspberry zero and issues were gone
> > > > > (same raspbian version).
> > > > > There may be more memory issues somewhere...
> > > > >
> > > > > Could you test it with valgrind?
> > > > >
> > > > > Sam
> > > > >
> > > > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :
> > > > >
> > > > > Hello,
> > > > >
> > > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote:
> > > > >
> > > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
> > > > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,
> > > > >
> > > > > we can
> > > > >
> > > > > release a new minor version for all those stable branches.
> > > > >
> > > > > After the patches, on my tests, I didn't get any memory leaks or
> > > > > double-free issues.
> > > > >
> > > > > I issues a new 1.12, 1.14, and 1.16 release.
> > > > >
> > > > > Thanks,
> > > > > Gregor
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > > Mauro  
> >
> >
> >
> > Thanks,
> > Mauro  



Thanks,
Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Gregor Jasny]

Hello,

On 28.03.19 20:17, Mauro Carvalho Chehab wrote:
> Em Thu, 28 Mar 2019 19:48:35 +0100
> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
>> 1) I did reproduce the bug with a serial console, but the serial
>> console was also frozen, and there was no message before freeze.
> 
> The only way for a machine to freeze even on serial console is due to
> a very serious Kernel or hardware bug, like a bad lock/semaphore.

Does anything speak against upgrading Debian to latest 1.16.5 release?

Thanks,
Gregor

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Mauro Carvalho Chehab]

Em Thu, 28 Mar 2019 20:48:34 +0100
Gregor Jasny <gjasny@googlemail.com> escreveu:

> Hello,
> 
> On 28.03.19 20:17, Mauro Carvalho Chehab wrote:
> > Em Thu, 28 Mar 2019 19:48:35 +0100
> > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:  
> >> 1) I did reproduce the bug with a serial console, but the serial
> >> console was also frozen, and there was no message before freeze.  
> > 
> > The only way for a machine to freeze even on serial console is due to
> > a very serious Kernel or hardware bug, like a bad lock/semaphore.  
> 
> Does anything speak against upgrading Debian to latest 1.16.5 release?

I don't think so. The changes we did were just at memory allocation/free
with standard glibc malloc()/calloc()/free() functions.

The kind of errors Samuel are experiencing are machine freezes. 
dvbv5-utils can't cause such kind of errors, as it doesn't run in 
privileged mode, nor do anything potentially harmful.

So, even the most serious issue there won't cause a machine crash.

For a machine to crash, it has to be triggering a Kernel bug
or some memory corruption outside its memory space (e.g. triggering
some glibc bug - with also seems very unlikely).

Thanks,
Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Samuel CHEMLA]

Le jeu. 28 mars 2019 à 20:18, Mauro Carvalho Chehab
<mchehab+samsung@kernel.org> a écrit :
>
> Em Thu, 28 Mar 2019 19:48:35 +0100
> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
>
> > Hi,
>
> Please, don't top post.
> >
> >
> > 1) I did reproduce the bug with a serial console, but the serial
> > console was also frozen, and there was no message before freeze.
>
> The only way for a machine to freeze even on serial console is due to
> a very serious Kernel or hardware bug, like a bad lock/semaphore.
>
> > 2) I also tried a powered USB HUB but it didn't change anything.
>
> Ok.
>
> > 3) I tried DVBJet, from https://github.com/lightful/DVBdirect , it is
> > a tuner that directly does ioctl on linux kernel DVB device.
> > I couldn't reproduce the issue with DVBJet.
>
> I don't know DVBJet. If it uses the same Kernel drivers, maybe
> it could then be a race issue: send commands on a slower way to
> the Kernel would be solving the issue.
>
> > It's been running for almost 48Hrs now, without a crash.
> > I collect all PIDs on the multiplex, just like dvbv5-zap, so the USB
> > port is solicited at it's maximum.
> >
> > That makes me think it is not a hardware issue.
> > I'm pretty sure dvbv5-zap can still improve, and it already did when
> > you fixed: struct arguments args = {};
>
> Everything can be improved, but the thing is that we need first to
> discover the root cause :-)
>
> Can you post on pastebin (or equivalent) the dmesg with the
> DVBJet running? Maybe it could be printing some Kernel messages
> that might help to discover what's going wrong.

Here is the dmesg: https://pastebin.com/3XRim4XL
Just look at the code, it's quite straightforward, just a few ioctl.

>
> Btw, could you also apply this patch:
>         https://patchwork.linuxtv.org/patch/55274/

OK, I'll give it a try and provide feedback

>
> It is probably unrelated, but this is the kind of bug that could cause
> such issues.
>
> >
> >
> > Regards
> >
> > Le mar. 26 mars 2019 à 16:31, Mauro Carvalho Chehab
> > <mchehab+samsung@kernel.org> a écrit :
> > >
> > > Em Tue, 26 Mar 2019 16:10:33 +0100
> > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
> > >
> > > > Hi,
> > > >
> > > >
> > > > > Earlier you said "random hangs are back". When this happens, does the whole
> > > > > device become unresponsive or just dvbv5-zap?
> > > > The device completely freeze, you can't even switch numlock on/off.
> > >
> > > dvbv5-tools can't hang the machine. this is very likely happening due to
> > > a Kernel crash.
> > >
> > > > I said "the issue is back", it is because I **thought** it was gone.
> > > > To be more precise:
> > > > - on raspberry zero W, the issue is gone since Mauro's patch
> > > > (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89)
> > > > - on raspberry 2, the issue, it is still there and the patch has no
> > > > effect (the issue was and is still there)
> > >
> > > RPi2 has a serious issue with their USB ports: on devices that require
> > > more than a few mW to work, it causes several device disconnection and
> > > re-connection, as it cannot sustain the required 5V.
> > >
> > > Depending on how fast this happens, it could be triggering some Kernel
> > > bug.
> > >
> > > That affects most V4L and DVB devices. You should either use a powered
> > > USB 2.0 hub (with may be problematic, as the USB host driver on RPi
> > > has issues - and may cause data loss on high sustained ISOC traffic,
> > > specially when used with hubs) or a device that has its own power
> > > supply, like DVBSky T680C or S960.
> > >
> > > > > Since this issue is "back",
> > > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
> > > > The issue doesn't appear anymore on raspberry zero W since Mauro's commit.
> > > > So it did improve on that platform.
> > > >
> > > > > It would be useful to see the output from dmesg (best thing would be after
> > > > > the issue occurs).
> > > > You can't, device is frozen.
> > > > Logs are not flushed to disk, and journalctl -f freeze before showing anything
> > >
> > > You can use a serial port in order to get the logs. On a serial console,
> > > use something like:
> > >
> > >         # dmesg -n 8
> > >
> > > In order to make sure it will display all Kernel messages at console.
> > >
> > > >
> > > > > Also what dvb hardware are you using?
> > > > I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM
> > > > and a TerraTec Cinergy T Stick+
> > >
> > > None of them supports an external power supply.
> > >
> > > > You can found all the details here:
> > > > https://bugs.launchpad.net/raspbian/+bug/1819650
> > > >
> > > >
> > > > Sam
> > > >
> > > >
> > > > Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit :
> > > > >
> > > > > Hi Sam,
> > > > >
> > > > > On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote:
> > > > > > Hi,
> > > > > >
> > > > > >
> > > > > > I am struggling with valgrind because it always complain with either :
> > > > > >     ASan runtime does not come first in initial library list; you
> > > > > > should either link runtime to your application or manually preload it
> > > > > > with LD_PRELOAD
> > > > > >     -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find
> > > > > > any core dump
> > > > > >
> > > > > > or, if I link statically libasan with -static-libasan:
> > > > > >     Shadow memory range interleaves with an existing memory mapping.
> > > > > > ASan cannot proceed correctly. ABORTING.
> > > > > >     ASan shadow was supposed to be located in the
> > > > > > [0x00007fff7000-0x10007fff7fff] range.
> > > > > >
> > > > > >
> > > > > > I retested again on my raspberry zero W, and I confirm i cannot
> > > > > > reproduce the hang.
> > > > > > Your fix did work on that device.
> > > > > > I am testing with same OS (raspbian with latest updates, same kernel),
> > > > > > same configure options, same USB dongle... :-(
> > > > > > The only differences are CPU architecture (armv6 vs armv7), memory
> > > > > > constraints, and I was not using the same channels.conf, I'll fix that
> > > > > > today and re-check
> > > > >
> > > > > Earlier you said "random hangs are back". When this happens, does the whole
> > > > > device become unresponsive or just dvbv5-zap? Since this issue is "back",
> > > > > I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
> > > > >
> > > > > It would be useful to see the output from dmesg (best thing would be after
> > > > > the issue occurs).
> > > > >
> > > > > Also what dvb hardware are you using?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > san
> > > > >
> > > > > >
> > > > > >
> > > > > > Sam
> > > > > >
> > > > > > On 25/03/2019 18:08, Mauro Carvalho Chehab wrote:
> > > > > >
> > > > > > Em Mon, 25 Mar 2019 17:33:30 +0100
> > > > > > Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
> > > > > >
> > > > > > Hi guys,
> > > > > >
> > > > > > I'm afraid I'm coming with sad news.
> > > > > > I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
> > > > > > random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
> > > > > > ).
> > > > > > I previously test both branches on a raspberry zero and issues were gone
> > > > > > (same raspbian version).
> > > > > > There may be more memory issues somewhere...
> > > > > >
> > > > > > Could you test it with valgrind?
> > > > > >
> > > > > > Sam
> > > > > >
> > > > > > Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > On 21.03.19 12:30, Mauro Carvalho Chehab wrote:
> > > > > >
> > > > > > I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
> > > > > > -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,
> > > > > >
> > > > > > we can
> > > > > >
> > > > > > release a new minor version for all those stable branches.
> > > > > >
> > > > > > After the patches, on my tests, I didn't get any memory leaks or
> > > > > > double-free issues.
> > > > > >
> > > > > > I issues a new 1.12, 1.14, and 1.16 release.
> > > > > >
> > > > > > Thanks,
> > > > > > Gregor
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > > Mauro
> > >
> > >
> > >
> > > Thanks,
> > > Mauro
>
>
>
> Thanks,
> Mauro

Re: [Bug report] dvbv5-zap crash dvb-tool ARMHF builds

[Samuel CHEMLA]

On 01/04/2019 20:58, Samuel CHEMLA wrote:
> Le jeu. 28 mars 2019 à 20:18, Mauro Carvalho Chehab
> <mchehab+samsung@kernel.org> a écrit :
>> Em Thu, 28 Mar 2019 19:48:35 +0100
>> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
>>
>>> Hi,
>> Please, don't top post.
>>>
>>> 1) I did reproduce the bug with a serial console, but the serial
>>> console was also frozen, and there was no message before freeze.
>> The only way for a machine to freeze even on serial console is due to
>> a very serious Kernel or hardware bug, like a bad lock/semaphore.
>>
>>> 2) I also tried a powered USB HUB but it didn't change anything.
>> Ok.
>>
>>> 3) I tried DVBJet, from https://github.com/lightful/DVBdirect , it is
>>> a tuner that directly does ioctl on linux kernel DVB device.
>>> I couldn't reproduce the issue with DVBJet.
>> I don't know DVBJet. If it uses the same Kernel drivers, maybe
>> it could then be a race issue: send commands on a slower way to
>> the Kernel would be solving the issue.
>>
>>> It's been running for almost 48Hrs now, without a crash.
>>> I collect all PIDs on the multiplex, just like dvbv5-zap, so the USB
>>> port is solicited at it's maximum.
>>>
>>> That makes me think it is not a hardware issue.
>>> I'm pretty sure dvbv5-zap can still improve, and it already did when
>>> you fixed: struct arguments args = {};
>> Everything can be improved, but the thing is that we need first to
>> discover the root cause :-)
>>
>> Can you post on pastebin (or equivalent) the dmesg with the
>> DVBJet running? Maybe it could be printing some Kernel messages
>> that might help to discover what's going wrong.
> Here is the dmesg: https://pastebin.com/3XRim4XL
> Just look at the code, it's quite straightforward, just a few ioctl.
>
>> Btw, could you also apply this patch:
>>          https://patchwork.linuxtv.org/patch/55274/
> OK, I'll give it a try and provide feedback

I tested you patch against 4.14.98 which is the current raspbian kernel, 
but it didn't change anything...

>
>> It is probably unrelated, but this is the kind of bug that could cause
>> such issues.
>>
>>>
>>> Regards
>>>
>>> Le mar. 26 mars 2019 à 16:31, Mauro Carvalho Chehab
>>> <mchehab+samsung@kernel.org> a écrit :
>>>> Em Tue, 26 Mar 2019 16:10:33 +0100
>>>> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
>>>>
>>>>> Hi,
>>>>>
>>>>>
>>>>>> Earlier you said "random hangs are back". When this happens, does the whole
>>>>>> device become unresponsive or just dvbv5-zap?
>>>>> The device completely freeze, you can't even switch numlock on/off.
>>>> dvbv5-tools can't hang the machine. this is very likely happening due to
>>>> a Kernel crash.
>>>>
>>>>> I said "the issue is back", it is because I **thought** it was gone.
>>>>> To be more precise:
>>>>> - on raspberry zero W, the issue is gone since Mauro's patch
>>>>> (https://git.linuxtv.org/v4l-utils.git/commit/?id=22b06353227e04695b1b0a9622b896b948adba89)
>>>>> - on raspberry 2, the issue, it is still there and the patch has no
>>>>> effect (the issue was and is still there)
>>>> RPi2 has a serious issue with their USB ports: on devices that require
>>>> more than a few mW to work, it causes several device disconnection and
>>>> re-connection, as it cannot sustain the required 5V.
>>>>
>>>> Depending on how fast this happens, it could be triggering some Kernel
>>>> bug.
>>>>
>>>> That affects most V4L and DVB devices. You should either use a powered
>>>> USB 2.0 hub (with may be problematic, as the USB host driver on RPi
>>>> has issues - and may cause data loss on high sustained ISOC traffic,
>>>> specially when used with hubs) or a device that has its own power
>>>> supply, like DVBSky T680C or S960.
>>>>
>>>>>> Since this issue is "back",
>>>>>> I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
>>>>> The issue doesn't appear anymore on raspberry zero W since Mauro's commit.
>>>>> So it did improve on that platform.
>>>>>
>>>>>> It would be useful to see the output from dmesg (best thing would be after
>>>>>> the issue occurs).
>>>>> You can't, device is frozen.
>>>>> Logs are not flushed to disk, and journalctl -f freeze before showing anything
>>>> You can use a serial port in order to get the logs. On a serial console,
>>>> use something like:
>>>>
>>>>          # dmesg -n 8
>>>>
>>>> In order to make sure it will display all Kernel messages at console.
>>>>
>>>>>> Also what dvb hardware are you using?
>>>>> I reproduced it with different two tuners: rtl2832U from RTL-SDR.COM
>>>>> and a TerraTec Cinergy T Stick+
>>>> None of them supports an external power supply.
>>>>
>>>>> You can found all the details here:
>>>>> https://bugs.launchpad.net/raspbian/+bug/1819650
>>>>>
>>>>>
>>>>> Sam
>>>>>
>>>>>
>>>>> Le mar. 26 mars 2019 à 14:26, Sean Young <sean@mess.org> a écrit :
>>>>>> Hi Sam,
>>>>>>
>>>>>> On Tue, Mar 26, 2019 at 08:35:44AM +0100, Samuel CHEMLA wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>>
>>>>>>> I am struggling with valgrind because it always complain with either :
>>>>>>>      ASan runtime does not come first in initial library list; you
>>>>>>> should either link runtime to your application or manually preload it
>>>>>>> with LD_PRELOAD
>>>>>>>      -> When I LD_PRELOAD, I'm getting a segfault, but I couldn't find
>>>>>>> any core dump
>>>>>>>
>>>>>>> or, if I link statically libasan with -static-libasan:
>>>>>>>      Shadow memory range interleaves with an existing memory mapping.
>>>>>>> ASan cannot proceed correctly. ABORTING.
>>>>>>>      ASan shadow was supposed to be located in the
>>>>>>> [0x00007fff7000-0x10007fff7fff] range.
>>>>>>>
>>>>>>>
>>>>>>> I retested again on my raspberry zero W, and I confirm i cannot
>>>>>>> reproduce the hang.
>>>>>>> Your fix did work on that device.
>>>>>>> I am testing with same OS (raspbian with latest updates, same kernel),
>>>>>>> same configure options, same USB dongle... :-(
>>>>>>> The only differences are CPU architecture (armv6 vs armv7), memory
>>>>>>> constraints, and I was not using the same channels.conf, I'll fix that
>>>>>>> today and re-check
>>>>>> Earlier you said "random hangs are back". When this happens, does the whole
>>>>>> device become unresponsive or just dvbv5-zap? Since this issue is "back",
>>>>>> I wouldn't be surprised this is unrelated to the fixes in 1.12.7 and 1.16.4.
>>>>>>
>>>>>> It would be useful to see the output from dmesg (best thing would be after
>>>>>> the issue occurs).
>>>>>>
>>>>>> Also what dvb hardware are you using?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> san
>>>>>>
>>>>>>>
>>>>>>> Sam
>>>>>>>
>>>>>>> On 25/03/2019 18:08, Mauro Carvalho Chehab wrote:
>>>>>>>
>>>>>>> Em Mon, 25 Mar 2019 17:33:30 +0100
>>>>>>> Samuel CHEMLA <chemla.samuel@gmail.com> escreveu:
>>>>>>>
>>>>>>> Hi guys,
>>>>>>>
>>>>>>> I'm afraid I'm coming with sad news.
>>>>>>> I just tried both stable-1.12 and stable-1.16 on a raspberry pi 2, and
>>>>>>> random hangs are back (see https://bugs.launchpad.net/raspbian/+bug/1819650
>>>>>>> ).
>>>>>>> I previously test both branches on a raspberry zero and issues were gone
>>>>>>> (same raspbian version).
>>>>>>> There may be more memory issues somewhere...
>>>>>>>
>>>>>>> Could you test it with valgrind?
>>>>>>>
>>>>>>> Sam
>>>>>>>
>>>>>>> Le jeu. 21 mars 2019 ŕ 20:59, Gregor Jasny <gjasny@googlemail.com> a écrit :
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> On 21.03.19 12:30, Mauro Carvalho Chehab wrote:
>>>>>>>
>>>>>>> I went ahead and cherry-picked the relevant patches to -1.12, -1.14 and
>>>>>>> -1.16, and tested both dvbv5-zap and dvbv5-scan with all versions. So,
>>>>>>>
>>>>>>> we can
>>>>>>>
>>>>>>> release a new minor version for all those stable branches.
>>>>>>>
>>>>>>> After the patches, on my tests, I didn't get any memory leaks or
>>>>>>> double-free issues.
>>>>>>>
>>>>>>> I issues a new 1.12, 1.14, and 1.16 release.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Gregor
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Mauro
>>>>
>>>>
>>>> Thanks,
>>>> Mauro
>>
>>
>> Thanks,
>> Mauro