poor performance of OpenEmbedded on BeagleBoneBlack compared to Debian

poor performance of OpenEmbedded on BeagleBoneBlack compared to Debian

[Peter A. Bigot]

I've been trying to figure out why OpenEmbedded on beaglebone is three 
times slower than Debian on the ray-tracing application described at:

http://scruss.com/blog/2013/09/23/beaglebone-black-slow-as-a-dog/

Part of it is due to meta-yocto-bsp's beaglebone config not enabling CPU 
frequency control (CONFIG_GENERIC_CPUFREQ_CPU0, 
CONFIG_VOLTAGE_DOMAIN_OMAP, and some THERMAL options).  That can be 
fixed for linux-yocto, and isn't an issue with linux-ti-staging, but it 
doesn't eliminate the discrepancies.

With debian, the sample program described in that blog posting runs this 
fast:

   debian@beaglebone:~$ g++ -Ofast -mfloat-abi=hard -o card card.cpp
   debian@beaglebone:~$ time ./card > /dev/null
   real    4m54.846s
   user    4m54.627s
   sys     0m0.057s

With poky master + meta-ti it runs this fast:

   root@beaglebone:~# time ./card > /dev/null
   real    9m37.901s
   user    9m37.200s
   sys     0m0.150s

Still nearly twice as slow.  (With linux-yocto and the default kernel 
configuration it's up around 14m.)

One anomaly I've found is the CPU frequency range.  On debian we have:

   debian@beaglebone:~$ cat 
/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
   300000 600000 800000 1000000

while on OE we have:

   root@beaglebone:~# cat 
/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
   300000 600000 720000 800000

So OE isn't allowing the BeagleBoneBlack to run at its maximum speed.  
This isn't enough to explain a nearly 2x performance difference, but 
it's an issue I want resolved.

I've been unable to trace back where these frequencies come from. Could 
somebody tell me what configures the available frequencies for the 
AM335x in linux-ti-staging?

And if anybody has suggestions of where else to look for an explanation, 
I'd appreciate it.  I've tried all the gcc flags I can think of 
(-mcpu=cortex-a8, -mfpu=neon, -mfloat-abi=hard, etc), though I'm going 
to keep poking at it because I still suspect the problem is there.

Thanks.

Peter

Re: poor performance of OpenEmbedded on BeagleBoneBlack compared to Debian

[Robert Nelson]

On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
> I've been trying to figure out why OpenEmbedded on beaglebone is three times
> slower than Debian on the ray-tracing application described at:
>
> http://scruss.com/blog/2013/09/23/beaglebone-black-slow-as-a-dog/
>
> Part of it is due to meta-yocto-bsp's beaglebone config not enabling CPU
> frequency control (CONFIG_GENERIC_CPUFREQ_CPU0, CONFIG_VOLTAGE_DOMAIN_OMAP,
> and some THERMAL options).  That can be fixed for linux-yocto, and isn't an
> issue with linux-ti-staging, but it doesn't eliminate the discrepancies.
>
> With debian, the sample program described in that blog posting runs this
> fast:
>
>   debian@beaglebone:~$ g++ -Ofast -mfloat-abi=hard -o card card.cpp
>   debian@beaglebone:~$ time ./card > /dev/null
>   real    4m54.846s
>   user    4m54.627s
>   sys     0m0.057s
>
> With poky master + meta-ti it runs this fast:
>
>   root@beaglebone:~# time ./card > /dev/null
>   real    9m37.901s
>   user    9m37.200s
>   sys     0m0.150s

This seems very odd, what version of g++ do you have on poky?

On debian wheezy, we have the old:

gcc version 4.6.3 (Debian 4.6.3-14)

Which really doesn't have any of the linaro special sauce..

Regards,

-- 
Robert Nelson
http://www.rcn-ee.com/

Re: poor performance of OpenEmbedded on BeagleBoneBlack compared to Debian

[Khem Raj]

On Wed, Sep 3, 2014 at 6:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
> debian@beaglebone:~$ g++ -Ofast -mfloat-abi=hard -o card card.cpp
>   debian@beaglebone:~$ time ./card > /dev/null
>   real    4m54.846s
>   user    4m54.627s
>   sys     0m0.057s
>
> With poky master + meta-ti it runs this fast:
>
>   root@beaglebone:~# time ./card > /dev/null
>   real    9m37.901s
>   user    9m37.200s
>   sys     0m0.150s


you should enable hardfloat ABI for poky as well for this kind of load.

Re: poor performance of OpenEmbedded on BeagleBoneBlack compared to Debian

[Peter A. Bigot]

On 09/03/2014 09:46 PM, Robert Nelson wrote:
> On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>> I've been trying to figure out why OpenEmbedded on beaglebone is three times
>> slower than Debian on the ray-tracing application described at:
>>
>> http://scruss.com/blog/2013/09/23/beaglebone-black-slow-as-a-dog/
>>
>> Part of it is due to meta-yocto-bsp's beaglebone config not enabling CPU
>> frequency control (CONFIG_GENERIC_CPUFREQ_CPU0, CONFIG_VOLTAGE_DOMAIN_OMAP,
>> and some THERMAL options).  That can be fixed for linux-yocto, and isn't an
>> issue with linux-ti-staging, but it doesn't eliminate the discrepancies.
>>
>> With debian, the sample program described in that blog posting runs this
>> fast:
>>
>>    debian@beaglebone:~$ g++ -Ofast -mfloat-abi=hard -o card card.cpp
>>    debian@beaglebone:~$ time ./card > /dev/null
>>    real    4m54.846s
>>    user    4m54.627s
>>    sys     0m0.057s
>>
>> With poky master + meta-ti it runs this fast:
>>
>>    root@beaglebone:~# time ./card > /dev/null
>>    real    9m37.901s
>>    user    9m37.200s
>>    sys     0m0.150s
> This seems very odd, what version of g++ do you have on poky?

4.9.1, also nothing special.

Peter

>
> On debian wheezy, we have the old:
>
> gcc version 4.6.3 (Debian 4.6.3-14)
>
> Which really doesn't have any of the linaro special sauce..
>
> Regards,
>

Re: poor performance of OpenEmbedded on BeagleBoneBlack compared to Debian

[Peter A. Bigot]

On 09/04/2014 12:39 AM, Khem Raj wrote:
> On Wed, Sep 3, 2014 at 6:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>> debian@beaglebone:~$ g++ -Ofast -mfloat-abi=hard -o card card.cpp
>>    debian@beaglebone:~$ time ./card > /dev/null
>>    real    4m54.846s
>>    user    4m54.627s
>>    sys     0m0.057s
>>
>> With poky master + meta-ti it runs this fast:
>>
>>    root@beaglebone:~# time ./card > /dev/null
>>    real    9m37.901s
>>    user    9m37.200s
>>    sys     0m0.150s
>
> you should enable hardfloat ABI for poky as well for this kind of load.

Yeah, I remember that from the big argument back in oe-core, but what's 
lacking in all that is any revelation of exactly how to do that.

The poky build using meta-ti's BSP for beaglebone puts gcc and 
gcc-runtime into cortexa8t2hf-vfp-neon-poky-linux-gnueabi which AFAICT 
is supposed to be hard-float.

-mfloat-abi=hard is being passed when gcc-runtime is built, and when the 
application is built.

What toolchain-build or runtime-build enables hardfloat if that isn't 
already being done?

Peter

Re: poor performance of OpenEmbedded on BeagleBoneBlack compared to Debian

[Dmytriyenko, Denys]

[-- Attachment #1: Type: text/plain, Size: 1661 bytes --]

You should be fine, as meta-ti machines now force hardfp...


Sent from my Android phone using TouchDown (www.nitrodesk.com)

-----Original Message-----
From: Peter A. Bigot [pab@pabigot.com]
Received: Thursday, 04 Sep 2014, 6:56
To: Khem Raj [raj.khem@gmail.com]
CC: meta-ti@yoctoproject.org [meta-ti@yoctoproject.org]
Subject: Re: [meta-ti] poor performance of OpenEmbedded on BeagleBoneBlack compared to Debian

On 09/04/2014 12:39 AM, Khem Raj wrote:
> On Wed, Sep 3, 2014 at 6:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>> debian@beaglebone:~$ g++ -Ofast -mfloat-abi=hard -o card card.cpp
>>    debian@beaglebone:~$ time ./card > /dev/null
>>    real    4m54.846s
>>    user    4m54.627s
>>    sys     0m0.057s
>>
>> With poky master + meta-ti it runs this fast:
>>
>>    root@beaglebone:~# time ./card > /dev/null
>>    real    9m37.901s
>>    user    9m37.200s
>>    sys     0m0.150s
>
> you should enable hardfloat ABI for poky as well for this kind of load.

Yeah, I remember that from the big argument back in oe-core, but what's
lacking in all that is any revelation of exactly how to do that.

The poky build using meta-ti's BSP for beaglebone puts gcc and
gcc-runtime into cortexa8t2hf-vfp-neon-poky-linux-gnueabi which AFAICT
is supposed to be hard-float.

-mfloat-abi=hard is being passed when gcc-runtime is built, and when the
application is built.

What toolchain-build or runtime-build enables hardfloat if that isn't
already being done?

Peter
--
_______________________________________________
meta-ti mailing list
meta-ti@yoctoproject.org
https://lists.yoctoproject.org/listinfo/meta-ti

[-- Attachment #2: Type: text/html, Size: 2927 bytes --]

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Khem Raj]

On Thu, Sep 4, 2014 at 8:55 AM, Peter A. Bigot <pab@pabigot.com> wrote:
> On 09/04/2014 10:48 AM, Denys Dmytriyenko wrote:
>>
>> On Thu, Sep 04, 2014 at 10:37:24AM -0500, Peter A. Bigot wrote:
>>>
>>> On 09/04/2014 10:00 AM, Denys Dmytriyenko wrote:
>>>>
>>>> On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
>>>>>
>>>>> On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>>>>>
>>>>>> One anomaly I've found is the CPU frequency range.  On debian we have:
>>>>>>
>>>>>>     debian@beaglebone:~$ cat
>>>>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>>>>     300000 600000 800000 1000000
>>>>>>
>>>>>> while on OE we have:
>>>>>>
>>>>>>     root@beaglebone:~# cat
>>>>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>>>>     300000 600000 720000 800000
>>>>>
>>>>> Stock yocto-bsp is missing a few things that can be found in Robert
>>>>> Nelson's patchset for 3.14 linux kernel.
>>>>>
>>>>> There are lots of other functionality that is missing from yocto-bsp
>>>>> kernel for Beaglebone. I suggest you take at the following repos and
>>>>> scavenge for what you need :P
>>>>
>>>> First of all, this is meta-ti mailing list for the corresponding BSP.
>>>> That's
>>>> what Peter was asking for, comparing to Robert's Debian and Yocto
>>>> reference
>>>> BSPs, not the other way around.
>>>>
>>>> Second, Yocto reference BSP is that way for a reason - it's a reference
>>>> BSP
>>>> done with pure mainline kernel and u-boot components w/o any patching on
>>>> top.
>>>> That's its entire purpose. For anything else special, including
>>>> performance
>>>> tweaks, there are other BSPs available. If there is an issue with
>>>> performance
>>>> in meta-ti, we'll investigate it and try to match with Robert's BSP.
>>>
>>> Yes, at this time meta-ti's BSP performs as well as I've seen any
>>> OE-based system, and gets several things right that meta-yocto-bsp
>>> does not (and one thing wrong that meta-yocto-bsp gets right, I
>>> think; still investigating, will follow-up when I'm sure).
>>>
>>> I've also verified that performance with a native gcc 4.9.1 build on
>>> BeagleBone with hard float is poor, so it's not due to the way OE
>>> builds gcc.  I have several competing hypotheses to test.
>>
>> Can you please point to the test case you were using to measure time? I'd
>> like
>> to try it with few different toolchains here as well. BTW, we are
>> currently
>> using Linaro gcc-4.7.3 - I'm wondering how it performs.
>
>
>
> The context is:
> http://scruss.com/blog/2013/09/23/beaglebone-black-slow-as-a-dog/
>
> The specific application is: http://www.cs.utah.edu/~aek/code/card.cpp
>
> The "ought to work" compiler line is: g++ -Ofast card.cpp -o card
>
> The "kitchen-sink-still-doesn't-help" compiler line is:
>
> g++ \
>   -march=armv7-a -mtune=cortex-a8 \
>   -mthumb-interwork \
>   -mfloat-abi=hard -mfpu=neon \
>   -Ofast \
>   -o card \
>   card.cpp
>
> All my tests are with gcc-4.9.1, with or without the openembedded-core patch
> set.
>

can you profile run it.? As i understand, you have exact same hardware
and same way of running
it with two different rootfs.

>
>>
>>> But I'm still looking for a way to set the CPU frequency to the
>>> higher values supported on Beaglebone Black.  I had hoped meta-ti's
>>> would be able to do that, since it has bone vs boneblack device
>>> trees and u-boot detection.
>>>
>>> Any hints where to look for clock settings?
>>
>> Some of those values are considered overclocking and according to the
>> manual
>> will reduce the lifespan of the part...
>
>
> So the Beagle Bone Black SRM's claim that the AM3358BZCZ100 can run at 1GHz
> is not entirely true?  Or is it just that the operating-points in that patch
> are too aggressive?
>
> Peter
>
> --
> _______________________________________________
> meta-ti mailing list
> meta-ti@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/meta-ti

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Denys Dmytriyenko]

On Thu, Sep 04, 2014 at 02:09:02PM -0500, Robert Nelson wrote:
> On Thu, Sep 4, 2014 at 2:07 PM, Robert Nelson <robertcnelson@gmail.com> wrote:
> >> Indeed, the A5C has a 962A chip which is Si 2.0, while the B has a 962B chip
> >> that is Si 2.1, per SPRZ360F.
> >
> > Found my "A4" don't think anything earlier went out
> >
> > 962: 1.0
> 
> This one was obviously binned, it's still running 1Gh, almost 2 years old..

I have A1 somewhere around here, but I don't think that was available outside 
of TI. It has an upside-down mSD card slot... I might be able to check if it 
does 1GHz. :)

-- 
Denys

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Denys Dmytriyenko]

On Thu, Sep 04, 2014 at 02:03:07PM -0500, Robert Nelson wrote:
> > Robert, Peter,
> >
> > So, after some internal discussion, here's an update:
> >
> > 1. In TI kernel we do dynamic opp setting based on eFuse check in the file
> > arch/arm/mach-omap2/opp33xx_data.c
> >
> > 2. Since most of our EVMs and EVM-SKs use 2.1 Si, everything works properly
> > with eFuse check and dynamic opp setting.
> >
> > 3. But eFuse was not available in 2.0 rev of silicon, only in 2.1 rev. There
> > were many BBB boards made with 2.0 Si that were binned to run at 1GHz. And
> > older 1.0 Si revs were not 1GHz capable. Currently checking with HW and
> > marketing teams if all/most 2.0 chips are safe to run at 1GHz... As w/o eFuse
> > there's no way to determine if the chip is 1GHz capable or not.
> >
> > 4. If your BBB board has 2.1 Si rev, then you should get 1GHz automatically.
> > We'll try to figure something out for 2.0 Si boards, but setting opps
> > statically in dts is considered incorrect and risky...
> 
> Thanks Denys!!!
> 
> I'll take look at my bbb collection, i have a few of the "beta" bbb's
> which have the earliest non-production silicon, so i can find out what
> rev we started with.
> 
> BTW, since we at bb.org, are moving to this 3.14 ti base, i'll be
> patching this for 1Ghz operation. ;)

Further investigation with marketing and sales shows that all BBB in retail 
channels with 2.0 rev are guaranteed to have 1HGz capable silicon in them! 
Unfortunately, the same silicon revision w/o 1GHz capability was used in other 
boards by our customers. So, it's not possible to set opps just based on the 
silicon rev. Looks like we would have to do it similarly in boneblack.dtb to 
limit it just to BBB baords...

-- 
Denys

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Robert Nelson]

On Thu, Sep 4, 2014 at 2:07 PM, Robert Nelson <robertcnelson@gmail.com> wrote:
>> Indeed, the A5C has a 962A chip which is Si 2.0, while the B has a 962B chip
>> that is Si 2.1, per SPRZ360F.
>
> Found my "A4" don't think anything earlier went out
>
> 962: 1.0

This one was obviously binned, it's still running 1Gh, almost 2 years old..

Regards,

-- 
Robert Nelson
http://www.rcn-ee.com/

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Robert Nelson]

> Indeed, the A5C has a 962A chip which is Si 2.0, while the B has a 962B chip
> that is Si 2.1, per SPRZ360F.

Found my "A4" don't think anything earlier went out

962: 1.0

Regards,

-- 
Robert Nelson
http://www.rcn-ee.com/

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Peter A. Bigot]

On 09/04/2014 01:10 PM, Denys Dmytriyenko wrote:
> On Thu, Sep 04, 2014 at 10:55:02AM -0500, Robert Nelson wrote:
>> On Thu, Sep 4, 2014 at 10:48 AM, Denys Dmytriyenko <denys@ti.com> wrote:
>>> On Thu, Sep 04, 2014 at 10:37:24AM -0500, Peter A. Bigot wrote:
>>>> On 09/04/2014 10:00 AM, Denys Dmytriyenko wrote:
>>>>> On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
>>>>>> On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>>>>>>
>>>>>>> One anomaly I've found is the CPU frequency range.  On debian we have:
>>>>>>>
>>>>>>>     debian@beaglebone:~$ cat
>>>>>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>>>>>     300000 600000 800000 1000000
>>>>>>>
>>>>>>> while on OE we have:
>>>>>>>
>>>>>>>     root@beaglebone:~# cat
>>>>>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>>>>>     300000 600000 720000 800000
>>>>>> Stock yocto-bsp is missing a few things that can be found in Robert
>>>>>> Nelson's patchset for 3.14 linux kernel.
>>>>>>
>>>>>> There are lots of other functionality that is missing from yocto-bsp
>>>>>> kernel for Beaglebone. I suggest you take at the following repos and
>>>>>> scavenge for what you need :P
>>>>> First of all, this is meta-ti mailing list for the corresponding BSP. That's
>>>>> what Peter was asking for, comparing to Robert's Debian and Yocto reference
>>>>> BSPs, not the other way around.
>>>>>
>>>>> Second, Yocto reference BSP is that way for a reason - it's a reference BSP
>>>>> done with pure mainline kernel and u-boot components w/o any patching on top.
>>>>> That's its entire purpose. For anything else special, including performance
>>>>> tweaks, there are other BSPs available. If there is an issue with performance
>>>>> in meta-ti, we'll investigate it and try to match with Robert's BSP.
>>>> Yes, at this time meta-ti's BSP performs as well as I've seen any
>>>> OE-based system, and gets several things right that meta-yocto-bsp
>>>> does not (and one thing wrong that meta-yocto-bsp gets right, I
>>>> think; still investigating, will follow-up when I'm sure).
>>>>
>>>> I've also verified that performance with a native gcc 4.9.1 build on
>>>> BeagleBone with hard float is poor, so it's not due to the way OE
>>>> builds gcc.  I have several competing hypotheses to test.
>>> Can you please point to the test case you were using to measure time? I'd like
>>> to try it with few different toolchains here as well. BTW, we are currently
>>> using Linaro gcc-4.7.3 - I'm wondering how it performs.
>>>
>>>
>>>> But I'm still looking for a way to set the CPU frequency to the
>>>> higher values supported on Beaglebone Black.  I had hoped meta-ti's
>>>> would be able to do that, since it has bone vs boneblack device
>>>> trees and u-boot detection.
>>>>
>>>> Any hints where to look for clock settings?
>>> Some of those values are considered overclocking and according to the manual
>>> will reduce the lifespan of the part...
>> That's an odd statement, as the parts used on the beaglebone black are
>> binned and sold for 1Ghz operation.
> Robert, Peter,
>
> So, after some internal discussion, here's an update:
>
> 1. In TI kernel we do dynamic opp setting based on eFuse check in the file
> arch/arm/mach-omap2/opp33xx_data.c
>
> 2. Since most of our EVMs and EVM-SKs use 2.1 Si, everything works properly
> with eFuse check and dynamic opp setting.
>
> 3. But eFuse was not available in 2.0 rev of silicon, only in 2.1 rev. There
> were many BBB boards made with 2.0 Si that were binned to run at 1GHz. And
> older 1.0 Si revs were not 1GHz capable. Currently checking with HW and
> marketing teams if all/most 2.0 chips are safe to run at 1GHz... As w/o eFuse
> there's no way to determine if the chip is 1GHz capable or not.
>
> 4. If your BBB board has 2.1 Si rev, then you should get 1GHz automatically.
> We'll try to figure something out for 2.0 Si boards, but setting opps
> statically in dts is considered incorrect and risky...

Thank you for the update; that makes many things more clear.

I normally develop using an old rev A5C BBB in a jig with a breadboard 
attached, and never saw the faster speed listed as available.  Today to 
make it easier to compare variations without constantly rebooting, I 
started running yocto-bsp on that board, and pulled out a rev B BBB on 
which I'm running meta-ti's kernel.  On that one the 1GHz option has 
suddenly shown up.

Indeed, the A5C has a 962A chip which is Si 2.0, while the B has a 962B 
chip that is Si 2.1, per SPRZ360F.

I really thought I was losing it there....

(And no, that didn't fix the performance issue, though it's down to 
7m43s under meta-ti+Yocto compared to Debian's 4m54s.  Current 
hypothesis under test is a GCC performance regression between 4.6 and 
4.9.1.)

Peter

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Robert Nelson]

> Robert, Peter,
>
> So, after some internal discussion, here's an update:
>
> 1. In TI kernel we do dynamic opp setting based on eFuse check in the file
> arch/arm/mach-omap2/opp33xx_data.c
>
> 2. Since most of our EVMs and EVM-SKs use 2.1 Si, everything works properly
> with eFuse check and dynamic opp setting.
>
> 3. But eFuse was not available in 2.0 rev of silicon, only in 2.1 rev. There
> were many BBB boards made with 2.0 Si that were binned to run at 1GHz. And
> older 1.0 Si revs were not 1GHz capable. Currently checking with HW and
> marketing teams if all/most 2.0 chips are safe to run at 1GHz... As w/o eFuse
> there's no way to determine if the chip is 1GHz capable or not.
>
> 4. If your BBB board has 2.1 Si rev, then you should get 1GHz automatically.
> We'll try to figure something out for 2.0 Si boards, but setting opps
> statically in dts is considered incorrect and risky...

Thanks Denys!!!

I'll take look at my bbb collection, i have a few of the "beta" bbb's
which have the earliest non-production silicon, so i can find out what
rev we started with.

BTW, since we at bb.org, are moving to this 3.14 ti base, i'll be
patching this for 1Ghz operation. ;)

Regards,

-- 
Robert Nelson
http://www.rcn-ee.com/

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Denys Dmytriyenko]

On Thu, Sep 04, 2014 at 10:55:02AM -0500, Robert Nelson wrote:
> On Thu, Sep 4, 2014 at 10:48 AM, Denys Dmytriyenko <denys@ti.com> wrote:
> > On Thu, Sep 04, 2014 at 10:37:24AM -0500, Peter A. Bigot wrote:
> >> On 09/04/2014 10:00 AM, Denys Dmytriyenko wrote:
> >> >On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
> >> >>On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
> >> >>
> >> >>>One anomaly I've found is the CPU frequency range.  On debian we have:
> >> >>>
> >> >>>    debian@beaglebone:~$ cat
> >> >>>/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
> >> >>>    300000 600000 800000 1000000
> >> >>>
> >> >>>while on OE we have:
> >> >>>
> >> >>>    root@beaglebone:~# cat
> >> >>>/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
> >> >>>    300000 600000 720000 800000
> >> >>Stock yocto-bsp is missing a few things that can be found in Robert
> >> >>Nelson's patchset for 3.14 linux kernel.
> >> >>
> >> >>There are lots of other functionality that is missing from yocto-bsp
> >> >>kernel for Beaglebone. I suggest you take at the following repos and
> >> >>scavenge for what you need :P
> >> >First of all, this is meta-ti mailing list for the corresponding BSP. That's
> >> >what Peter was asking for, comparing to Robert's Debian and Yocto reference
> >> >BSPs, not the other way around.
> >> >
> >> >Second, Yocto reference BSP is that way for a reason - it's a reference BSP
> >> >done with pure mainline kernel and u-boot components w/o any patching on top.
> >> >That's its entire purpose. For anything else special, including performance
> >> >tweaks, there are other BSPs available. If there is an issue with performance
> >> >in meta-ti, we'll investigate it and try to match with Robert's BSP.
> >>
> >> Yes, at this time meta-ti's BSP performs as well as I've seen any
> >> OE-based system, and gets several things right that meta-yocto-bsp
> >> does not (and one thing wrong that meta-yocto-bsp gets right, I
> >> think; still investigating, will follow-up when I'm sure).
> >>
> >> I've also verified that performance with a native gcc 4.9.1 build on
> >> BeagleBone with hard float is poor, so it's not due to the way OE
> >> builds gcc.  I have several competing hypotheses to test.
> >
> > Can you please point to the test case you were using to measure time? I'd like
> > to try it with few different toolchains here as well. BTW, we are currently
> > using Linaro gcc-4.7.3 - I'm wondering how it performs.
> >
> >
> >> But I'm still looking for a way to set the CPU frequency to the
> >> higher values supported on Beaglebone Black.  I had hoped meta-ti's
> >> would be able to do that, since it has bone vs boneblack device
> >> trees and u-boot detection.
> >>
> >> Any hints where to look for clock settings?
> >
> > Some of those values are considered overclocking and according to the manual
> > will reduce the lifespan of the part...
> 
> That's an odd statement, as the parts used on the beaglebone black are
> binned and sold for 1Ghz operation.

Robert, Peter,

So, after some internal discussion, here's an update:

1. In TI kernel we do dynamic opp setting based on eFuse check in the file 
arch/arm/mach-omap2/opp33xx_data.c

2. Since most of our EVMs and EVM-SKs use 2.1 Si, everything works properly 
with eFuse check and dynamic opp setting.

3. But eFuse was not available in 2.0 rev of silicon, only in 2.1 rev. There 
were many BBB boards made with 2.0 Si that were binned to run at 1GHz. And 
older 1.0 Si revs were not 1GHz capable. Currently checking with HW and 
marketing teams if all/most 2.0 chips are safe to run at 1GHz... As w/o eFuse 
there's no way to determine if the chip is 1GHz capable or not.

4. If your BBB board has 2.1 Si rev, then you should get 1GHz automatically. 
We'll try to figure something out for 2.0 Si boards, but setting opps 
statically in dts is considered incorrect and risky...

-- 
Denys

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Peter A. Bigot]

On 09/04/2014 10:41 AM, Peter A. Bigot wrote:
> On 09/04/2014 10:37 AM, Peter A. Bigot wrote:
>>
>> But I'm still looking for a way to set the CPU frequency to the 
>> higher values supported on Beaglebone Black.  I had hoped meta-ti's 
>> would be able to do that, since it has bone vs boneblack device trees 
>> and u-boot detection.
>>
>> Any hints where to look for clock settings?
>>
>
> Apologies; I didn't read Mikhail's full email before responding. 
> Thanks for the links; I'll check those out, and maybe provide meta-ti 
> patches so this gets fixed at the "source".

And with the clues from the referenced patch I finally found this in 
arch/arm/mach-omap2/opp33xx_data.c where it's apparently been for some 
time.  Dunno why I wasn't seeing it on meta-ti kernels; juggling too 
many variables I guess.

Peter

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Peter A. Bigot]

On 09/04/2014 10:48 AM, Denys Dmytriyenko wrote:
> On Thu, Sep 04, 2014 at 10:37:24AM -0500, Peter A. Bigot wrote:
>> On 09/04/2014 10:00 AM, Denys Dmytriyenko wrote:
>>> On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
>>>> On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>>>>
>>>>> One anomaly I've found is the CPU frequency range.  On debian we have:
>>>>>
>>>>>     debian@beaglebone:~$ cat
>>>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>>>     300000 600000 800000 1000000
>>>>>
>>>>> while on OE we have:
>>>>>
>>>>>     root@beaglebone:~# cat
>>>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>>>     300000 600000 720000 800000
>>>> Stock yocto-bsp is missing a few things that can be found in Robert
>>>> Nelson's patchset for 3.14 linux kernel.
>>>>
>>>> There are lots of other functionality that is missing from yocto-bsp
>>>> kernel for Beaglebone. I suggest you take at the following repos and
>>>> scavenge for what you need :P
>>> First of all, this is meta-ti mailing list for the corresponding BSP. That's
>>> what Peter was asking for, comparing to Robert's Debian and Yocto reference
>>> BSPs, not the other way around.
>>>
>>> Second, Yocto reference BSP is that way for a reason - it's a reference BSP
>>> done with pure mainline kernel and u-boot components w/o any patching on top.
>>> That's its entire purpose. For anything else special, including performance
>>> tweaks, there are other BSPs available. If there is an issue with performance
>>> in meta-ti, we'll investigate it and try to match with Robert's BSP.
>> Yes, at this time meta-ti's BSP performs as well as I've seen any
>> OE-based system, and gets several things right that meta-yocto-bsp
>> does not (and one thing wrong that meta-yocto-bsp gets right, I
>> think; still investigating, will follow-up when I'm sure).
>>
>> I've also verified that performance with a native gcc 4.9.1 build on
>> BeagleBone with hard float is poor, so it's not due to the way OE
>> builds gcc.  I have several competing hypotheses to test.
> Can you please point to the test case you were using to measure time? I'd like
> to try it with few different toolchains here as well. BTW, we are currently
> using Linaro gcc-4.7.3 - I'm wondering how it performs.


The context is: 
http://scruss.com/blog/2013/09/23/beaglebone-black-slow-as-a-dog/

The specific application is: http://www.cs.utah.edu/~aek/code/card.cpp

The "ought to work" compiler line is: g++ -Ofast card.cpp -o card

The "kitchen-sink-still-doesn't-help" compiler line is:

g++ \
   -march=armv7-a -mtune=cortex-a8 \
   -mthumb-interwork \
   -mfloat-abi=hard -mfpu=neon \
   -Ofast \
   -o card \
   card.cpp

All my tests are with gcc-4.9.1, with or without the openembedded-core 
patch set.

>
>> But I'm still looking for a way to set the CPU frequency to the
>> higher values supported on Beaglebone Black.  I had hoped meta-ti's
>> would be able to do that, since it has bone vs boneblack device
>> trees and u-boot detection.
>>
>> Any hints where to look for clock settings?
> Some of those values are considered overclocking and according to the manual
> will reduce the lifespan of the part...

So the Beagle Bone Black SRM's claim that the AM3358BZCZ100 can run at 
1GHz is not entirely true?  Or is it just that the operating-points in 
that patch are too aggressive?

Peter

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Robert Nelson]

On Thu, Sep 4, 2014 at 10:48 AM, Denys Dmytriyenko <denys@ti.com> wrote:
> On Thu, Sep 04, 2014 at 10:37:24AM -0500, Peter A. Bigot wrote:
>> On 09/04/2014 10:00 AM, Denys Dmytriyenko wrote:
>> >On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
>> >>On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>> >>
>> >>>One anomaly I've found is the CPU frequency range.  On debian we have:
>> >>>
>> >>>    debian@beaglebone:~$ cat
>> >>>/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>> >>>    300000 600000 800000 1000000
>> >>>
>> >>>while on OE we have:
>> >>>
>> >>>    root@beaglebone:~# cat
>> >>>/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>> >>>    300000 600000 720000 800000
>> >>Stock yocto-bsp is missing a few things that can be found in Robert
>> >>Nelson's patchset for 3.14 linux kernel.
>> >>
>> >>There are lots of other functionality that is missing from yocto-bsp
>> >>kernel for Beaglebone. I suggest you take at the following repos and
>> >>scavenge for what you need :P
>> >First of all, this is meta-ti mailing list for the corresponding BSP. That's
>> >what Peter was asking for, comparing to Robert's Debian and Yocto reference
>> >BSPs, not the other way around.
>> >
>> >Second, Yocto reference BSP is that way for a reason - it's a reference BSP
>> >done with pure mainline kernel and u-boot components w/o any patching on top.
>> >That's its entire purpose. For anything else special, including performance
>> >tweaks, there are other BSPs available. If there is an issue with performance
>> >in meta-ti, we'll investigate it and try to match with Robert's BSP.
>>
>> Yes, at this time meta-ti's BSP performs as well as I've seen any
>> OE-based system, and gets several things right that meta-yocto-bsp
>> does not (and one thing wrong that meta-yocto-bsp gets right, I
>> think; still investigating, will follow-up when I'm sure).
>>
>> I've also verified that performance with a native gcc 4.9.1 build on
>> BeagleBone with hard float is poor, so it's not due to the way OE
>> builds gcc.  I have several competing hypotheses to test.
>
> Can you please point to the test case you were using to measure time? I'd like
> to try it with few different toolchains here as well. BTW, we are currently
> using Linaro gcc-4.7.3 - I'm wondering how it performs.
>
>
>> But I'm still looking for a way to set the CPU frequency to the
>> higher values supported on Beaglebone Black.  I had hoped meta-ti's
>> would be able to do that, since it has bone vs boneblack device
>> trees and u-boot detection.
>>
>> Any hints where to look for clock settings?
>
> Some of those values are considered overclocking and according to the manual
> will reduce the lifespan of the part...

That's an odd statement, as the parts used on the beaglebone black are
binned and sold for 1Ghz operation.

Regards,

-- 
Robert Nelson
http://www.rcn-ee.com/

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Denys Dmytriyenko]

On Thu, Sep 04, 2014 at 10:37:24AM -0500, Peter A. Bigot wrote:
> On 09/04/2014 10:00 AM, Denys Dmytriyenko wrote:
> >On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
> >>On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
> >>
> >>>One anomaly I've found is the CPU frequency range.  On debian we have:
> >>>
> >>>    debian@beaglebone:~$ cat
> >>>/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
> >>>    300000 600000 800000 1000000
> >>>
> >>>while on OE we have:
> >>>
> >>>    root@beaglebone:~# cat
> >>>/sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
> >>>    300000 600000 720000 800000
> >>Stock yocto-bsp is missing a few things that can be found in Robert
> >>Nelson's patchset for 3.14 linux kernel.
> >>
> >>There are lots of other functionality that is missing from yocto-bsp
> >>kernel for Beaglebone. I suggest you take at the following repos and
> >>scavenge for what you need :P
> >First of all, this is meta-ti mailing list for the corresponding BSP. That's
> >what Peter was asking for, comparing to Robert's Debian and Yocto reference
> >BSPs, not the other way around.
> >
> >Second, Yocto reference BSP is that way for a reason - it's a reference BSP
> >done with pure mainline kernel and u-boot components w/o any patching on top.
> >That's its entire purpose. For anything else special, including performance
> >tweaks, there are other BSPs available. If there is an issue with performance
> >in meta-ti, we'll investigate it and try to match with Robert's BSP.
> 
> Yes, at this time meta-ti's BSP performs as well as I've seen any
> OE-based system, and gets several things right that meta-yocto-bsp
> does not (and one thing wrong that meta-yocto-bsp gets right, I
> think; still investigating, will follow-up when I'm sure).
> 
> I've also verified that performance with a native gcc 4.9.1 build on
> BeagleBone with hard float is poor, so it's not due to the way OE
> builds gcc.  I have several competing hypotheses to test.

Can you please point to the test case you were using to measure time? I'd like 
to try it with few different toolchains here as well. BTW, we are currently 
using Linaro gcc-4.7.3 - I'm wondering how it performs.


> But I'm still looking for a way to set the CPU frequency to the
> higher values supported on Beaglebone Black.  I had hoped meta-ti's
> would be able to do that, since it has bone vs boneblack device
> trees and u-boot detection.
> 
> Any hints where to look for clock settings?

Some of those values are considered overclocking and according to the manual 
will reduce the lifespan of the part...

-- 
Denys

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Peter A. Bigot]

On 09/04/2014 10:37 AM, Peter A. Bigot wrote:
> On 09/04/2014 10:00 AM, Denys Dmytriyenko wrote:
>> On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
>>> On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>>>
>>>> One anomaly I've found is the CPU frequency range.  On debian we have:
>>>>
>>>>     debian@beaglebone:~$ cat
>>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>>     300000 600000 800000 1000000
>>>>
>>>> while on OE we have:
>>>>
>>>>     root@beaglebone:~# cat
>>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>>     300000 600000 720000 800000
>>> Stock yocto-bsp is missing a few things that can be found in Robert
>>> Nelson's patchset for 3.14 linux kernel.
>>>
>>> There are lots of other functionality that is missing from yocto-bsp
>>> kernel for Beaglebone. I suggest you take at the following repos and
>>> scavenge for what you need :P
>> First of all, this is meta-ti mailing list for the corresponding BSP. 
>> That's
>> what Peter was asking for, comparing to Robert's Debian and Yocto 
>> reference
>> BSPs, not the other way around.
>>
>> Second, Yocto reference BSP is that way for a reason - it's a 
>> reference BSP
>> done with pure mainline kernel and u-boot components w/o any patching 
>> on top.
>> That's its entire purpose. For anything else special, including 
>> performance
>> tweaks, there are other BSPs available. If there is an issue with 
>> performance
>> in meta-ti, we'll investigate it and try to match with Robert's BSP.
>
> Yes, at this time meta-ti's BSP performs as well as I've seen any 
> OE-based system, and gets several things right that meta-yocto-bsp 
> does not (and one thing wrong that meta-yocto-bsp gets right, I think; 
> still investigating, will follow-up when I'm sure).
>
> I've also verified that performance with a native gcc 4.9.1 build on 
> BeagleBone with hard float is poor, so it's not due to the way OE 
> builds gcc.  I have several competing hypotheses to test.
>
> But I'm still looking for a way to set the CPU frequency to the higher 
> values supported on Beaglebone Black.  I had hoped meta-ti's would be 
> able to do that, since it has bone vs boneblack device trees and 
> u-boot detection.
>
> Any hints where to look for clock settings?
>

Apologies; I didn't read Mikhail's full email before responding. Thanks 
for the links; I'll check those out, and maybe provide meta-ti patches 
so this gets fixed at the "source".

Peter

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Peter A. Bigot]

On 09/04/2014 10:00 AM, Denys Dmytriyenko wrote:
> On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
>> On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
>>
>>> One anomaly I've found is the CPU frequency range.  On debian we have:
>>>
>>>     debian@beaglebone:~$ cat
>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>     300000 600000 800000 1000000
>>>
>>> while on OE we have:
>>>
>>>     root@beaglebone:~# cat
>>> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>>>     300000 600000 720000 800000
>> Stock yocto-bsp is missing a few things that can be found in Robert
>> Nelson's patchset for 3.14 linux kernel.
>>
>> There are lots of other functionality that is missing from yocto-bsp
>> kernel for Beaglebone. I suggest you take at the following repos and
>> scavenge for what you need :P
> First of all, this is meta-ti mailing list for the corresponding BSP. That's
> what Peter was asking for, comparing to Robert's Debian and Yocto reference
> BSPs, not the other way around.
>
> Second, Yocto reference BSP is that way for a reason - it's a reference BSP
> done with pure mainline kernel and u-boot components w/o any patching on top.
> That's its entire purpose. For anything else special, including performance
> tweaks, there are other BSPs available. If there is an issue with performance
> in meta-ti, we'll investigate it and try to match with Robert's BSP.

Yes, at this time meta-ti's BSP performs as well as I've seen any 
OE-based system, and gets several things right that meta-yocto-bsp does 
not (and one thing wrong that meta-yocto-bsp gets right, I think; still 
investigating, will follow-up when I'm sure).

I've also verified that performance with a native gcc 4.9.1 build on 
BeagleBone with hard float is poor, so it's not due to the way OE builds 
gcc.  I have several competing hypotheses to test.

But I'm still looking for a way to set the CPU frequency to the higher 
values supported on Beaglebone Black.  I had hoped meta-ti's would be 
able to do that, since it has bone vs boneblack device trees and u-boot 
detection.

Any hints where to look for clock settings?

Peter

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Denys Dmytriyenko]

On Thu, Sep 04, 2014 at 02:50:03PM +0000, Mikhail Zakharov wrote:
> On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:
> 
> > One anomaly I've found is the CPU frequency range.  On debian we have:
> >
> >    debian@beaglebone:~$ cat 
> > /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
> >    300000 600000 800000 1000000
> >
> > while on OE we have:
> >
> >    root@beaglebone:~# cat 
> > /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
> >    300000 600000 720000 800000
> 
> Stock yocto-bsp is missing a few things that can be found in Robert
> Nelson's patchset for 3.14 linux kernel.
> 
> There are lots of other functionality that is missing from yocto-bsp 
> kernel for Beaglebone. I suggest you take at the following repos and
> scavenge for what you need :P

First of all, this is meta-ti mailing list for the corresponding BSP. That's 
what Peter was asking for, comparing to Robert's Debian and Yocto reference 
BSPs, not the other way around.

Second, Yocto reference BSP is that way for a reason - it's a reference BSP 
done with pure mainline kernel and u-boot components w/o any patching on top. 
That's its entire purpose. For anything else special, including performance 
tweaks, there are other BSPs available. If there is an issue with performance 
in meta-ti, we'll investigate it and try to match with Robert's BSP.

BTW, why would you base your own BSP on the Yocto reference one - isn't it too 
much work to patch it up to your liking?

-- 
Denys

Re: poor performance of OpenEmbedded on, BeagleBoneBlack compared to Debian

[Mikhail Zakharov]

On Wed, Sep 3, 2014 at 8:59 PM, Peter A. Bigot <pab@pabigot.com> wrote:

> One anomaly I've found is the CPU frequency range.  On debian we have:
>
>    debian@beaglebone:~$ cat 
> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>    300000 600000 800000 1000000
>
> while on OE we have:
>
>    root@beaglebone:~# cat 
> /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies
>    300000 600000 720000 800000

Stock yocto-bsp is missing a few things that can be found in Robert
Nelson's patchset for 3.14 linux kernel.

You can find a fix in the following overlay:
https://github.com/mzakharo/meta-yocto-bone

The patches you are looking for are here:
https://github.com/mzakharo/meta-yocto-bone/blob/master/recipes-kernel/linux/files/dts/0002-arm-dts-am335x-boneblack-add-cpu0-opp-points.patch
https://github.com/mzakharo/meta-yocto-bone/blob/master/recipes-kernel/linux/files/misc/cpufreq.cfg


There are lots of other functionality that is missing from yocto-bsp 
kernel for Beaglebone. I suggest you take at the following repos and
scavenge for what you need :P
https://github.com/RobertBerger/meta-mainline/tree/daisy-training-v3.14.x/beagle-bone-black-ml/recipes-kernel/linux/linux-yocto-custom/patches
https://github.com/RobertCNelson/linux-dev/tree/am33x-v3.14/patches
https://github.com/RobertCNelson/ti-linux-kernel-dev/tree/ti-linux-3.14.y/patches