Default value for FULL_OPTIMIZATION

Default value for FULL_OPTIMIZATION

[Pascal Bach]

Hi everybody

I noticed an inconsistency in oe-core regarding the variable: FULL_OPTIMIZATION

In bitbake.conf the variable is defined as follows:

    FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"

But if I look into the documentation.conf it tells me the following:

    FULL_OPTIMIZATION[doc]= "The options to pass in TARGET_CFLAGS and CFLAGS when compiling an optimized system. This variable defaults to '-fexpensive-optimizations -fomit-frame-pointer -frename-registers -O2'."

I'm not sure if the documentation is wrong or if the default set in bitbake.conf is incorrect.
Maybe somebody can shed some light on this.

Regards
Pascal

Re: Default value for FULL_OPTIMIZATION

[Mike Looijmans]

You'd have to check the GCC documentation to be sure, but I suspect that all 
of "-fexpensive-optimizations -fomit-frame-pointer -frename-registers" are 
already in effect at -O2 optimization level, so they're redundant.


On 02-02-16 14:26, Pascal Bach wrote:
> Hi everybody
>
> I noticed an inconsistency in oe-core regarding the variable: FULL_OPTIMIZATION
>
> In bitbake.conf the variable is defined as follows:
>
>      FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
>
> But if I look into the documentation.conf it tells me the following:
>
>      FULL_OPTIMIZATION[doc]= "The options to pass in TARGET_CFLAGS and CFLAGS when compiling an optimized system. This variable defaults to '-fexpensive-optimizations -fomit-frame-pointer -frename-registers -O2'."
>
> I'm not sure if the documentation is wrong or if the default set in bitbake.conf is incorrect.
> Maybe somebody can shed some light on this.
>
> Regards
> Pascal
>



Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Visit us at: Embedded World 2016 Germany, Hall 1, stand number 220.
Click here to schedule an appointment with us at Embedded World, free ticket included

http://topicembeddedproducts.com/schedule-an-appointment/

Re: Default value for FULL_OPTIMIZATION

[Richard Purdie]

On Tue, 2016-02-02 at 14:26 +0100, Pascal Bach wrote:
> Hi everybody
> 
> I noticed an inconsistency in oe-core regarding the variable:
> FULL_OPTIMIZATION
> 
> In bitbake.conf the variable is defined as follows:
> 
>     FULL_OPTIMIZATION = "-O2 -pipe ${DEBUG_FLAGS}"
> 
> But if I look into the documentation.conf it tells me the following:
> 
>     FULL_OPTIMIZATION[doc]= "The options to pass in TARGET_CFLAGS and
> CFLAGS when compiling an optimized system. This variable defaults to
> '-fexpensive-optimizations -fomit-frame-pointer -frename-registers 
> -O2'."
> 
> I'm not sure if the documentation is wrong or if the default set in
> bitbake.conf is incorrect.
> Maybe somebody can shed some light on this.

I think bitbake.conf is correct and we should update the documentation.
A patch would be most welcome.

Cheers,

Richard

Re: Default value for FULL_OPTIMIZATION

[Pascal Bach]

Am 02.02.2016 um 15:26 schrieb Mike Looijmans:
> You'd have to check the GCC documentation to be sure, but I suspect that all of "-fexpensive-optimizations -fomit-frame-pointer -frename-registers" are already in effect at -O2 optimization level, so they're redundant.
>

- "-fexpensive-optimizations" is included in -O2
- "-fomit-frame-pointer" is included in -O but "only on machines where doing so does not interfere with debugging", whatever that means ;)
- "-frename-registers" doesn't seem to be included

Soruce: https://gcc.gnu.org/onlinedocs/gcc-5.2.0/gcc/Optimize-Options.html

Re: Default value for FULL_OPTIMIZATION

[Phil Blundell]

On Tue, 2016-02-02 at 14:26 +0100, Pascal Bach wrote:
> I'm not sure if the documentation is wrong or if the default set in bitbake.conf is incorrect.
> Maybe somebody can shed some light on this.

FULL_OPTIMIZATION in bitbake.conf did use to match what the
documentation says (or at least, was much closer to it).  It was changed
to the current value by commit 9cb71137 which apparently omitted to
update the documentation. 

As an aside, this commit also added "-pipe" to both FULL_OPTIMIZATION
and DEBUG_OPTIMIZATION, which seems a bit weird since (a) -pipe doesn't
have any effect on the generated code; and (b) if it's going to enabled
for both debug and release builds then it might as well go in ${CC} or
somewhere.  Also, that commit seems to have deliberately turned down the
level of optimisation somewhat in order to facilitate debugging, which
is perhaps not what folks who request FULL_OPTIMIZATION would expect.

But anyway, irrespective of the detailed merits or otherwise of that
patch, I think that updating the documentation to match what the code is
now doing would be a fine idea.

p.

Re: Default value for FULL_OPTIMIZATION

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1777 bytes --]


> On Feb 2, 2016, at 12:52 PM, Phil Blundell <pb@pbcl.net> wrote:
> 
> On Tue, 2016-02-02 at 14:26 +0100, Pascal Bach wrote:
>> I'm not sure if the documentation is wrong or if the default set in bitbake.conf is incorrect.
>> Maybe somebody can shed some light on this.
> 
> FULL_OPTIMIZATION in bitbake.conf did use to match what the
> documentation says (or at least, was much closer to it).  It was changed
> to the current value by commit 9cb71137 which apparently omitted to
> update the documentation.

yes thats a gap agreed.

> 
> As an aside, this commit also added "-pipe" to both FULL_OPTIMIZATION
> and DEBUG_OPTIMIZATION, which seems a bit weird since (a) -pipe doesn't
> have any effect on the generated code; and (b) if it's going to enabled
> for both debug and release builds then it might as well go in ${CC} or
> somewhere.  Also, that commit seems to have deliberately turned down the
> level of optimisation somewhat in order to facilitate debugging, which
> is perhaps not what folks who request FULL_OPTIMIZATION would expect.

-pipe does speed up build time so its an optimization. FULL_OPTIMIZATION is not about
code generation only. However, I agree to the fact that if we want to use
it in all combination then it should go into a common global variable which is common
across target/native/nativesdk recipes.

> 
> But anyway, irrespective of the detailed merits or otherwise of that
> patch, I think that updating the documentation to match what the code is
> now doing would be a fine idea.
> 
> p.
> 
> 
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

Re: Default value for FULL_OPTIMIZATION

[Phil Blundell]

On Tue, 2016-02-02 at 22:17 -0800, Khem Raj wrote:
> -pipe does speed up build time so its an optimization. 

How much does it really speed it up by?  I seem to recall the effect was
fairly negligible, but maybe I am remembering that wrong.

p.

Re: Default value for FULL_OPTIMIZATION

[Khem Raj]

On Wed, Feb 3, 2016 at 12:48 PM, Phil Blundell <pb@pbcl.net> wrote:
> On Tue, 2016-02-02 at 22:17 -0800, Khem Raj wrote:
>> -pipe does speed up build time so its an optimization.
>
> How much does it really speed it up by?  I seem to recall the effect was
> fairly negligible, but maybe I am remembering that wrong.

it does not create files for intermediate objects so if you have slow
media it can be significant
but if you have SSDs then it might be diminished.

>
> p.
>
>

Re: Default value for FULL_OPTIMIZATION

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 490 bytes --]

On 3 February 2016 at 20:51, Khem Raj <raj.khem@gmail.com> wrote:

> it does not create files for intermediate objects so if you have slow
> media it can be significant
> but if you have SSDs then it might be diminished.
>

That's the theory but in my experience with a modern file system on
spinning rust the file is so short-lived it doesn't appear to actually
touch the disk at all...  at least I didn't notice any performance change
when adding just -pipe in a test.

Ross

[-- Attachment #2: Type: text/html, Size: 905 bytes --]

Re: Default value for FULL_OPTIMIZATION

[Phil Blundell]

On Wed, 2016-02-03 at 12:51 -0800, Khem Raj wrote:
> On Wed, Feb 3, 2016 at 12:48 PM, Phil Blundell <pb@pbcl.net> wrote:
> > On Tue, 2016-02-02 at 22:17 -0800, Khem Raj wrote:
> >> -pipe does speed up build time so its an optimization.
> >
> > How much does it really speed it up by?  I seem to recall the effect was
> > fairly negligible, but maybe I am remembering that wrong.
> 
> it does not create files for intermediate objects so if you have slow
> media it can be significant
> but if you have SSDs then it might be diminished.

The files go in /tmp though, right?  Even if that isn't a tmpfs, which I
think it is on almost all systems nowadays, they will almost certainly
just sit in the buffer cache and never actually get written to the
disk. 

p.

Re: Default value for FULL_OPTIMIZATION

[Khem Raj]

On Wed, Feb 3, 2016 at 12:57 PM, Phil Blundell <pb@pbcl.net> wrote:
> On Wed, 2016-02-03 at 12:51 -0800, Khem Raj wrote:
>> On Wed, Feb 3, 2016 at 12:48 PM, Phil Blundell <pb@pbcl.net> wrote:
>> > On Tue, 2016-02-02 at 22:17 -0800, Khem Raj wrote:
>> >> -pipe does speed up build time so its an optimization.
>> >
>> > How much does it really speed it up by?  I seem to recall the effect was
>> > fairly negligible, but maybe I am remembering that wrong.
>>
>> it does not create files for intermediate objects so if you have slow
>> media it can be significant
>> but if you have SSDs then it might be diminished.
>
> The files go in /tmp though, right?  Even if that isn't a tmpfs, which I
> think it is on almost all systems nowadays, they will almost certainly
> just sit in the buffer cache and never actually get written to the
> disk.

in systems of  this age it should be yet.

>
> p.
>
>

Re: Default value for FULL_OPTIMIZATION

[Phil Blundell]

On Wed, 2016-02-03 at 12:59 -0800, Khem Raj wrote:
> in systems of  this age it should be yet.

Sorry, I didn't really understand that comment.

For what it's worth, as a very quick and fairly unscientific comparison,
I did "time make -j4" in a glibc tree and got:

real	1m38.812s
user	4m6.420s
sys	0m23.148s

and then I cleaned the tree and did "time make -j4 CFLAGS='-O2 -g
-pipe'" and got:

real	1m37.299s
user	4m21.420s
sys	0m27.304s

I'm not sure these results are statistically significant but on this
very limited evidence it certainly doesn't look like -pipe is providing
any transformational benefits.  The real time has gone down a tiny bit,
which suggests that -pipe is giving some extra parallelism, but the
system and user time have both actually gone up in the second case. 

p.

Re: Default value for FULL_OPTIMIZATION

[Khem Raj]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]


> On Feb 3, 2016, at 1:08 PM, Phil Blundell <pb@pbcl.net> wrote:
> 
> On Wed, 2016-02-03 at 12:59 -0800, Khem Raj wrote:
>> in systems of  this age it should be yet.
> 
> Sorry, I didn't really understand that comment.

systems of today ( new systems ). read ‘yet' as 'yes'

> 
> For what it's worth, as a very quick and fairly unscientific comparison,
> I did "time make -j4" in a glibc tree and got:
> 
> real	1m38.812s
> user	4m6.420s
> sys	0m23.148s
> 
> and then I cleaned the tree and did "time make -j4 CFLAGS='-O2 -g
> -pipe'" and got:
> 
> real	1m37.299s
> user	4m21.420s
> sys	0m27.304s
> 
> I'm not sure these results are statistically significant but on this
> very limited evidence it certainly doesn't look like -pipe is providing
> any transformational benefits.  The real time has gone down a tiny bit,
> which suggests that -pipe is giving some extra parallelism, but the
> system and user time have both actually gone up in the second case.

Some bigger loads like webkit might be a better sample.

> 
> p.
> 
> 


[-- Attachment #2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 211 bytes --]

Re: Default value for FULL_OPTIMIZATION

[Mike Looijmans]

On 03-02-16 21:59, Khem Raj wrote:
> On Wed, Feb 3, 2016 at 12:57 PM, Phil Blundell <pb@pbcl.net> wrote:
>> On Wed, 2016-02-03 at 12:51 -0800, Khem Raj wrote:
>>> On Wed, Feb 3, 2016 at 12:48 PM, Phil Blundell <pb@pbcl.net> wrote:
>>>> On Tue, 2016-02-02 at 22:17 -0800, Khem Raj wrote:
>>>>> -pipe does speed up build time so its an optimization.
>>>>
>>>> How much does it really speed it up by?  I seem to recall the effect was
>>>> fairly negligible, but maybe I am remembering that wrong.
>>>
>>> it does not create files for intermediate objects so if you have slow
>>> media it can be significant
>>> but if you have SSDs then it might be diminished.
>>
>> The files go in /tmp though, right?  Even if that isn't a tmpfs, which I
>> think it is on almost all systems nowadays, they will almost certainly
>> just sit in the buffer cache and never actually get written to the
>> disk.
>
> in systems of  this age it should be yet.

In the lab, yes.

In the real world, the system may be under load. On desktop computers, /tmp/ 
is usually NOT on tmpfs.

At some point, in order to "write" to your filesystem, it will need to read it 
first (e.g. to get a free allocation spot). In that cause, the read request 
will have to get in line behind the other IO requests in the queue, and it 
suddenly might take seconds to perform that simple request. Even on SSD, this 
will happen and give a noticable drop in performance.

You might want to run that test while reading your disk (dd if=/dev/sda 
of=/dev/null) and see what happens.


Kind regards,

Mike Looijmans
System Expert

TOPIC Embedded Products
Eindhovenseweg 32-C, NL-5683 KH Best
Postbus 440, NL-5680 AK Best
Telefoon: +31 (0) 499 33 69 79
E-mail: mike.looijmans@topicproducts.com
Website: www.topicproducts.com

Please consider the environment before printing this e-mail

Visit us at: Embedded World 2016 Germany, Hall 1, stand number 220.
Click here to schedule an appointment with us at Embedded World, free ticket included

http://topicembeddedproducts.com/schedule-an-appointment/