Requested FAQ addition - Mandrake and partial-i686 platforms

Requested FAQ addition - Mandrake and partial-i686 platforms

[Brandon Stewart]

Apparently, there is an issue with glibc on versions less than 2.3.1-15 
(and maybe others), where it mistakenly treats CPUs as full i686 
compliant when they only execute a subset of the i686 instructions. For 
example, the VIA C3 supports pretty much everything i686 except CMOV, 
yet the broken versions of glibc will detect it as fully i686 compliant.

 From someone who emailed me privately, this apparently affects K6-III 
as well. Possibly other Cyrix or AMD CPUs are affected, though I don't 
have a complete list.

The problem is that Mandrake 9.1 ships with a broken glibc. So you would 
expect that the incorrectly detected CPUs just wouldn't work. But 
apparently, Mandrake added a CMOV instruction emulator patch to their 
kernel, both the one that ships precompiled and the source rpm.

So people will find that compiling the Mandrake version works fine, yet 
any kernel downloaded from kernel.org, 2.6 or other, will not work at 
all. The symptom is that booting the shiny new kernel will hang after 
"Freeing unused kernel memory". Doing a magic sysreq will reveal that 
/sbin/init is executing do_invalid_op(). You can keep pressing the magic 
sysreq stack dump key, and you will keep getting a new stack trace. 
Caps-lock works, and CTRL-ALT-DEL will reboot the machine.

There are three possible workarounds:
1) Upgrade glibc to a working version. I haven't done this myself, so I 
don't know if the bug has been fixed yet. But it would be the best solution.
2) Remove i686 libraries from glibc. This can be done by 'mv /lib/i686 
/lib/i686.invalid'. This is what I did, and it works. While some 
performance is lost, it's not noticeable, especially given that the 
stock Mandrake kernel is i386 compatible, and so has limited optimization.
3) Reapply the CMOV emulation patch to your downloaded kernel. Not 
recommended since it turns one CPU cycle into 400.

-Brandon

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]

On Tue, 12 Aug 2003 10:48:59 EDT, Brandon Stewart <rbrandonstewart@yahoo.com>  said:

> 3) Reapply the CMOV emulation patch to your downloaded kernel. Not 
> recommended since it turns one CPU cycle into 400.

True, as far as it goes.  However, you need to balance the huge hit you take
on CMOV and how often it's actually used in glibc, against how many places
you save 5 or 10 cycles due to more optimized code...

No, I don't have numbers for either side of the question...


[-- Attachment #2: Type: application/pgp-signature, Size: 226 bytes --]

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Dave Jones]

On Tue, Aug 12, 2003 at 10:48:59AM -0400, Brandon Stewart wrote:
 > Apparently, there is an issue with glibc on versions less than 2.3.1-15 
 > (and maybe others), where it mistakenly treats CPUs as full i686 
 > compliant when they only execute a subset of the i686 instructions. For 
 > example, the VIA C3 supports pretty much everything i686 except CMOV, 
 > yet the broken versions of glibc will detect it as fully i686 compliant.

It's actually a problem that gcc assumes 686 = 686+cmov. The glibc
isn't broken, its just compiled for gcc's view of what 686 is.

 > From someone who emailed me privately, this apparently affects K6-III 
 > as well. Possibly other Cyrix or AMD CPUs are affected, though I don't 
 > have a complete list.

No. K6-III is a family 5 processor, so should get built for 586.

 > There are three possible workarounds:
 > 1) Upgrade glibc to a working version. I haven't done this myself, so I 
 > don't know if the bug has been fixed yet. But it would be the best solution.

Its not a glibc bug.

 > 2) Remove i686 libraries from glibc. This can be done by 'mv /lib/i686 
 > /lib/i686.invalid'. This is what I did, and it works. While some 
 > performance is lost, it's not noticeable, especially given that the 
 > stock Mandrake kernel is i386 compatible, and so has limited optimization.

This is the best of the bunch, and is the same solution debian users
were using for some time a while back when someone made a '686' version
of libssl.

		Dave

-- 
 Dave Jones     http://www.codemonkey.org.uk

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Alan Cox]

On Maw, 2003-08-12 at 15:48, Brandon Stewart wrote:
> Apparently, there is an issue with glibc on versions less than 2.3.1-15 
> (and maybe others), where it mistakenly treats CPUs as full i686 
> compliant when they only execute a subset of the i686 instructions

VIA C3 has the full set of i686 required instructions. The whole story
is a lot more complex

gcc i686 mode outputs cmov instruction sequences without checking cmov
is present at runtime. So gcc "i686" is actually "i686 and a bit". It
actually doesn't really make sense to do a true i686 mode without cmov
either.

Red Hat's rpm knows about this so I'm suprised the Mandrake one gets it
wrong and installs arch=686 packages without checking for cmov.

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Juan Quintela]

>>>>> "alan" == Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

Hi

alan> gcc i686 mode outputs cmov instruction sequences without checking cmov
alan> is present at runtime. So gcc "i686" is actually "i686 and a bit". It
alan> actually doesn't really make sense to do a true i686 mode without cmov
alan> either.

alan> Red Hat's rpm knows about this so I'm suprised the Mandrake one gets it
alan> wrong and installs arch=686 packages without checking for cmov.

again, it is a bit more complex than that :p

Mandrake glibc _alsa_ has a /lib/i686/ directory (i.e. it is not a
separate package).  ld.so looks at the architecture for choice about
what lib to load.

Problem, as others stated is that kernel i686 definition and gcc i686
definition are different (gcc definition is i686+cmov basically).

Mandrake kernels workaround that telling ld.so that i686 without cmov
are i586 class machines, not i686 class machines.

It will be more elegant to make the decission in ld.so, but there are
other problems with dlopen() that I don't remember.

To make things worse, via c3 implement cmov instruction if all
operands are in registers (i.e. no operand in memory), I know that
this faked somebody that did a test on cmov :(

To make history more intersting, new Via C3 have a complet cmov
instruction.

Later, Juan.

-- 
In theory, practice and theory are the same, but in practice they 
are different -- Larry McVoy

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Jan Rychter]

[-- Attachment #1: Type: text/plain, Size: 502 bytes --]

>>>>> "Juan" == Juan Quintela <quintela@mandrakesoft.com> writes:
[...]
 Juan> To make things worse, via c3 implement cmov instruction if all
 Juan> operands are in registers (i.e. no operand in memory), I know
 Juan> that this faked somebody that did a test on cmov :(

 Juan> To make history more intersting, new Via C3 have a complet cmov
 Juan> instruction.

Does anybody have the actual CPU revisions corresponding to these
changes? There has been a lot of confusion over this.

--J.

[-- Attachment #2: Type: application/pgp-signature, Size: 188 bytes --]

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Alan Cox]

On Sul, 2003-08-17 at 19:51, Jan Rychter wrote:
> Does anybody have the actual CPU revisions corresponding to these
> changes? There has been a lot of confusion over this.

Ezra -> 3dnow, no cmov  (500MHz->1Ghz)
Nemeiah -> sse, cmov (1Ghz-)
Anataur -> dunno yet, I'd assume sse

The chips report cmov only if they have full cmov instructions, so
a look at /proc/cpuinfo will tell you.

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Jamie Lokier]

Alan Cox wrote:
> On Sul, 2003-08-17 at 19:51, Jan Rychter wrote:
> > Does anybody have the actual CPU revisions corresponding to these
> > changes? There has been a lot of confusion over this.
> 
> Ezra -> 3dnow, no cmov  (500MHz->1Ghz)
> Nemeiah -> sse, cmov (1Ghz-)
> Anataur -> dunno yet, I'd assume sse
> 
> The chips report cmov only if they have full cmov instructions, so
> a look at /proc/cpuinfo will tell you.

So the register-only cmov on the Cyrix which you mentioned does not
come with the cpuid cmov flag?

-- Jamie

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Alan Cox]

> > The chips report cmov only if they have full cmov instructions, so
> > a look at /proc/cpuinfo will tell you.
> 
> So the register-only cmov on the Cyrix which you mentioned does not
> come with the cpuid cmov flag?

processor       : 0
vendor_id       : CentaurHauls
cpu family      : 6
model           : 7
model name      : VIA Samuel 2
stepping        : 3
cpu MHz         : 531.829
cache size      : 64 KB
physical id     : 0
siblings        : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu de tsc msr cx8 mtrr pge mmx 3dnow
bogomips        : 1061.68

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[Dave Jones]

On Sun, Aug 17, 2003 at 11:51:50AM -0700, Jan Rychter wrote:
 >  Juan> To make history more intersting, new Via C3 have a complet cmov
 >  Juan> instruction.
 > Does anybody have the actual CPU revisions corresponding to these
 > changes? There has been a lot of confusion over this.

Nehemiah is family 6, model 9. I've seen stepping 1 & 3 'out there'.

		Dave
-- 
 Dave Jones     http://www.codemonkey.org.uk

Re: Requested FAQ addition - Mandrake and partial-i686 platforms

[insecure]

On Tuesday 12 August 2003 17:48, Brandon Stewart wrote:
> Apparently, there is an issue with glibc on versions less than 2.3.1-15
> (and maybe others), where it mistakenly treats CPUs as full i686
> compliant when they only execute a subset of the i686 instructions. For
> example, the VIA C3 supports pretty much everything i686 except CMOV,
> yet the broken versions of glibc will detect it as fully i686 compliant.
>
>  From someone who emailed me privately, this apparently affects K6-III
> as well. Possibly other Cyrix or AMD CPUs are affected, though I don't
> have a complete list.
>
> The problem is that Mandrake 9.1 ships with a broken glibc. So you would
> expect that the incorrectly detected CPUs just wouldn't work. But
> apparently, Mandrake added a CMOV instruction emulator patch to their
> kernel, both the one that ships precompiled and the source rpm.
>
> So people will find that compiling the Mandrake version works fine, yet
> any kernel downloaded from kernel.org, 2.6 or other, will not work at
> all. The symptom is that booting the shiny new kernel will hang after
> "Freeing unused kernel memory". Doing a magic sysreq will reveal that
> /sbin/init is executing do_invalid_op(). You can keep pressing the magic
> sysreq stack dump key, and you will keep getting a new stack trace.
> Caps-lock works, and CTRL-ALT-DEL will reboot the machine.

Hm. I was right. ;)

> There are three possible workarounds:
> 1) Upgrade glibc to a working version. I haven't done this myself, so I
> don't know if the bug has been fixed yet. But it would be the best
> solution. 2) Remove i686 libraries from glibc. This can be done by 'mv
> /lib/i686 /lib/i686.invalid'. This is what I did, and it works. While some
> performance is lost, it's not noticeable, especially given that the
> stock Mandrake kernel is i386 compatible, and so has limited optimization.
> 3) Reapply the CMOV emulation patch to your downloaded kernel. Not
> recommended since it turns one CPU cycle into 400.

4) Never never never never NEVER compile for 586+

You lost several days debugging this. It's $days*24*60*60=$days*86400 seconds
~= $days * 86400000000000 CPU cycles. A bit high price for using optimized
binaries, eh?

IMHO:
Speed optimizations make sense in heavy CPU bound tasks like bzip2.
CPU-heavy part of code is usually small, can be hand-optimized.
The remaining 99,999% of code is best optimized for size.
At least you will save on pagein and icache footprint.

After you happily compiled a piece of code with all bells and
whistles for your new shiny 986+ processor, do take a look at
generated assembly. There might be surprizes.

BTW, will anyone bet that gcc generates better code with cmov's
than without? ;)
--
vda