TSCs are a no-no on i386

TSCs are a no-no on i386

[Jan-Benedict Glaw]

Hi!

This small patch fixes a long-standing problem for bare i386 CPUs. These
don't have TSCs and so, a recent 2.4.x kernel will simply halt the
machine leaving a text like "This CPU has no TSC feature (ie was
compiled for Pentium+) but I do depend on it".

Please apply. Worst to say, even Debian seems to start using i486+
features (ie. libstdc++5 is SIGILLed on Am386 because there's no
"lock" insn available)...

MfG, JBG


--- linux-2.4.22-pre9/arch/i386/config.in.orig	2003-07-30 15:00:27.000000000 +0200
+++ linux-2.4.22-pre9/arch/i386/config.in	2003-07-30 15:01:56.000000000 +0200
@@ -56,6 +56,7 @@
    define_bool CONFIG_RWSEM_XCHGADD_ALGORITHM n
    define_bool CONFIG_X86_PPRO_FENCE y
    define_bool CONFIG_X86_F00F_WORKS_OK n
+   define_bool CONFIG_X86_HAS_TSC n
 else
    define_bool CONFIG_X86_WP_WORKS_OK y
    define_bool CONFIG_X86_INVLPG y


-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

Re: TSCs are a no-no on i386

[Maciej W. Rozycki]

On Wed, 30 Jul 2003, Jan-Benedict Glaw wrote:

> This small patch fixes a long-standing problem for bare i386 CPUs. These
> don't have TSCs and so, a recent 2.4.x kernel will simply halt the

 Actually i486 processors and some Pentium-like clones have none, too.

> machine leaving a text like "This CPU has no TSC feature (ie was
> compiled for Pentium+) but I do depend on it".

 That only happens if you change existing .config -- it may happen for all
options that are unset instead of being set to "n" -- and it can be worked
around by running `make oldconfig' twice. 

> Please apply. Worst to say, even Debian seems to start using i486+
> features (ie. libstdc++5 is SIGILLed on Am386 because there's no
> "lock" insn available)...

 You need the change for CONFIG_M486 and CONFIG_M586 as well.

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1798 bytes --]

On Wed, 2003-07-30 16:18:11 +0200, Maciej W. Rozycki <macro@ds2.pg.gda.pl>
wrote in message <Pine.GSO.3.96.1030730161309.911E-100000@delta.ds2.pg.gda.pl>:
> On Wed, 30 Jul 2003, Jan-Benedict Glaw wrote:
> > This small patch fixes a long-standing problem for bare i386 CPUs. These
> > don't have TSCs and so, a recent 2.4.x kernel will simply halt the
> 
>  Actually i486 processors and some Pentium-like clones have none, too.

Ah, I didn't know that. Here's the updated patch:


--- linux-2.4.22-pre9/arch/i386/config.in.orig	2003-07-30 16:38:03.000000000 +0200
+++ linux-2.4.22-pre9/arch/i386/config.in	2003-07-30 16:39:20.000000000 +0200
@@ -56,6 +56,7 @@
    define_bool CONFIG_RWSEM_XCHGADD_ALGORITHM n
    define_bool CONFIG_X86_PPRO_FENCE y
    define_bool CONFIG_X86_F00F_WORKS_OK n
+   define_bool CONFIG_X86_HAS_TSC n
 else
    define_bool CONFIG_X86_WP_WORKS_OK y
    define_bool CONFIG_X86_INVLPG y
@@ -72,6 +73,7 @@
    define_bool CONFIG_X86_ALIGNMENT_16 y
    define_bool CONFIG_X86_PPRO_FENCE y
    define_bool CONFIG_X86_F00F_WORKS_OK n
+   define_bool CONFIG_X86_HAS_TSC n
 fi
 if [ "$CONFIG_M586" = "y" ]; then
    define_int  CONFIG_X86_L1_CACHE_SHIFT 5
@@ -79,6 +81,7 @@
    define_bool CONFIG_X86_ALIGNMENT_16 y
    define_bool CONFIG_X86_PPRO_FENCE y
    define_bool CONFIG_X86_F00F_WORKS_OK n
+   define_bool CONFIG_X86_HAS_TSC n
 fi
 if [ "$CONFIG_M586TSC" = "y" ]; then
    define_int  CONFIG_X86_L1_CACHE_SHIFT 5



MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Matthew Garrett]

Jan-Benedict Glaw wrote:

>Please apply. Worst to say, even Debian seems to start using i486+
>features (ie. libstdc++5 is SIGILLed on Am386 because there's no
>"lock" insn available)...

g++ >=3.2 use 486-specific instructions in order to do atomic operations
in C++ code. 3.3 includes a 386 version of the same code, but it's not
possible to use a mixture of the two (ie, code compiled with a "486" g++
will not work on the "386" version), and so to keep ABI compatibility
with the other distributions it's necessary to break 386. The "obvious"
solution (dragging this back on topic) is a kernel patch to emulate 486
instructions on a 386.

-- 
Matthew Garrett | mjg59-chiark.mail.linux-rutgers.kernel@srcf.ucam.org

Re: TSCs are a no-no on i386

[Alan Cox]

On Mer, 2003-07-30 at 17:58, Matthew Garrett wrote:
> g++ >=3.2 use 486-specific instructions in order to do atomic operations
> in C++ code. 3.3 includes a 386 version of the same code, but it's not
> possible to use a mixture of the two (ie, code compiled with a "486" g++
> will not work on the "386" version), and so to keep ABI compatibility
> with the other distributions it's necessary to break 386. The "obvious"
> solution (dragging this back on topic) is a kernel patch to emulate 486
> instructions on a 386.

You don't need to mash the kernel up for this - you can write yourself a
SIGILL handler to do the work - take a look at things like the pure
software MMX preloader someone wrote.

Re: TSCs are a no-no on i386

[Adrian Bunk]

On Wed, Jul 30, 2003 at 03:56:23PM +0200, Jan-Benedict Glaw wrote:
>...
> Please apply. Worst to say, even Debian seems to start using i486+
> features (ie. libstdc++5 is SIGILLed on Am386 because there's no
> "lock" insn available)...

Shouldn't the 486 emulation in the latest 386 kernel images in Debian
unstable take care of this?

> MfG, JBG
>...

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: TSCs are a no-no on i386

[Mike Fedyk]

On Wed, Jul 30, 2003 at 08:10:06PM +0200, Adrian Bunk wrote:
> On Wed, Jul 30, 2003 at 03:56:23PM +0200, Jan-Benedict Glaw wrote:
> >...
> > Please apply. Worst to say, even Debian seems to start using i486+
> > features (ie. libstdc++5 is SIGILLed on Am386 because there's no
> > "lock" insn available)...
> 
> Shouldn't the 486 emulation in the latest 386 kernel images in Debian
> unstable take care of this?

What emulation?

Re: TSCs are a no-no on i386

[Adrian Bunk]

On Wed, Jul 30, 2003 at 11:30:33AM -0700, Mike Fedyk wrote:
> On Wed, Jul 30, 2003 at 08:10:06PM +0200, Adrian Bunk wrote:
> > On Wed, Jul 30, 2003 at 03:56:23PM +0200, Jan-Benedict Glaw wrote:
> > >...
> > > Please apply. Worst to say, even Debian seems to start using i486+
> > > features (ie. libstdc++5 is SIGILLed on Am386 because there's no
> > > "lock" insn available)...
> > 
> > Shouldn't the 486 emulation in the latest 386 kernel images in Debian
> > unstable take care of this?
> 
> What emulation?

486 emulation
CONFIG_CPU_EMU486
  When used on a 386, Linux can emulate 3 instructions from the 486 set.
  This allows user space programs compiled for 486 to run on a 386
  without crashing with a SIGILL. As any emulation, performance will be
 very low, but since these instruction are not often used, this might
  not hurt.  The emulated instructions are:
     - bswap (does the same as htonl())
     - cmpxchg (used in multi-threading, mutex locking)
     - xadd (rarely used)

  Note that this can also allow Step-A 486's to correctly run multi-thread
  applications since cmpxchg has a wrong opcode on this early CPU.

  Don't use this to enable multi-threading on an SMP machine, the lock
  atomicity can't be guaranted!

  Although it's highly preferable that you only execute programs targetted
  for your CPU, it may happen that, consecutively to a hardware replacement,
  or during rescue of a damaged system, you have to execute such programs
  on an inadapted processor.  In this case, this option will help you get
  your programs working, even if they will be slower.

  It is recommended that you say N here in any case, except for the
  kernels that you will use on your rescue disks.
  
  This option should not be left on by default, because it means that
  you execute a program not targetted for your CPU.  You should recompile
  your applications whenever possible.

  If you are not sure, say N.


cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: TSCs are a no-no on i386

[Alan Cox]

On Mer, 2003-07-30 at 19:45, Adrian Bunk wrote:
>   Note that this can also allow Step-A 486's to correctly run multi-thread
>   applications since cmpxchg has a wrong opcode on this early CPU.
> 
>   Don't use this to enable multi-threading on an SMP machine, the lock
>   atomicity can't be guaranted!

That is of course the other problem with this approach - you can't
really get the intended results without some extremely heavyweight code
(using an IPI to halt all CPU's, doing the access and then resuming
them)

The bigger problem (and certainly with some of the cmov emulation hacks
I've seen) is getting the security checking right when you need to
reprocess the user data - another reason to do it in userspace with a
preload 8)

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On Wed, 2003-07-30 20:10:06 +0200, Adrian Bunk <bunk@fs.tum.de>
wrote in message <20030730181006.GB21734@fs.tum.de>:
> On Wed, Jul 30, 2003 at 03:56:23PM +0200, Jan-Benedict Glaw wrote:
> >...
> > Please apply. Worst to say, even Debian seems to start using i486+
> > features (ie. libstdc++5 is SIGILLed on Am386 because there's no
> > "lock" insn available)...
> 
> Shouldn't the 486 emulation in the latest 386 kernel images in Debian
> unstable take care of this?

Specifically patched kernel? Sounds lame to me...

Generic solution would be to have a generic implementation, IMHO.
Up to now, I've nowhere found some hard facts that the new opcodes do
measureable speed up things. Sure, saving some hundreds/thousands/...
CPU cycles is nice - but not, if that's only 0.1% of the whole number of
CPU cycles burned in a run. That doesn't, IMHO, really legitimate do
unsupport the i386.

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1550 bytes --]

On Wed, 2003-07-30 20:45:29 +0200, Adrian Bunk <bunk@fs.tum.de>
wrote in message <20030730184529.GE21734@fs.tum.de>:
> On Wed, Jul 30, 2003 at 11:30:33AM -0700, Mike Fedyk wrote:
> > On Wed, Jul 30, 2003 at 08:10:06PM +0200, Adrian Bunk wrote:
> > > On Wed, Jul 30, 2003 at 03:56:23PM +0200, Jan-Benedict Glaw wrote:
> > > > Please apply. Worst to say, even Debian seems to start using i486+
> > > > features (ie. libstdc++5 is SIGILLed on Am386 because there's no
> > > > "lock" insn available)...
> > > 
> > > Shouldn't the 486 emulation in the latest 386 kernel images in Debian
> > > unstable take care of this?
> > 
> > What emulation?
> 
> 486 emulation
> CONFIG_CPU_EMU486
>   When used on a 386, Linux can emulate 3 instructions from the 486 set.
>   This allows user space programs compiled for 486 to run on a 386
>   without crashing with a SIGILL. As any emulation, performance will be
>  very low, but since these instruction are not often used, this might
>   not hurt.  The emulated instructions are:
>      - bswap (does the same as htonl())
>      - cmpxchg (used in multi-threading, mutex locking)
>      - xadd (rarely used)

libstdc++ (and it's main user, apt-get) break at a LOCK insn.

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]

On Wed, 2003-07-30 21:01:00 +0100, Alan Cox <alan@lxorguk.ukuu.org.uk>
wrote in message <1059595260.10447.6.camel@dhcp22.swansea.linux.org.uk>:
> On Mer, 2003-07-30 at 19:45, Adrian Bunk wrote:
> >   Note that this can also allow Step-A 486's to correctly run multi-thread
> >   applications since cmpxchg has a wrong opcode on this early CPU.
> > 
> >   Don't use this to enable multi-threading on an SMP machine, the lock
> >   atomicity can't be guaranted!
> 
> The bigger problem (and certainly with some of the cmov emulation hacks
> I've seen) is getting the security checking right when you need to
> reprocess the user data - another reason to do it in userspace with a
> preload 8)

Well... For sure, I can write a LD_PRELOAD lib dealing with SIGILL, but
how do I enable it for the whole system. That is, I'd need to give
LD_PRELOAD=xxx at the kernel's boot prompt to have it as en environment
variable for each and every process?

That sounds a tad inelegant to me. Really, I'd prefer to see libstdc++
be compiled for i386 ...

...and IFF those new opcodes bring _that_ much performance, then we
should think about another Debian distribution for i686-linux. Up to
now, I was really proud of having _one_ distribution that's basically
capable of running on all and any machines I own...

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Petr Vandrovec]

On Wed, Jul 30, 2003 at 10:28:22PM +0200, Jan-Benedict Glaw wrote:
> On Wed, 2003-07-30 20:45:29 +0200, Adrian Bunk <bunk@fs.tum.de>
> >   When used on a 386, Linux can emulate 3 instructions from the 486 set.
> >   This allows user space programs compiled for 486 to run on a 386
> >   without crashing with a SIGILL. As any emulation, performance will be
> >  very low, but since these instruction are not often used, this might
> >   not hurt.  The emulated instructions are:
> >      - bswap (does the same as htonl())
> >      - cmpxchg (used in multi-threading, mutex locking)
> >      - xadd (rarely used)
> 
> libstdc++ (and it's main user, apt-get) break at a LOCK insn.

There is no such instruction. Skip LOCK prefix and decode again,
you'll get either cmpxchg or xadd (or cmpxchg8b, but then it does
not work on i486 too).

And yes, it speeds some workloads a lot. Best to look at code,
with these instructions you can do couple of operations without
doing IPC to synchronize with other threads.

And as far as I know, userspace solution does not qualify: major user
is synchronization between threads, so with userspace solution you
will have to do very hard job to get it right (first implement your
own synchronization, if possible without busy waiting... and then
implement cmpxchg on the top of it), while kernel solution can do 
that simple & correct for UP case.
				Best regards,
					Petr Vandrovec
					vandrove@vc.cvut.cz

Re: TSCs are a no-no on i386

[J.A. Magallon]

On 07.30, Jan-Benedict Glaw wrote:
[...]
> 
> ...and IFF those new opcodes bring _that_ much performance, then we
> should think about another Debian distribution for i686-linux. Up to
> now, I was really proud of having _one_ distribution that's basically
> capable of running on all and any machines I own...
> 

Do as Mandrake (and I suppose another distros), and put optimized libraries
in /lib/i686, /lib/mmx, /lib/sse, /usr/lib/i686, etc. New ld.so supports
mapping different versions of library depending on arch.

-- 
J.A. Magallon <jamagallon@able.es>      \                 Software is like
sex:
werewolf.able.es                         \           It's better when it's
free
Mandrake Linux release 9.2 (Cooker) for i586
Linux 2.4.22-pre9-jam1m (gcc 3.3.1 (Mandrake Linux 9.2 3.3.1-0.7mdk))

Re: TSCs are a no-no on i386

[Alan Cox]

On Mer, 2003-07-30 at 21:33, Jan-Benedict Glaw wrote:
> Well... For sure, I can write a LD_PRELOAD lib dealing with SIGILL, but
> how do I enable it for the whole system. That is, I'd need to give
> LD_PRELOAD=xxx at the kernel's boot prompt to have it as en environment
> variable for each and every process?


/etc/ld.preload

> That sounds a tad inelegant to me. Really, I'd prefer to see libstdc++
> be compiled for i386 ...

True

Re: TSCs are a no-no on i386

[Alan Cox]

On Mer, 2003-07-30 at 22:50, Petr Vandrovec wrote:
> There is no such instruction. Skip LOCK prefix and decode again,
> you'll get either cmpxchg or xadd (or cmpxchg8b, but then it does
> not work on i486 too).

And if the byte you are looking at was patched by another thread you've
blown it. Your emulation can only be so good 8) People do stuff like
patching instructions under software decode as a robustness check - its
normally pretty amusing

Re: TSCs are a no-no on i386

[Adrian Bunk]

On Wed, Jul 30, 2003 at 10:33:18PM +0200, Jan-Benedict Glaw wrote:
>...
> That sounds a tad inelegant to me. Really, I'd prefer to see libstdc++
> be compiled for i386 ...
> 
> ...and IFF those new opcodes bring _that_ much performance, then we
> should think about another Debian distribution for i686-linux. Up to
> now, I was really proud of having _one_ distribution that's basically
> capable of running on all and any machines I own...

The 486 emlation patch for 386 is the way to still allow 386's to run 
Debian.

To compile libstdc++ for 486 wasn't a performance question - a
libstdc++.so.5 compiled for 386 would have meant that C++ binaries
compiled on Debian wouldn't run on other Linux distributions and vice
versa [1] (it's a bug in libstdc++ that will AFAIR be fixed in gcc 3.4).

> MfG, JBG

cu
Adrian

[1] http://lists.debian.org/debian-devel/2003/debian-devel-200304/msg01895.html

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1444 bytes --]

On Thu, 2003-07-31 00:19:25 +0200, J.A. Magallon <jamagallon@able.es>
wrote in message <20030730221925.GA2858@werewolf.able.es>:
> On 07.30, Jan-Benedict Glaw wrote:
> [...]
> > ...and IFF those new opcodes bring _that_ much performance, then we
> > should think about another Debian distribution for i686-linux. Up to
> > now, I was really proud of having _one_ distribution that's basically
> > capable of running on all and any machines I own...
> 
> Do as Mandrake (and I suppose another distros), and put optimized libraries
> in /lib/i686, /lib/mmx, /lib/sse, /usr/lib/i686, etc. New ld.so supports
> mapping different versions of library depending on arch.

I know. ...and this won't work. Think about this:

	- Some lib links (statically) some parts of libstdc++5.a. This
	  is done on a i386.
	- Some app (compiled for i486 on some other box) links against
	  the above lib (which was copied over).

Now, you've got two ways to access your atomic variables. Do that
multithreaded, and you'll boom.

It's all about "eventually" and "possibly", but you break binary
compatibility at some point.

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1691 bytes --]

On Thu, 2003-07-31 02:22:31 +0200, Adrian Bunk <bunk@fs.tum.de>
wrote in message <20030731002230.GE22991@fs.tum.de>:
> On Wed, Jul 30, 2003 at 10:33:18PM +0200, Jan-Benedict Glaw wrote:
> >...
> > That sounds a tad inelegant to me. Really, I'd prefer to see libstdc++
> > be compiled for i386 ...
> > 
> > ...and IFF those new opcodes bring _that_ much performance, then we
> > should think about another Debian distribution for i686-linux. Up to
> > now, I was really proud of having _one_ distribution that's basically
> > capable of running on all and any machines I own...
> 
> The 486 emlation patch for 386 is the way to still allow 386's to run 
> Debian.

Okay, I'll have a look at it. Where's the 2.6.x version?

> To compile libstdc++ for 486 wasn't a performance question - a
> libstdc++.so.5 compiled for 386 would have meant that C++ binaries
> compiled on Debian wouldn't run on other Linux distributions and vice
> versa [1] (it's a bug in libstdc++ that will AFAIR be fixed in gcc 3.4).

I've got no idea when this version will come up. What's the (Debian)
timeline for a gcc-3.4 based libstdc++ then? I've already tried
gcc-snapshot's libstdc++, but that's libstdc++6, which won't link with
todays applications...

Well, it's not only that. Seen the thread named like "Time loss on
i486"? That'll be some fun, too:)

Itching my head, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1268 bytes --]

On Wed, 2003-07-30 23:50:32 +0200, Petr Vandrovec <vandrove@vc.cvut.cz>
wrote in message <20030730215032.GA18892@vana.vc.cvut.cz>:
> On Wed, Jul 30, 2003 at 10:28:22PM +0200, Jan-Benedict Glaw wrote:
> > On Wed, 2003-07-30 20:45:29 +0200, Adrian Bunk <bunk@fs.tum.de>

> And yes, it speeds some workloads a lot. Best to look at code,
> with these instructions you can do couple of operations without
> doing IPC to synchronize with other threads.

Which ones? I am always told "it's faster, then", but nobody really
proofed that. Take some multithreadded apps. How often do they *really*
lock/unlock mutexes, and in which ratio does that compare to their
"normal" computing needs?

If an application's main job is locking/unlocking mutexes, then the
author should possibly think about that. If it's main task is to do the
computational stuff, then I've got no (real) problem with this extra
Linux^Wtax, esp. on those faster boxes...

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Willy Tarreau]

On Thu, Jul 31, 2003 at 08:22:52AM +0200, Jan-Benedict Glaw wrote:
> > The 486 emlation patch for 386 is the way to still allow 386's to run 
> > Debian.
> 
> Okay, I'll have a look at it. Where's the 2.6.x version?

It doesn't exist, but could certainly easily be ported from 2.4.

But I think that people want to use it the wrong way. I wrote it to allow older
machines to *occasionally* run code designed for the wrong architecture. Eg,
your machine dies, you unplug the disk and puts it in another one, a smaller
one, and cross your fingers for the system to work again. In this case, I
think it's easier to compile a kernel with the right options to allow emulation
than to reinstall the system with the proper packages.

Now it seems to me that people want to use it as an excuse not to respect
architectural constraints and processors instructions sets. Of course, it's
easy to compile everything for i686 and think "well, if the machine doesn't
support this or that, then the kernel will do...". But you quickly end up in
even more slowing down small machines.

The other problem lies with the lock :
When a 486 executes "LOCK ; CMPXCHG", it locks the bus during the whole cmpxchg
instruction. If a 386 executes the same code, it will get an exception which
will be caught by the emulator. I don't see how we can do such an atomic
operation while holding a lock. At best, we would think about a global memory
based shared lock during the operation (eg: int bus_lock;), but it's not
implemented at the moment, and will only be compatible with processors sharing
the same code. Add-on processors, such as co-processors, transputer cards, or
DSPs, will know nothing about such a lock emulation. And it would result in
even poorer performance of course !

And yes, I've done intensive benchmarks on the code, as did Denis Vladensko.
I used it daily on my 386sx firewall, which avoided me to recompile my pppd
for this architecture... At this time, I had a wrong gcc+libc combinations
and always got BSWAP instructions all over the code. Although pppd did not
suffer from emulation, I have seen other programs which didn't like it at
all (gzip or bzip2 for example). CPU intensive programs optimized for i686
which use lots of CMOV for example, need about 400 cycles to do a simple CMOV !
this can slow down things to an order of more than 100 when placed in an inner
loop.

So to resume, everything can be done through emulation, but that's probably
not what we want as a standard for performance reasons. When I have time, I
may port it to 2.6, but that's not no my priority list.

Cheers,
Willy

Re: TSCs are a no-no on i386

[Richard B. Johnson]

On Wed, 31 Jul 2003, Alan Cox wrote:

> On Mer, 2003-07-30 at 21:33, Jan-Benedict Glaw wrote:
> > Well... For sure, I can write a LD_PRELOAD lib dealing with SIGILL, but
> > how do I enable it for the whole system. That is, I'd need to give
> > LD_PRELOAD=xxx at the kernel's boot prompt to have it as en environment
> > variable for each and every process?
>
>
> /etc/ld.preload
>
> > That sounds a tad inelegant to me. Really, I'd prefer to see libstdc++
> > be compiled for i386 ...
>
> True
>

What is a runtime library doing with a TSC? That's the basic problem.
These things are for operating systems and, last time I checked, the
'C' runtime libraries weren't (but maybe GNU changed that definition, no?)


Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
            Note 96.31% of all statistics are fiction.

Emulating i486+ insn on i386 (was: TSCs are a no-no on i386)

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1530 bytes --]

On Thu, 2003-07-31 07:11:42 -0400, Richard B. Johnson <root@chaos.analogic.com>
wrote in message <Pine.LNX.4.53.0307310709150.3917@chaos>:
> On Wed, 31 Jul 2003, Alan Cox wrote:
> 
> > On Mer, 2003-07-30 at 21:33, Jan-Benedict Glaw wrote:
> > > Well... For sure, I can write a LD_PRELOAD lib dealing with SIGILL, but
> > > how do I enable it for the whole system. That is, I'd need to give
> > > LD_PRELOAD=xxx at the kernel's boot prompt to have it as en environment
> > > variable for each and every process?
> >
> > /etc/ld.preload
> >
> > > That sounds a tad inelegant to me. Really, I'd prefer to see libstdc++
> > > be compiled for i386 ...
> 
> What is a runtime library doing with a TSC? That's the basic problem.
> These things are for operating systems and, last time I checked, the
> 'C' runtime libraries weren't (but maybe GNU changed that definition, no?)

Oh, sorry:) One problem is that TSCs may be enabled on i386. They crash
upon boot-up. While I sent the patch, I ranted about i486 optimized
libstdc++5 Debian is shipping, which (also, but in a different way)
break i386. A thread has started about the later issue. Maybe someone
should have switched subject, though...

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1877 bytes --]

On Thu, 2003-07-31 09:17:19 +0200, Willy Tarreau <willy@w.ods.org>
wrote in message <20030731071719.GA26249@alpha.home.local>:
> On Thu, Jul 31, 2003 at 08:22:52AM +0200, Jan-Benedict Glaw wrote:
> > > The 486 emlation patch for 386 is the way to still allow 386's to run 
> > > Debian.
> > 
> > Okay, I'll have a look at it. Where's the 2.6.x version?
> 
> It doesn't exist, but could certainly easily be ported from 2.4.

> So to resume, everything can be done through emulation, but that's probably
> not what we want as a standard for performance reasons. When I have time, I
> may port it to 2.6, but that's not no my priority list.

Thanks for that. In the meantime, I've started to give a try to the
userspace version (using a LD_PRELOAD lib). My current Problem:

amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so ls
sigill.c:_init():69: sigill started, sigaction() = 0
build.sh  intercept.h  libsigill.so  run.sh  sigill.c  sigill.o
amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so apt-get update
Illegal instruction

See? It's loaded at the "ls" call, but it seems to be not loaded for
apt-get.

I've tried to put the lib's name into /etc/ld.so.preload, but it seems
it's never loaded then (for apt-get, ls is doing fine...):

amtus:~/sigill_catcher# cat /etc/ld.so.preload 
sigill.c:_init():69: sigill started, sigaction() = 0
/root/sigill_catcher/libsigill.so
amtus:~/sigill_catcher# apt-get update
Illegal instruction

Where's the difference? Seems I don't know enough about ld.so...
Any useful hints for me?

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 867 bytes --]

On Thu, 2003-07-31 00:05:53 +0100, Alan Cox <alan@lxorguk.ukuu.org.uk>
wrote in message <1059606259.10505.20.camel@dhcp22.swansea.linux.org.uk>:
> On Mer, 2003-07-30 at 21:33, Jan-Benedict Glaw wrote:
> > Well... For sure, I can write a LD_PRELOAD lib dealing with SIGILL, but
> > how do I enable it for the whole system. That is, I'd need to give
> > LD_PRELOAD=xxx at the kernel's boot prompt to have it as en environment
> > variable for each and every process?
> 
> /etc/ld.preload

/etc/ld.so.preload, that is. Only for the records:)

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Alan Cox]

On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> Thanks for that. In the meantime, I've started to give a try to the
> userspace version (using a LD_PRELOAD lib). My current Problem:
> 
> amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so ls
> sigill.c:_init():69: sigill started, sigaction() = 0
> build.sh  intercept.h  libsigill.so  run.sh  sigill.c  sigill.o
> amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so apt-get update
> Illegal instruction
> 
> See? It's loaded at the "ls" call, but it seems to be not loaded for
> apt-get.

Remember you need to overload signal setting functions like sigaction.
My guess is apt decided to disable your signal and you didnt stop it

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 2463 bytes --]

On Thu, 2003-07-31 12:51:09 +0100, Alan Cox <alan@lxorguk.ukuu.org.uk>
wrote in message <1059652268.16608.8.camel@dhcp22.swansea.linux.org.uk>:
> On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> > Thanks for that. In the meantime, I've started to give a try to the
> > userspace version (using a LD_PRELOAD lib). My current Problem:
> > 
> > amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so ls
> > sigill.c:_init():69: sigill started, sigaction() = 0
> > build.sh  intercept.h  libsigill.so  run.sh  sigill.c  sigill.o
> > amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so apt-get update
> > Illegal instruction
> > 
> > See? It's loaded at the "ls" call, but it seems to be not loaded for
> > apt-get.
> 
> Remember you need to overload signal setting functions like sigaction.
> My guess is apt decided to disable your signal and you didnt stop it

I thought about that. Though, strace shows this is straight after ldso
mapped all libs. So apt-get's main() didn't yet start.

amtus:~/sigill_catcher# cat run.sh 
#!/bin/sh

LD_PRELOAD=./libsigill.so apt-get update

amtus:~/sigill_catcher# strace -o xxx -f -ff ./run.sh 
Process 681 attached (waiting for parent)
Process 681 resumed (parent 680 ready)
Process 680 suspended
Process 680 resumed
Process 681 detached
./run.sh: line 3:   681 Illegal instruction LD_PRELOAD=./libsigill.so apt-get update
amtus:~/sigill_catcher# grep -i sig xx* | grep -i ILL
xxx:stat64("/root/sigill_catcher", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
xxx:wait4(-1, [WIFSIGNALED(s) && WTERMSIG(s) == SIGILL], 0, NULL) = 681
xxx.681:open("./libsigill.so", O_RDONLY)        = 3
xxx.681:getcwd("/root/sigill_catcher", 128)     = 21
xxx.681:--- SIGILL (Illegal instruction) @ 0 (0) ---

So my lib gets loaded, but it's _init() function seems to be called too
late (in prior, some parts of libstdc++5 decided to commit suicide).

Looking at the strace, libsigill is the loaded as the very first action.
Then, all the other libs are mapped. It seems that _init()ing them
happens somewhat in reverse order or something like that... I'm a but
hosed in that case:)

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1916 bytes --]

On Thu, 2003-07-31 14:14:48 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de>
wrote in message <20030731121448.GW1873@lug-owl.de>:
> On Thu, 2003-07-31 12:51:09 +0100, Alan Cox <alan@lxorguk.ukuu.org.uk>
> wrote in message <1059652268.16608.8.camel@dhcp22.swansea.linux.org.uk>:
> > On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> > > Thanks for that. In the meantime, I've started to give a try to the
> > > userspace version (using a LD_PRELOAD lib). My current Problem:
> > > 
> > > amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so ls
> > > sigill.c:_init():69: sigill started, sigaction() = 0
> > > build.sh  intercept.h  libsigill.so  run.sh  sigill.c  sigill.o
> > > amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so apt-get update
> > > Illegal instruction
> > > 
> > > See? It's loaded at the "ls" call, but it seems to be not loaded for
> > > apt-get.

So there we are. Thanks to someone who pointed me to LD_DEBUG et al., I
see that my _init() is called too late:

amtus:~/sigill_catcher# LD_DEBUG=all LD_VERBOSE=1
LD_PRELOAD=./libsigill.so apt-get update 2> ld_infos                                 
Illegal instruction
amtus:~/sigill_catcher# grep 'calling init' ld_infos 
00691:  calling init: /lib/libc.so.6
00691:  calling init: /lib/libdl.so.2
00691:  calling init: /lib/libgcc_s.so.1
00691:  calling init: /lib/libm.so.6
00691:  calling init: /usr/lib/libstdc++.so.5

As I guessed, libstdc++5's _init() is called before mine (LD_PRELOADed)
_init() function and thus, I cannot intercept this SIGILL, as it
seems...

Any useful hints from here?

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Jamie Lokier]

Willy Tarreau wrote:
> The other problem lies with the lock :
> When a 486 executes "LOCK ; CMPXCHG", it locks the bus during the whole cmpxchg
> instruction. If a 386 executes the same code, it will get an exception which
> will be caught by the emulator. I don't see how we can do such an atomic
> operation while holding a lock. At best, we would think about a global memory
> based shared lock during the operation (eg: int bus_lock;), but it's not
> implemented at the moment, and will only be compatible with processors sharing
> the same code. Add-on processors, such as co-processors, transputer cards, or
> DSPs, will know nothing about such a lock emulation. And it would result in
> even poorer performance of course !

Of course this is not a problem when "lock;cmpxchg" is used only for thread
synchronisation on uniprocessor 386s...  The lock prefix is irrelevant then.

Perhaps the emulation should refuse to pretend to work on an SMP 386 :)

-- Jamie

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Martin Schlemmer]

On Thu, 2003-07-31 at 15:01, Jan-Benedict Glaw wrote:
> On Thu, 2003-07-31 14:14:48 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de>
> wrote in message <20030731121448.GW1873@lug-owl.de>:
> > On Thu, 2003-07-31 12:51:09 +0100, Alan Cox <alan@lxorguk.ukuu.org.uk>
> > wrote in message <1059652268.16608.8.camel@dhcp22.swansea.linux.org.uk>:
> > > On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> > > > Thanks for that. In the meantime, I've started to give a try to the
> > > > userspace version (using a LD_PRELOAD lib). My current Problem:
> > > > 
> > > > amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so ls
> > > > sigill.c:_init():69: sigill started, sigaction() = 0
> > > > build.sh  intercept.h  libsigill.so  run.sh  sigill.c  sigill.o
> > > > amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so apt-get update
> > > > Illegal instruction
> > > > 
> > > > See? It's loaded at the "ls" call, but it seems to be not loaded for
> > > > apt-get.
> 
> So there we are. Thanks to someone who pointed me to LD_DEBUG et al., I
> see that my _init() is called too late:
> 
> amtus:~/sigill_catcher# LD_DEBUG=all LD_VERBOSE=1
> LD_PRELOAD=./libsigill.so apt-get update 2> ld_infos                                 
> Illegal instruction
> amtus:~/sigill_catcher# grep 'calling init' ld_infos 
> 00691:  calling init: /lib/libc.so.6
> 00691:  calling init: /lib/libdl.so.2
> 00691:  calling init: /lib/libgcc_s.so.1
> 00691:  calling init: /lib/libm.so.6
> 00691:  calling init: /usr/lib/libstdc++.so.5
> 
> As I guessed, libstdc++5's _init() is called before mine (LD_PRELOADed)
> _init() function and thus, I cannot intercept this SIGILL, as it
> seems...
> 

You could do what we do with our path sandbox - basically you
use a wrapper that setup the environment (prob not needed in
your case), and then spawn bash with our sandbox lib preloaded,
which then calls whatever should be 'path sandboxed'.

Should work with what you do as well, as bash/whatever do not use
c++ code, and when it execute the actual program, your lib should
have hooked into whatever that needs be.


Regards,

-- 
Martin Schlemmer

Re: TSCs are a no-no on i386

[Jamie Lokier]

Alan Cox wrote:
> On Mer, 2003-07-30 at 22:50, Petr Vandrovec wrote:
> > There is no such instruction. Skip LOCK prefix and decode again,
> > you'll get either cmpxchg or xadd (or cmpxchg8b, but then it does
> > not work on i486 too).
> 
> And if the byte you are looking at was patched by another thread you've
> blown it. Your emulation can only be so good 8) People do stuff like
> patching instructions under software decode as a robustness check - its
> normally pretty amusing

On a uniprocessor 386, this is not a problem.  Just disable preemption
in the kernel decoder.

Of course if you do it in userspace using SIGILL, then it is broken :)

-- Jamie

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Jamie Lokier]

Alan Cox wrote:
> On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> > Thanks for that. In the meantime, I've started to give a try to the
> > userspace version (using a LD_PRELOAD lib). My current Problem:
> > 
> > amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so ls
> > sigill.c:_init():69: sigill started, sigaction() = 0
> > build.sh  intercept.h  libsigill.so  run.sh  sigill.c  sigill.o
> > amtus:~/sigill_catcher# LD_PRELOAD=./libsigill.so apt-get update
> > Illegal instruction
> > 
> > See? It's loaded at the "ls" call, but it seems to be not loaded for
> > apt-get.
> 
> Remember you need to overload signal setting functions like sigaction.
> My guess is apt decided to disable your signal and you didnt stop it

An application might install its own SIGILL handler to emulate or trap
_other_ instructions.  To do it properly, you have to chain the handlers.

Not sure how to do this, when you get to the stage of two LD_PRELOAD
libraries each wanting to overload sigaction.

-- Jamie

Re: TSCs are a no-no on i386

[Willy Tarreau]

On Thu, Jul 31, 2003 at 04:07:58PM +0100, Jamie Lokier wrote:
 
> Of course this is not a problem when "lock;cmpxchg" is used only for thread
> synchronisation on uniprocessor 386s...  The lock prefix is irrelevant then.

This is exactly what it written in configure.help ;-)

> Perhaps the emulation should refuse to pretend to work on an SMP 386 :)

I've seen such a config once. In fact, it was a board with two i376 (the 386
equivalent which does protected mode only, no M86 nor real mode). But I never
knew if they were connected to the same bus or totally independant.

Cheers,
Willy

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1338 bytes --]

On Thu, 2003-07-31 16:12:26 +0100, Jamie Lokier <jamie@shareable.org>
wrote in message <20030731151226.GG6410@mail.jlokier.co.uk>:
> Alan Cox wrote:
> > On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> > > See? It's loaded at the "ls" call, but it seems to be not loaded for
> > > apt-get.
> > 
> > Remember you need to overload signal setting functions like sigaction.
> > My guess is apt decided to disable your signal and you didnt stop it
> 
> An application might install its own SIGILL handler to emulate or trap
> _other_ instructions.  To do it properly, you have to chain the handlers.
> 
> Not sure how to do this, when you get to the stage of two LD_PRELOAD
> libraries each wanting to overload sigaction.

That's not (yet) my problem and I think it's not impossible to hook
them. _But_ before, I need to get called at all _before_ libstdc++5's
_init(). For now, I haven't managed to do that...

My hack^H^H^H^Hsources are at
http://www.lug-owl.de/~jbglaw/software/catch_sigill/ .

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1950 bytes --]

On Thu, 2003-07-31 17:09:19 +0200, Martin Schlemmer <azarah@gentoo.org>
wrote in message <1059664158.5031.26.camel@workshop.saharacpt.lan>:
> On Thu, 2003-07-31 at 15:01, Jan-Benedict Glaw wrote:
> > On Thu, 2003-07-31 14:14:48 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de>
> > wrote in message <20030731121448.GW1873@lug-owl.de>:
> > > On Thu, 2003-07-31 12:51:09 +0100, Alan Cox <alan@lxorguk.ukuu.org.uk>
> > > wrote in message <1059652268.16608.8.camel@dhcp22.swansea.linux.org.uk>:
> > > > On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> > So there we are. Thanks to someone who pointed me to LD_DEBUG et al., I
> > see that my _init() is called too late:
> > 
> > amtus:~/sigill_catcher# LD_DEBUG=all LD_VERBOSE=1
> > LD_PRELOAD=./libsigill.so apt-get update 2> ld_infos                                 
> > Illegal instruction
> > amtus:~/sigill_catcher# grep 'calling init' ld_infos 
> > 00691:  calling init: /lib/libc.so.6
> > 00691:  calling init: /lib/libdl.so.2
> > 00691:  calling init: /lib/libgcc_s.so.1
> > 00691:  calling init: /lib/libm.so.6
> > 00691:  calling init: /usr/lib/libstdc++.so.5
> > 
> > As I guessed, libstdc++5's _init() is called before mine (LD_PRELOADed)
> > _init() function and thus, I cannot intercept this SIGILL, as it
> > seems...
> > 
> 
> You could do what we do with our path sandbox - basically you
> use a wrapper that setup the environment (prob not needed in
> your case), and then spawn bash with our sandbox lib preloaded,
> which then calls whatever should be 'path sandboxed'.

Care to send some URL? I haven't found it ad hoc by Google...

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Richard B. Johnson]

On Thu, 31 Jul 2003, Jamie Lokier wrote:

> Willy Tarreau wrote:
> > The other problem lies with the lock :
> > When a 486 executes "LOCK ; CMPXCHG", it locks the bus during the whole cmpxchg
> > instruction. If a 386 executes the same code, it will get an exception which
> > will be caught by the emulator. I don't see how we can do such an atomic
> > operation while holding a lock. At best, we would think about a global memory
> > based shared lock during the operation (eg: int bus_lock;), but it's not
> > implemented at the moment, and will only be compatible with processors sharing
> > the same code. Add-on processors, such as co-processors, transputer cards, or
> > DSPs, will know nothing about such a lock emulation. And it would result in
> > even poorer performance of course !
>
> Of course this is not a problem when "lock;cmpxchg" is used only for thread
> synchronisation on uniprocessor 386s...  The lock prefix is irrelevant then.
>
> Perhaps the emulation should refuse to pretend to work on an SMP 386 :)
>
> -- Jamie
> -

You can use the lock instruction ahead of any 386 instruction
without creating an exception. When relevent, it locks the whole
bus. When not, it's just a no-op. The trap on the lock instruction
came with the '486. With the '486, if the instruction doesn't
modify memory, then the lock prefix is invalid and will generate
an invalid-opcode exception.

It is not correct to use a lock instruction in front of every
op-code of course, and it might not have been tested for all
corner cases, but generally it's harmless on a '386.

The bad op-code for the i386 is cmpxchg. This is what triggers
the trap. This can be emulated, although the emulation is
not SMP compatible. You worry about this when somebody makes
a dual '386 machine ;^).  Also, the best performing emulation
for any op-codes should be done within the kernel. That way,
the invalid-opcode trap works just like the math emulator. You
don't need the overhead of calling a user-mode handler. If
this is emulation is implimented, then one should also emulate
BSWAP and XADD. This makes '486 code compatible with '386
machines.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: TSCs are a no-no on i386

[Alan Cox]

On Iau, 2003-07-31 at 16:10, Jamie Lokier wrote:
> > And if the byte you are looking at was patched by another thread you've
> > blown it. Your emulation can only be so good 8) People do stuff like
> > patching instructions under software decode as a robustness check - its
> > normally pretty amusing
> 
> On a uniprocessor 386, this is not a problem.  Just disable preemption
> in the kernel decoder.

Wrong again. Thats a common myth but you see I can put the instruction on a page
so that its executed from a page I just did a read() into in another process. If
I'm really bored I'll use O_DIRECT but thats mostly for makign life really bad 
for non cache coherent setups 8)

Re: TSCs are a no-no on i386

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1427 bytes --]

On Thu, 2003-07-31 11:50:27 -0400, Richard B. Johnson <root@chaos.analogic.com>
wrote in message <Pine.LNX.4.53.0307311126530.770@chaos>:
> On Thu, 31 Jul 2003, Jamie Lokier wrote:
> > Willy Tarreau wrote:

> The bad op-code for the i386 is cmpxchg. This is what triggers
> the trap. This can be emulated, although the emulation is
> not SMP compatible. You worry about this when somebody makes
> a dual '386 machine ;^).  Also, the best performing emulation
> for any op-codes should be done within the kernel. That way,
> the invalid-opcode trap works just like the math emulator. You
> don't need the overhead of calling a user-mode handler. If
> this is emulation is implimented, then one should also emulate
> BSWAP and XADD. This makes '486 code compatible with '386
> machines.

Thanks for this fine explanation! Maybe I'd take the 2.4.x emulator
kernel patch and bring it in line with 2.6.x?

However, that doesn't release gcc's developers from providing a solution
to the initial problem (i386 and i486 compiled libs/programs
incompatible) from the task to work on libstdc++ :->

MfG, JBG

-- 
   Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481
   "Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg
    fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!
      ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: TSCs are a no-no on i386

[Jamie Lokier]

Alan Cox wrote:
> > > And if the byte you are looking at was patched by another thread you've
> > > blown it. Your emulation can only be so good 8) People do stuff like
> > > patching instructions under software decode as a robustness check - its
> > > normally pretty amusing
> > 
> > On a uniprocessor 386, this is not a problem.  Just disable preemption
> > in the kernel decoder.
> 
> Wrong again. Thats a common myth but you see I can put the instruction on a page
> so that its executed from a page I just did a read() into in another process. If
> I'm really bored I'll use O_DIRECT but thats mostly for makign life really bad 
> for non cache coherent setups 8)

I wouldn't be surprised if even real CPUs mis-decode when there's a
concurrent DMA into the area they are reading instructions from! :)

-- Jamie

Re: TSCs are a no-no on i386

[Alan Cox]

On Iau, 2003-07-31 at 19:37, Jamie Lokier wrote:
> I wouldn't be surprised if even real CPUs mis-decode when there's a
> concurrent DMA into the area they are reading instructions from! :)

I believe there is a PIII errata about putting an instruction across 
a memory type boundary and executingit in a loop

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Martin Schlemmer]

On Thu, 2003-07-31 at 17:33, Jan-Benedict Glaw wrote:
> On Thu, 2003-07-31 17:09:19 +0200, Martin Schlemmer <azarah@gentoo.org>
> wrote in message <1059664158.5031.26.camel@workshop.saharacpt.lan>:
> > On Thu, 2003-07-31 at 15:01, Jan-Benedict Glaw wrote:
> > > On Thu, 2003-07-31 14:14:48 +0200, Jan-Benedict Glaw <jbglaw@lug-owl.de>
> > > wrote in message <20030731121448.GW1873@lug-owl.de>:
> > > > On Thu, 2003-07-31 12:51:09 +0100, Alan Cox <alan@lxorguk.ukuu.org.uk>
> > > > wrote in message <1059652268.16608.8.camel@dhcp22.swansea.linux.org.uk>:
> > > > > On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> > > So there we are. Thanks to someone who pointed me to LD_DEBUG et al., I
> > > see that my _init() is called too late:
> > > 
> > > amtus:~/sigill_catcher# LD_DEBUG=all LD_VERBOSE=1
> > > LD_PRELOAD=./libsigill.so apt-get update 2> ld_infos                                 
> > > Illegal instruction
> > > amtus:~/sigill_catcher# grep 'calling init' ld_infos 
> > > 00691:  calling init: /lib/libc.so.6
> > > 00691:  calling init: /lib/libdl.so.2
> > > 00691:  calling init: /lib/libgcc_s.so.1
> > > 00691:  calling init: /lib/libm.so.6
> > > 00691:  calling init: /usr/lib/libstdc++.so.5
> > > 
> > > As I guessed, libstdc++5's _init() is called before mine (LD_PRELOADed)
> > > _init() function and thus, I cannot intercept this SIGILL, as it
> > > seems...
> > > 
> > 
> > You could do what we do with our path sandbox - basically you
> > use a wrapper that setup the environment (prob not needed in
> > your case), and then spawn bash with our sandbox lib preloaded,
> > which then calls whatever should be 'path sandboxed'.
> 
> Care to send some URL? I haven't found it ad hoc by Google...
> 

http://www.gentoo.org/cgi-bin/viewcvs.cgi/portage/src/sandbox-1.1/?cvsroot=gentoo-src


Cheers,

-- 
Martin Schlemmer
Gentoo Linux Developer, Desktop Team
Cape Town, South Africa

Re: TSCs are a no-no on i386

[Pavel Machek]

Hi!

> >   Note that this can also allow Step-A 486's to correctly run multi-thread
> >   applications since cmpxchg has a wrong opcode on this early CPU.
> > 
> >   Don't use this to enable multi-threading on an SMP machine, the lock
> >   atomicity can't be guaranted!
> 
> That is of course the other problem with this approach - you can't
> really get the intended results without some extremely heavyweight code
> (using an IPI to halt all CPU's, doing the access and then resuming
> them)

Hopefully there are not too many SMP-486 machines out there ;-).
								Pavel
-- 
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]

Re: TSCs are a no-no on i386

[Maciej W. Rozycki]

On Wed, 6 Aug 2003, Pavel Machek wrote:

> > >   Note that this can also allow Step-A 486's to correctly run multi-thread
> > >   applications since cmpxchg has a wrong opcode on this early CPU.
> > > 
> > >   Don't use this to enable multi-threading on an SMP machine, the lock
> > >   atomicity can't be guaranted!
> > 
> > That is of course the other problem with this approach - you can't
> > really get the intended results without some extremely heavyweight code
> > (using an IPI to halt all CPU's, doing the access and then resuming
> > them)
> 
> Hopefully there are not too many SMP-486 machines out there ;-).

 Step-A i486 processors are supposedly very rare and they were the
earliest ones (predating the DX suffix), so they were probably never used
for SMP systems.  And I think the i82489DX APIC was introduced a bit later
anyway -- we don't support non-APIC SMP implementations. 

 BTW, the step-A i486 reused opcodes of the famous i386's ibts and xbts
instructions.  Once the chips were out, Intel discovered there is actually
some software out there expecting ibts/xbts semantics with these opcodes,
so the encoding of cmpxchg was quickly changed. ;-)

-- 
+  Maciej W. Rozycki, Technical University of Gdansk, Poland   +
+--------------------------------------------------------------+
+        e-mail: macro@ds2.pg.gda.pl, PGP key available        +

Re: Emulating i486 on i386 (was: TSCs are a no-no on i386)

[Ben Pfaff]

Jan-Benedict Glaw <jbglaw@lug-owl.de> writes:

> On Thu, 2003-07-31 16:12:26 +0100, Jamie Lokier <jamie@shareable.org>
> wrote in message <20030731151226.GG6410@mail.jlokier.co.uk>:
> > Alan Cox wrote:
> > > On Iau, 2003-07-31 at 12:38, Jan-Benedict Glaw wrote:
> > > > See? It's loaded at the "ls" call, but it seems to be not loaded for
> > > > apt-get.
> > > 
> > > Remember you need to overload signal setting functions like sigaction.
> > > My guess is apt decided to disable your signal and you didnt stop it
> > 
> > An application might install its own SIGILL handler to emulate or trap
> > _other_ instructions.  To do it properly, you have to chain the handlers.
> > 
> > Not sure how to do this, when you get to the stage of two LD_PRELOAD
> > libraries each wanting to overload sigaction.
> 
> That's not (yet) my problem and I think it's not impossible to hook
> them. _But_ before, I need to get called at all _before_ libstdc++5's
> _init(). For now, I haven't managed to do that...

One way to get loaded before anything else is to do ELF binary
loading by hand in userspace.  The kernel exec loads your
program, which sets up trap handlers etc., then loads the actual
target binary by hand.  I did this for an application sandbox.
Writing the loader took less than half a day since I was able to
use a lot of kernel code from binfmt_elf.c directly, just
translating internal kernel calls into open, mmap, etc. system
calls.
-- 
"Note that nobody reads every post in linux-kernel.   In fact, nobody who
 expects to have time left over to  actually do any real kernel work will
 read even half.  Except Alan Cox, but he's actually not human, but about
 a thousand gnomes working in under-ground caves in Swansea." --Linus