[Bug 1877794] [NEW] Constant Folding on 64-bit Subtraction causes SIGILL on linux-user glxgears ppc64le to x86_64 by way of generating bad shift instruction with c=-1

* [Bug 1877794] [NEW] Constant Folding on 64-bit Subtraction causes SIGILL on linux-user glxgears ppc64le to x86_64 by way of generating bad shift instruction with c=-1
@ 2020-05-09 21:55 Catherine A. Frederick
  2020-05-10  8:33 ` [Bug 1877794] " Laurent Vivier
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Catherine A. Frederick @ 2020-05-09 21:55 UTC (permalink / raw)
  To: qemu-devel

Public bug reported:

Hello, I've been recently working on my own little branch of QEMU
implementing the drm IOCTLs, when I discovered that glxgears seems to
crash in GLXSwapBuffers(); with a SIGILL. I investigated this for about
2 weeks, manually trying to trace the call stack, only to find that we
seemingly crash in a bad shift instruction. Originally intended to be an
shr_i64 generated to an RLDICL, we end up with an all ones(-1) c value,
which gets thrown to the macro for generating the MB, and replaces the
instruction with mostly ones. This new instruction, FFFFFFE0 is invalid
on ppc64le, and crashes in a host SIGILL in codegen_buffer. I tried to
see if the output of translate.c had this bad instruction, but all I got
were two (shr eax, cl) instructions, and upon creating a test program
with shr (eax, cl) in it, nothing happened. Then figuring that there was
nothing actually wrong with the instruction in the first place, I turned
my eye to the optimizer, and completely disabled constant folding for
arithmetic instructions.  This seemed to actually resolve the issue, and
then I slowly enabled constant folding again on various instructions
only to find that enabling not on the shifts, but on subtraction seemed
to cause the bug to reappear. I am bewildered and frankly at this point
I'm not sure I have a chance in hell of figuring out what causes it, so
I'm throwing it here.

** Affects: qemu
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1877794

Title:
  Constant Folding on 64-bit Subtraction causes SIGILL on linux-user
  glxgears ppc64le to x86_64 by way of generating bad shift instruction
  with c=-1

Status in QEMU:
  New

Bug description:
  Hello, I've been recently working on my own little branch of QEMU
  implementing the drm IOCTLs, when I discovered that glxgears seems to
  crash in GLXSwapBuffers(); with a SIGILL. I investigated this for
  about 2 weeks, manually trying to trace the call stack, only to find
  that we seemingly crash in a bad shift instruction. Originally
  intended to be an shr_i64 generated to an RLDICL, we end up with an
  all ones(-1) c value, which gets thrown to the macro for generating
  the MB, and replaces the instruction with mostly ones. This new
  instruction, FFFFFFE0 is invalid on ppc64le, and crashes in a host
  SIGILL in codegen_buffer. I tried to see if the output of translate.c
  had this bad instruction, but all I got were two (shr eax, cl)
  instructions, and upon creating a test program with shr (eax, cl) in
  it, nothing happened. Then figuring that there was nothing actually
  wrong with the instruction in the first place, I turned my eye to the
  optimizer, and completely disabled constant folding for arithmetic
  instructions.  This seemed to actually resolve the issue, and then I
  slowly enabled constant folding again on various instructions only to
  find that enabling not on the shifts, but on subtraction seemed to
  cause the bug to reappear. I am bewildered and frankly at this point
  I'm not sure I have a chance in hell of figuring out what causes it,
  so I'm throwing it here.

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1877794/+subscriptions

^ permalink raw reply	[flat|nested] 6+ messages in thread