Setting nx bit in virtual CPU

Setting nx bit in virtual CPU

[Richard Simpson]

Hello,

Summary: How can I have a virtual CPU with the nx bit set whilst
enjoying KVM acceleration?

My Host - AMD Athlon(tm) 64 Processor 3200+ running Gentoo
My VM - KVM running hardened Gentoo
My KVM version - 0.12.3
My Task - Implement restricted secure VM to handle services exposed to
internet.
My Command - kvm -hda /dev/mapper/vols-andrew -kernel ./bzImage -append
root=/dev/hda2 -cpu host -runas xxx -net nic -net user -m 256 -k en-gb
-vnc :1 -monitor stdio

In order to maximise the security of my VM, I have enabled PaX which is
supposed to prevent various address space attacks.  Sadly, when I run
'paxtest' it reports that my VM is still vulnerable.  I have concluded
that the problem is most likely caused by the virtual CPU not having the
nx bit set.

Flags in virtual CPU: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall mmxext fxsr_opt
lm rep_good pni cx16 lahf_lm

Flags in host CPU: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt
rdtscp lm 3dnowext 3dnow rep_good nopl pni cx16 lahf_lm svm extapic
cr8_legacy

As you can see, despite using the '-cpu host' command, several host
flags, including nx, are missing in the VM.  Setting '-cpu host,+nx'
doesn't make any difference.

If however, I remove the '-cpu host' option and add the '-no-kvm' option
the virtual CPU has the nx flag and paxtest reports that my VM is
secure.  Of course the down side is that everything runs much slower.

Confusingly, the following page about tuning KVM
(http://www.linux-kvm.org/page/Tuning_KVM) lists the flags for the
default qemu64 cpu and nx is clearly included.  But, when I set '-cpu
qemu64' I get a model name of QEMU Virtual CPU, but no sign of an nx bit.

So, is there any way of having the nx bit and the benefits of KVM
acceleration.

Thank you.

Re: Setting nx bit in virtual CPU

[Chris Wright]

* Richard Simpson (rs1002@huskydog.org.uk) wrote:
> So, is there any way of having the nx bit and the benefits of KVM
> acceleration.

WFM here (both current git tree and 0.12.3) w/ either -cpu host or -cpu
qemu64.  The code definitly does what you'd expect in both those cases.

thanks,
-chris

Re: Setting nx bit in virtual CPU

[Richard Simpson]

OK, thanks for that.  Clearly something wrong with my installation.  At
least now I know it is possible I can keep fiddling until it works.

Richard

On 30/03/10 03:12, Chris Wright wrote:
> * Richard Simpson (rs1002@huskydog.org.uk) wrote:
>> So, is there any way of having the nx bit and the benefits of KVM
>> acceleration.
> 
> WFM here (both current git tree and 0.12.3) w/ either -cpu host or -cpu
> qemu64.  The code definitly does what you'd expect in both those cases.
> 
> thanks,
> -chris

Re: Setting nx bit in virtual CPU

[Avi Kivity]

On 03/30/2010 01:16 AM, Richard Simpson wrote:
> Hello,
>
> Summary: How can I have a virtual CPU with the nx bit set whilst
> enjoying KVM acceleration?
>
> My Host - AMD Athlon(tm) 64 Processor 3200+ running Gentoo
> My VM - KVM running hardened Gentoo
> My KVM version - 0.12.3
> My Task - Implement restricted secure VM to handle services exposed to
> internet.
> My Command - kvm -hda /dev/mapper/vols-andrew -kernel ./bzImage -append
> root=/dev/hda2 -cpu host -runas xxx -net nic -net user -m 256 -k en-gb
> -vnc :1 -monitor stdio
>
>    


Are you running a 32-bit non-pae host kernel?  In that case, nx is 
disabled both for the guest and host.  Switch to a pae (or 64-bit) 
kernel and all should be well.

-- 
error compiling committee.c: too many arguments to function

Re: Setting nx bit in virtual CPU

[Richard Simpson]

Nope, both Kernels are 64 bit.

uname -a Host: Linux gordon 2.6.27-gentoo-r8 #5 Sat Mar 14 18:01:59 GMT
2009 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux

uname -a Guest: Linux andrew 2.6.28-hardened-r9 #4 Mon Jan 18 22:39:31
GMT 2010 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux

As you can see, both kernels are a little old, and I have been wondering
if that might be part of the problem.  The Guest one is old because that
is the latest stable hardened version in Gentoo.  The host one is old
because of:

(gordon:~) rs10% uptime
 22:01:37 up 374 days, 23:29,  1 user,  load average: 1.09, 0.42, 0.18

Now that I have managed to smash the psychologically important 1 year
uptime for the first time ever (Woo!) I shall probably upgrade the host
kernel in the near future.  Of course, it is important to remember that
with the --no-kvm switch it works just fine (only slowly) with exactly
the same two kernels.

Thanks

On 01/04/10 09:43, Avi Kivity wrote:
> On 03/30/2010 01:16 AM, Richard Simpson wrote:
>> Hello,
>>
>> Summary: How can I have a virtual CPU with the nx bit set whilst
>> enjoying KVM acceleration?
>>
>> My Host - AMD Athlon(tm) 64 Processor 3200+ running Gentoo
>> My VM - KVM running hardened Gentoo
>> My KVM version - 0.12.3
>> My Task - Implement restricted secure VM to handle services exposed to
>> internet.
>> My Command - kvm -hda /dev/mapper/vols-andrew -kernel ./bzImage -append
>> root=/dev/hda2 -cpu host -runas xxx -net nic -net user -m 256 -k en-gb
>> -vnc :1 -monitor stdio
>>
>>    
> 
> 
> Are you running a 32-bit non-pae host kernel?  In that case, nx is
> disabled both for the guest and host.  Switch to a pae (or 64-bit)
> kernel and all should be well.
> 

Re: Setting nx bit in virtual CPU

[Avi Kivity]

On 04/03/2010 12:07 AM, Richard Simpson wrote:
> Nope, both Kernels are 64 bit.
>
> uname -a Host: Linux gordon 2.6.27-gentoo-r8 #5 Sat Mar 14 18:01:59 GMT
> 2009 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux
>
> uname -a Guest: Linux andrew 2.6.28-hardened-r9 #4 Mon Jan 18 22:39:31
> GMT 2010 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux
>
> As you can see, both kernels are a little old, and I have been wondering
> if that might be part of the problem.  The Guest one is old because that
> is the latest stable hardened version in Gentoo.  The host one is old
> because of:
>    

2.6.27 should be plenty fine for nx.  Really the important bit is that 
the host kernel has nx enabled.  Can you check if that is so?

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: Setting nx bit in virtual CPU

[Richard Simpson]

On 05/04/10 09:27, Avi Kivity wrote:
> On 04/03/2010 12:07 AM, Richard Simpson wrote:
>> Nope, both Kernels are 64 bit.
>>
>> uname -a Host: Linux gordon 2.6.27-gentoo-r8 #5 Sat Mar 14 18:01:59 GMT
>> 2009 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux
>>
>> uname -a Guest: Linux andrew 2.6.28-hardened-r9 #4 Mon Jan 18 22:39:31
>> GMT 2010 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux
>>
>> As you can see, both kernels are a little old, and I have been wondering
>> if that might be part of the problem.  The Guest one is old because that
>> is the latest stable hardened version in Gentoo.  The host one is old
>> because of:
>>    
> 
> 2.6.27 should be plenty fine for nx.  Really the important bit is that
> the host kernel has nx enabled.  Can you check if that is so?
> 
Umm, could you give me a clue about how to do that.  It is some time
since I configured the host kernel, but I do have a /proc/config.gz.
Could I check by looking in that?

Thanks

Re: Setting nx bit in virtual CPU

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 512 bytes --]

On 04/07/2010 01:31 AM, Richard Simpson wrote:
>
>> 2.6.27 should be plenty fine for nx.  Really the important bit is that
>> the host kernel has nx enabled.  Can you check if that is so?
>>
>>      
> Umm, could you give me a clue about how to do that.  It is some time
> since I configured the host kernel, but I do have a /proc/config.gz.
> Could I check by looking in that?
>    

The attached script should verify it.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


[-- Attachment #2: check-nx --]
[-- Type: text/plain, Size: 497 bytes --]

#!/usr/bin/python

class msr(object):
    def __init__(self):
        try:
            self.f = file('/dev/cpu/0/msr')
        except:
            self.f = file('/dev/msr0')
    def read(self, index, default = None):
        import struct
        self.f.seek(index)
        try:
            return struct.unpack('Q', self.f.read(8))[0]
        except:
            return default

efer = msr().read(0xc0000080, 0)
nx = (efer >> 11) & 1

if nx:
   print 'nx: enabled'
else:
    print 'nx: disabled'

Re: Setting nx bit in virtual CPU

[Richard Simpson]

On 07/04/10 06:39, Avi Kivity wrote:
> On 04/07/2010 01:31 AM, Richard Simpson wrote:
>>
>>> 2.6.27 should be plenty fine for nx.  Really the important bit is that
>>> the host kernel has nx enabled.  Can you check if that is so?
>>>
>>>      
>> Umm, could you give me a clue about how to do that.  It is some time
>> since I configured the host kernel, but I do have a /proc/config.gz.
>> Could I check by looking in that?
>>    
> 
> The attached script should verify it.
> 

rs% ./check-nx
Traceback (most recent call last):
  File "./check-nx", line 17, in <module>
    efer = msr().read(0xc0000080, 0)
  File "./check-nx", line 8, in __init__
    self.f = file('/dev/msr0')
IOError: [Errno 2] No such file or directory: '/dev/msr0'

Sorry!

Re: Setting nx bit in virtual CPU

[Avi Kivity]

On 04/07/2010 03:10 PM, Richard Simpson wrote:
> On 07/04/10 06:39, Avi Kivity wrote:
>    
>> On 04/07/2010 01:31 AM, Richard Simpson wrote:
>>      
>>>        
>>>> 2.6.27 should be plenty fine for nx.  Really the important bit is that
>>>> the host kernel has nx enabled.  Can you check if that is so?
>>>>
>>>>
>>>>          
>>> Umm, could you give me a clue about how to do that.  It is some time
>>> since I configured the host kernel, but I do have a /proc/config.gz.
>>> Could I check by looking in that?
>>>
>>>        
>> The attached script should verify it.
>>
>>      
> rs% ./check-nx
> Traceback (most recent call last):
>    File "./check-nx", line 17, in<module>
>      efer = msr().read(0xc0000080, 0)
>    File "./check-nx", line 8, in __init__
>      self.f = file('/dev/msr0')
> IOError: [Errno 2] No such file or directory: '/dev/msr0'
>
>    

Run as root, please.  And check first that you have a file named 
/dev/cpu/0/msr.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: Setting nx bit in virtual CPU

[Richard Simpson]

On 07/04/10 13:23, Avi Kivity wrote:
> On 04/07/2010 03:10 PM, Richard Simpson wrote:
>> On 07/04/10 06:39, Avi Kivity wrote:
>>   
>>> On 04/07/2010 01:31 AM, Richard Simpson wrote:
>>>     
>>>>       
>>>>> 2.6.27 should be plenty fine for nx.  Really the important bit is that
>>>>> the host kernel has nx enabled.  Can you check if that is so?
>>> The attached script should verify it.
>> IOError: [Errno 2] No such file or directory: '/dev/msr0'
> 
> Run as root, please.  And check first that you have a file named
> /dev/cpu/0/msr.

Doh!

gordon Code # ./check-nx
nx: enabled
gordon Code #

OK, seems to be enabled just fine.  Any other ideas?  I am beginning to
get that horrible feeling that there isn't a real problem and it is just
me being dumb!

Re: Setting nx bit in virtual CPU

[Avi Kivity]

On 04/07/2010 11:38 PM, Richard Simpson wrote:
> On 07/04/10 13:23, Avi Kivity wrote:
>    
>> On 04/07/2010 03:10 PM, Richard Simpson wrote:
>>      
>>> On 07/04/10 06:39, Avi Kivity wrote:
>>>
>>>        
>>>> On 04/07/2010 01:31 AM, Richard Simpson wrote:
>>>>
>>>>          
>>>>>
>>>>>            
>>>>>> 2.6.27 should be plenty fine for nx.  Really the important bit is that
>>>>>> the host kernel has nx enabled.  Can you check if that is so?
>>>>>>              
>>>> The attached script should verify it.
>>>>          
>>> IOError: [Errno 2] No such file or directory: '/dev/msr0'
>>>        
>> Run as root, please.  And check first that you have a file named
>> /dev/cpu/0/msr.
>>      
> Doh!
>
> gordon Code # ./check-nx
> nx: enabled
> gordon Code #
>
> OK, seems to be enabled just fine.  Any other ideas?  I am beginning to
> get that horrible feeling that there isn't a real problem and it is just
> me being dumb!
>    

I really hope so, because I am out of ideas... :)

Can you verify check-nx returns disabled on the guest?
Does /proc/cpuinfo show nx in the guest?

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: Setting nx bit in virtual CPU

[Richard Simpson]

>> gordon Code # ./check-nx
>> nx: enabled
>> gordon Code #
>>
>> OK, seems to be enabled just fine.  Any other ideas?  I am beginning to
>> get that horrible feeling that there isn't a real problem and it is just
>> me being dumb!
>>
> I really hope so, because I am out of ideas... :)
> 
> Can you verify check-nx returns disabled on the guest?
> Does /proc/cpuinfo show nx in the guest?
> 

OK, time for a summary:

Host:  /proc/cpuinfo shows 'nx' and check-nx shows 'enabled'

Guest: /proc/cpuinfo doesn't show nx and check-nx shows 'disabled'

Guest (with -no-kvm option): /proc/cpuinfo shows 'nx', but check-nx
still shows 'disabled'

Below I have included all the listings which I think might be useful,
but if you would like to see anything else then please ask.

HOST:

/proc/cpuinfo

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 79
model name      : AMD Athlon(tm) 64 Processor 3200+
stepping        : 2
cpu MHz         : 1000.000
cache size      : 512 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt
rdtscp lm 3dnowext 3dnow rep_good nopl pni cx16 lahf_lm svm extapic
cr8_legacy
bogomips        : 2000.06
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc

GUEST with command line - kvm -hda /dev/mapper/vols-andrew -kernel
./bzImage -append root=/dev/hda2 -cpu host -runas xx -net nic -net user
-m 256 -k en-gb -vnc :1 -monitor stdio

/proc/cpuinfo

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 79
model name      : AMD Athlon(tm) 64 Processor 3200+
stepping        : 2
cpu MHz         : 10000.330
cache size      : 512 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall mmxext fxsr_opt lm
rep_good pni cx16 lahf_lm
bogomips        : 2000.06
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Results of paxtest

PaXtest - Copyright(c) 2003,2004 by Peter Busser <peter@adamantix.org>
Released under the GNU Public Licence version 2 or later

Mode: kiddie
Linux andrew 2.6.28-hardened-r9 #4 Mon Jan 18 22:39:31 GMT 2010 x86_64
AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux

Executable anonymous mapping             : Vulnerable
Executable bss                           : Vulnerable
Executable data                          : Vulnerable
Executable heap                          : Vulnerable
Executable stack                         : Vulnerable
Executable anonymous mapping (mprotect)  : Vulnerable
Executable bss (mprotect)                : Vulnerable
Executable data (mprotect)               : Vulnerable
Executable heap (mprotect)               : Vulnerable
Executable stack (mprotect)              : Vulnerable
Executable shared library bss (mprotect) : Vulnerable
Executable shared library data (mprotect): Vulnerable
Writable text segments                   : Killed
Anonymous mapping randomisation test     : 33 bits (guessed)
Heap randomisation test (ET_EXEC)        : 13 bits (guessed)
Heap randomisation test (ET_DYN)         : 40 bits (guessed)
Main executable randomisation (ET_EXEC)  : No randomisation
Main executable randomisation (ET_DYN)   : 12 bits (guessed)
Shared library randomisation test        : 33 bits (guessed)
Stack randomisation test (SEGMEXEC)      : 40 bits (guessed)
Stack randomisation test (PAGEEXEC)      : 40 bits (guessed)
Return to function (strcpy)              : paxtest: bad luck, try
different compiler options.
Return to function (memcpy)              : *** buffer overflow detected
***: rettofunc2 - terminated
rettofunc2: buffer overflow attack in function <unknown> - terminated
Report to http://bugs.gentoo.org/
Killed
Return to function (strcpy, RANDEXEC)    : paxtest: bad luck, try
different compiler options.
Return to function (memcpy, RANDEXEC)    : *** buffer overflow detected
***: rettofunc2x - terminated
rettofunc2x: buffer overflow attack in function <unknown> - terminated
Report to http://bugs.gentoo.org/
Killed
Executable shared library bss            : Killed
Executable shared library data           : Killed

GUEST with command line - kvm -hda /dev/mapper/vols-andrew -kernel
./bzImage -append root=/dev/hda2 -no-kvm -runas xx -net nic -net user -m
256 -k en-gb -vnc :1 -monitor stdio

/proc/cpuinfo

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 6
model           : 2
model name      : QEMU Virtual CPU version 0.12.3
stepping        : 3
cpu MHz         : 1998.067
cache size      : 512 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 4
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx lm pni cx16 lahf_lm
svm abm sse4a
bogomips        : 3996.13
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Results of paxtest

PaXtest - Copyright(c) 2003,2004 by Peter Busser <peter@adamantix.org>
Released under the GNU Public Licence version 2 or later

Mode: kiddie
Linux andrew 2.6.28-hardened-r9 #4 Mon Jan 18 22:39:31 GMT 2010 x86_64
QEMU Virtual CPU version 0.12.3 AuthenticAMD GNU/Linux

Executable anonymous mapping             : Killed
Executable bss                           : Killed
Executable data                          : Killed
Executable heap                          : Killed
Executable stack                         : Killed
Executable anonymous mapping (mprotect)  : Killed
Executable bss (mprotect)                : Killed
Executable data (mprotect)               : Killed
Executable heap (mprotect)               : Killed
Executable stack (mprotect)              : Killed
Executable shared library bss (mprotect) : Killed
Executable shared library data (mprotect): Killed
Writable text segments                   : Killed
Anonymous mapping randomisation test     : 33 bits (guessed)
Heap randomisation test (ET_EXEC)        : 13 bits (guessed)
Heap randomisation test (ET_DYN)         : 40 bits (guessed)
Main executable randomisation (ET_EXEC)  : No randomisation
Main executable randomisation (ET_DYN)   : 12 bits (guessed)
Shared library randomisation test        : 33 bits (guessed)
Stack randomisation test (SEGMEXEC)      : 40 bits (guessed)
Stack randomisation test (PAGEEXEC)      : 40 bits (guessed)
Return to function (strcpy)              : paxtest: bad luck, try
different compiler options.
Return to function (memcpy)              : *** buffer overflow detected
***: rettofunc2 - terminated
rettofunc2: buffer overflow attack in function <unknown> - terminated
Report to http://bugs.gentoo.org/
Killed
Return to function (strcpy, RANDEXEC)    : paxtest: bad luck, try
different compiler options.
Return to function (memcpy, RANDEXEC)    : *** buffer overflow detected
***: rettofunc2x - terminated
rettofunc2x: buffer overflow attack in function <unknown> - terminated
Report to http://bugs.gentoo.org/
Killed
Executable shared library bss            : Killed
Executable shared library data           : Killed

Re: Setting nx bit in virtual CPU

[Avi Kivity]

On 04/08/2010 02:13 AM, Richard Simpson wrote:
>    
>>> gordon Code # ./check-nx
>>> nx: enabled
>>> gordon Code #
>>>
>>> OK, seems to be enabled just fine.  Any other ideas?  I am beginning to
>>> get that horrible feeling that there isn't a real problem and it is just
>>> me being dumb!
>>>
>>>        
>> I really hope so, because I am out of ideas... :)
>>
>> Can you verify check-nx returns disabled on the guest?
>> Does /proc/cpuinfo show nx in the guest?
>>
>>      
> OK, time for a summary:
>
> Host:  /proc/cpuinfo shows 'nx' and check-nx shows 'enabled'
>
> Guest: /proc/cpuinfo doesn't show nx and check-nx shows 'disabled'
>    

Strange.  Can you hack qemu-kvm's cpuid code where it issues the ioctl 
KVM_SET_CPUID2 to show what the data is?  I'm not where that code is in 
your version of qemu-kvm.

-- 
error compiling committee.c: too many arguments to function

Re: Setting nx bit in virtual CPU

[Andre Przywara]

[-- Attachment #1: Type: text/plain, Size: 1062 bytes --]

Avi Kivity wrote:
> On 04/07/2010 11:38 PM, Richard Simpson wrote:
>> On 07/04/10 13:23, Avi Kivity wrote:
>>> Run as root, please.  And check first that you have a file named
>>> /dev/cpu/0/msr.
>>>      
>> Doh!
>>
>> gordon Code # ./check-nx
>> nx: enabled
>> gordon Code #
>>
>> OK, seems to be enabled just fine.  Any other ideas?  I am beginning to
>> get that horrible feeling that there isn't a real problem and it is just
>> me being dumb!
>>    
> 
> I really hope so, because I am out of ideas... :)
> 
> Can you verify check-nx returns disabled on the guest?
> Does /proc/cpuinfo show nx in the guest?

Can you try to boot the attached multiboot "kernel", which just outputs 
a brief CPUID dump?
$ qemu-kvm -kernel cpuid_mb -vnc :0
(Unfortunately I have no serial console support in there yet, so you 
either have to write the values down or screenshot it).
In the 4th line from the button it should print NX (after SYSCALL).

Regards,
Andre.

-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12

[-- Attachment #2: cpuid_mb --]
[-- Type: application/octet-stream, Size: 11744 bytes --]

Re: Setting nx bit in virtual CPU

[Richard Simpson]

[-- Attachment #1: Type: text/plain, Size: 591 bytes --]

On 08/04/10 09:52, Andre Przywara wrote:

> Can you try to boot the attached multiboot "kernel", which just outputs
> a brief CPUID dump?
> $ qemu-kvm -kernel cpuid_mb -vnc :0
> (Unfortunately I have no serial console support in there yet, so you
> either have to write the values down or screenshot it).
> In the 4th line from the button it should print NX (after SYSCALL).

OK, that was fun!  Resulting screen shots are attached.

...default.png		With command line above.
...cpu_host.png		With -cpu host option added.
...no_kvm.png		With -no-kvm option added.

I hope that helps!

Richard

[-- Attachment #2: cpuid_mb_screendump_cpu_host.png --]
[-- Type: image/png, Size: 8417 bytes --]

[-- Attachment #3: cpuid_mb_screendump_default.png --]
[-- Type: image/png, Size: 8030 bytes --]

[-- Attachment #4: cpuid_mb_screendump_no_kvm.png --]
[-- Type: image/png, Size: 8384 bytes --]

Re: Setting nx bit in virtual CPU

[Richard Simpson]

On 08/04/10 08:23, Avi Kivity wrote:

> Strange.  Can you hack qemu-kvm's cpuid code where it issues the ioctl
> KVM_SET_CPUID2 to show what the data is?  I'm not where that code is in
> your version of qemu-kvm.

Gad, the last time I tried to mess around with this sort of low level
code was many years ago when I was a keen young bachelor burning the
midnight oil trying to get the weird IDE controller on my Alpha to work
properly!  Anyway, I have tried to give it a go.

I found a file called qemu-kvm-x86.c

It contained a function called kvm_setup_cpuid2 which I modified as follows:

int kvm_setup_cpuid2(CPUState *env, int nent,
                     struct kvm_cpuid_entry2 *entries)
{
        struct kvm_cpuid2 *cpuid;
        int r, i;
        fprintf(stderr, "cpuid=nent %d\n", nent);
        for (i=0; i < nent; i++) {
                fprintf(stderr, "%x %x %x %x %x %x %x\n",
entries[i].function, entries[i].index, entries[i].flags, entries[i].eax,
entries[i].ebx, entries[i].ecx, entries[i].edx);
                }
        cpuid = qemu_malloc(sizeof(*cpuid) + nent * sizeof(*entries));

        cpuid->nent = nent;
        memcpy(cpuid->entries, entries, nent * sizeof(*entries));
        r = kvm_vcpu_ioctl(env, KVM_SET_CPUID2, cpuid);
        free(cpuid);
        return r;
}

So, basically I go round a loop and print out the contents of each
kvm_cpuid_entry2 structure.

Results below, using Andre Przywara's handy nano-kernel.  I do hope that
some of this makes some kind of sense!

qemu-kvm -kernel cpuid_mb -vnc :0

cpuid=nent 21
40000000 0 0 0 4b4d564b 564b4d56 4d
40000001 0 0 7 0 0 0
0 0 0 4 68747541 444d4163 69746e65
1 0 0 623 800 80002001 78bfbfd
2 0 0 1 0 0 2c307d
3 0 0 0 0 0 0
4 0 1 121 1c0003f 3f 1
4 1 1 122 1c0003f 3f 1
4 2 1 143 3c0003f fff 1
4 3 1 0 0 0 0
80000000 0 0 8000000a 68747541 444d4163 69746e65
80000001 0 0 623 0 1 2181abfd
80000002 0 0 554d4551 72695620 6c617574 55504320
80000003 0 0 72657620 6e6f6973 312e3020 332e32
80000004 0 0 0 0 0 0
80000005 0 0 1ff01ff 1ff01ff 40020140 40020140
80000006 0 0 0 42004200 2008140 0
80000007 0 0 0 0 0 0
80000008 0 0 3028 0 0 0
80000009 0 0 0 0 0 0
8000000a 0 0 1 10 0 0

qemu-kvm -kernel cpuid_mb -cpu host -vnc :0

cpuid=nent 29
40000000 0 0 0 4b4d564b 564b4d56 4d
40000001 0 0 7 0 0 0
0 0 0 1 68747541 444d4163 69746e65
1 0 0 40ff2 800 80002001 78bfbff
80000000 0 0 80000018 68747541 444d4163 69746e65
80000001 0 0 40ff2 0 1 23c3fbff
80000002 0 0 20444d41 6c687441 74286e6f 3620296d
80000003 0 0 72502034 7365636f 20726f73 30303233
80000004 0 0 2b 0 0 0
80000005 0 0 1ff01ff 1ff01ff 40020140 40020140
80000006 0 0 0 42004200 2008140 0
80000007 0 0 0 0 0 0
80000008 0 0 3028 0 0 0
80000009 0 0 0 0 0 0
8000000a 0 0 1 10 0 0
8000000b 0 0 0 0 0 0
8000000c 0 0 0 0 0 0
8000000d 0 0 0 0 0 0
8000000e 0 0 0 0 0 0
8000000f 0 0 0 0 0 0
80000010 0 0 0 0 0 0
80000011 0 0 0 0 0 0
80000012 0 0 0 0 0 0
80000013 0 0 0 0 0 0
80000014 0 0 0 0 0 0
80000015 0 0 0 0 0 0
80000016 0 0 0 0 0 0
80000017 0 0 0 0 0 0
80000018 0 0 0 0 0 0

If I try with -no-kvm then nothing gets printed, presumably because this
is a kvm specific function and doesn't get called in that case.

Re: Setting nx bit in virtual CPU

[Andre Przywara]

Richard Simpson wrote:
> On 08/04/10 09:52, Andre Przywara wrote:
> 
>> Can you try to boot the attached multiboot "kernel", which just outputs
>> a brief CPUID dump?
>> $ qemu-kvm -kernel cpuid_mb -vnc :0
>> (Unfortunately I have no serial console support in there yet, so you
>> either have to write the values down or screenshot it).
>> In the 4th line from the button it should print NX (after SYSCALL).
> 
> OK, that was fun!  Resulting screen shots are attached.
> 
> ...default.png		With command line above.
> ...cpu_host.png		With -cpu host option added.
> ...no_kvm.png		With -no-kvm option added.
> 
> I hope that helps!

OK, AFAIK there are several flags missing. I dimly remember there was a 
bug with masking the CPUID bits in older kernels, so I guess you have to 
celebrate your uptime for the last time and then give it a reboot with a 
more up-to-date host kernel.
(I also rebooted my desktop after I made the one year and now am gone 
green with turning it off over night ;-)
Maybe you get around with rebuilding fixed versions of kvm.ko and 
kvm_amd.ko, I can provide a fix for you if you wish (please point me to 
a way to get the actual kernel source you use).
The userspace was up-to-date? (qemu-kvm 0.12.3)?

Regards,
Andre.

-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 488-3567-12

Re: Setting nx bit in virtual CPU

[Avi Kivity]

On 04/09/2010 02:55 AM, Richard Simpson wrote:
> On 08/04/10 08:23, Avi Kivity wrote:
>
>    
>> Strange.  Can you hack qemu-kvm's cpuid code where it issues the ioctl
>> KVM_SET_CPUID2 to show what the data is?  I'm not where that code is in
>> your version of qemu-kvm.
>>      
>
> So, basically I go round a loop and print out the contents of each
> kvm_cpuid_entry2 structure.
>
> Results below, using Andre Przywara's handy nano-kernel.  I do hope that
> some of this makes some kind of sense!
>
> qemu-kvm -kernel cpuid_mb -vnc :0
>
>
> 80000000 0 0 8000000a 68747541 444d4163 69746e65
> 80000001 0 0 623 0 1 2181abfd
>    

80000001 edx bit 20 is NX, which is cleared.  So it is qemu-kvm at fault 
here.

-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

Re: Setting nx bit in virtual CPU

[Richard Simpson]

On 10/04/10 00:45, Andre Przywara wrote:
> Richard Simpson wrote:
>> On 08/04/10 09:52, Andre Przywara wrote:

> OK, AFAIK there are several flags missing. I dimly remember there was a
> bug with masking the CPUID bits in older kernels, so I guess you have to
> celebrate your uptime for the last time and then give it a reboot with a
> more up-to-date host kernel.
> (I also rebooted my desktop after I made the one year and now am gone
> green with turning it off over night ;-)
> Maybe you get around with rebuilding fixed versions of kvm.ko and
> kvm_amd.ko, I can provide a fix for you if you wish (please point me to
> a way to get the actual kernel source you use).
> The userspace was up-to-date? (qemu-kvm 0.12.3)?

Well, its funny how these things turn out.  On Saturday, my better half
insisted that I replace a mains socket.  It wasn't on the same ring main
as the server so I just switched off the relevant circuit breaker and
got on with it.  Unfortunately, whilst I was doing the work the main RCD
tripped, probably because I let the Neutral and Earth touch which can
cause that to happen sometimes.  Anyway, by the time I realised there
was a problem the old batteries in my UPS had given up and my uptime was
over with a grand total of 382 days, 12:07:24!  If this had happened a
few weeks ago I would probably have thrown myself from the roof!  In my
previous house, I installed a special circuit for the server not
protected by an RCD especially to avoid problems of this sort.  Adding
one in my current house has now moved a long way up my "To Do" list!

The server has started again, but a few things still aren't happy.  As
soon as I have them sorted I will step up to the latest kernel and let
you know what difference it makes.

Thanks,

	Richard