Re: FreeBSD PVHVM call for testing

Re: FreeBSD PVHVM call for testing

[Michael Sierchio]

[-- Attachment #1.1: Type: text/plain, Size: 671 bytes --]

On Mon, May 13, 2013 at 11:32 AM, Roger Pau Monné <roger.pau@citrix.com>
 wrote:

> Hello,
>
> Recently Justin T Gibbs, Will Andrews and myself have been working on
> improving the Xen support in FreeBSD. The main goal of this was to bring
> full PVHVM support to FreeBSD, right now FreeBSD is only using PV
> interfaces for disk and network interfaces when running as a HVM guest. ...


I think should be encouraged.  We're eagerly awaiting the ability to run
FreeBSD in AWS in something other than t1.micro or cluster compute
instances.  Should we keep holding out hope, or will AWS make HVM available
for all instance types before this happens?

-  M

[-- Attachment #1.2: Type: text/html, Size: 1045 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/13/13 11:52, Michael Sierchio wrote:
> I think should be encouraged.  We're eagerly awaiting the ability to run
> FreeBSD in AWS in something other than t1.micro or cluster compute
> instances.  Should we keep holding out hope, or will AWS make HVM available
> for all instance types before this happens?

Err... my AMIs run on all EC2 instance types.  On some you have to pay the
Windows rate, that's all.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Re: FreeBSD PVHVM call for testing

[Michael Sierchio]

[-- Attachment #1.1: Type: text/plain, Size: 1018 bytes --]

The Windoze tax is unacceptable for a number of reasons - the primary
reason is that I'm not running Windows.  I don't think the licensing scheme
is unfair for those actually running Windows, mind you.

At the AWS Summit, an assertion was made that HVM support might be coming
soon for all instance types for *NIX OSes.  I hope that's true.

- M


On Mon, May 13, 2013 at 11:59 AM, Colin Percival <cperciva@freebsd.org>wrote:

> On 05/13/13 11:52, Michael Sierchio wrote:
> > I think should be encouraged.  We're eagerly awaiting the ability to run
> > FreeBSD in AWS in something other than t1.micro or cluster compute
> > instances.  Should we keep holding out hope, or will AWS make HVM
> available
> > for all instance types before this happens?
>
> Err... my AMIs run on all EC2 instance types.  On some you have to pay the
> Windows rate, that's all.
>
> --
> Colin Percival
> Security Officer Emeritus, FreeBSD | The power to serve
> Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid
>

[-- Attachment #1.2: Type: text/html, Size: 1584 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/13/13 12:08, Michael Sierchio wrote:
> The Windoze tax is unacceptable for a number of reasons - the primary reason is
> that I'm not running Windows.  I don't think the licensing scheme is unfair for
> those actually running Windows, mind you.

Right, it's definitely annoying having to pay more -- I just wanted to point out
that the ability does exist, if you're willing to pay the price.

> At the AWS Summit, an assertion was made that HVM support might be coming soon
> for all instance types for *NIX OSes.  I hope that's true.

Was it indeed?  I must not have been present for that... it certainly would be
good news.  Certainly all the new instance types they've released in the past
few years have had UNIX HVM support.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Re: FreeBSD PVHVM call for testing

[George Dunlap]

On Mon, May 13, 2013 at 7:32 PM, Roger Pau Monné <roger.pau@citrix.com> wrote:
> Hello,
>
> Recently Justin T Gibbs, Will Andrews and myself have been working on
> improving the Xen support in FreeBSD. The main goal of this was to bring
> full PVHVM support to FreeBSD, right now FreeBSD is only using PV
> interfaces for disk and network interfaces when running as a HVM guest.
> The main benefits of this changes are that Xen virtual interrupts (event
> channels) are now delivered to the guest using a vector callback
> injection, that is a per-cpu mechanism that allows each vCPU to have
> different interrupts assigned, so for example network and disk
> interrupts are delivered to different vCPUs in order to improve
> performance. With this changes FreeBSD also uses PV timers when running
> as an HVM guest, which should provide better time keeping and reduce the
> virtualization overhead, since emulated timers are no longer used. PV
> IPIs can also be used inside a HVM guest, but this will be implemented
> later.
>
> Right now the code is in a state where it can be tested by users, so we
> would like to encourage FreeBSD and Xen users to test it and provide
> feedback.

Is this something we should try to put on the Xen.org blog?

 -George

Re: FreeBSD PVHVM call for testing

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1247 bytes --]

On mar, 2013-05-14 at 10:19 +0100, George Dunlap wrote:
> On Mon, May 13, 2013 at 7:32 PM, Roger Pau Monné <roger.pau@citrix.com> wrote:
> > Hello,
> >
> > Recently Justin T Gibbs, Will Andrews and myself have been working on
> > improving the Xen support in FreeBSD. The main goal of this was to bring
> > full PVHVM support to FreeBSD, right now FreeBSD is only using PV
> > interfaces for disk and network interfaces when running as a HVM guest.
>
> [..]
>
> > Right now the code is in a state where it can be tested by users, so we
> > would like to encourage FreeBSD and Xen users to test it and provide
> > feedback.
> 
Cool! :-)

> Is this something we should try to put on the Xen.org blog?
> 
I think it definitely should... Whoever is up to write a blog post about
it, please, get in touch to me.

Soon we'll have the new mailing lists and all the stuff, but for now,
just drop me a line, and I can put the post in the pipeline.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/13/13 11:32, Roger Pau Monné wrote:
> Right now the code is in a state where it can be tested by users, so we
> would like to encourage FreeBSD and Xen users to test it and provide
> feedback.
> 
> The code is available in the following git repository, under the branch
> pvhvm_v5:
> 
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary
> 
> Also, I've created a wiki page that explains how to set up a FreeBSD
> PVHVM for testing:
> 
> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM

I built a XENHVM kernel with this code on EC2, and it hanged after
> xenbusb_front0: <Xen Frontend Devices> on xenstore0

With a XENHVM kernel from FreeBSD HEAD the next line after that is
> xbd0: 10240MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0

Any ideas?

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 16/05/13 19:55, Colin Percival wrote:
> On 05/13/13 11:32, Roger Pau Monné wrote:
>> Right now the code is in a state where it can be tested by users, so we
>> would like to encourage FreeBSD and Xen users to test it and provide
>> feedback.
>>
>> The code is available in the following git repository, under the branch
>> pvhvm_v5:
>>
>> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary
>>
>> Also, I've created a wiki page that explains how to set up a FreeBSD
>> PVHVM for testing:
>>
>> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM
> 
> I built a XENHVM kernel with this code on EC2, and it hanged after
>> xenbusb_front0: <Xen Frontend Devices> on xenstore0
> 
> With a XENHVM kernel from FreeBSD HEAD the next line after that is
>> xbd0: 10240MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
> 
> Any ideas?

Hello Colin,

Thanks for testing this on EC2, could you post the full dmesg? So I can
see the hypervisor version and if the PV timer is loaded or not.

Roger.

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/16/13 17:43, Roger Pau Monné wrote:
> Thanks for testing this on EC2, could you post the full dmesg? So I can
> see the hypervisor version and if the PV timer is loaded or not.

Here's what I get on a cc2.8xlarge with boot_verbose=YES:

> Booting [/boot/kernel/kernel]...               
> -\|/-\|GDB: no debug ports present
> KDB: debugger backends: ddb
> KDB: current backend: ddb
> SMAP type=01 base=0000000000000000 len=000000000009fc00
> SMAP type=02 base=000000000009fc00 len=0000000000000400
> SMAP type=02 base=00000000000e0000 len=0000000000020000
> SMAP type=01 base=0000000000100000 len=00000000bff00000
> SMAP type=02 base=00000000fc000000 len=0000000004000000
> SMAP type=01 base=0000000100000000 len=0000000e63000000
> Table 'FACP' at 0xfc005ee0
> Table 'APIC' at 0xfc005fe0
> APIC: Found table at 0xfc005fe0
> APIC: Using the MADT enumerator.
> MADT: Found CPU APIC ID 0 ACPI ID 0: enabled
> SMP: Added CPU 0 (AP)
> MADT: Found CPU APIC ID 2 ACPI ID 1: enabled
> SMP: Added CPU 2 (AP)
> MADT: Found CPU APIC ID 4 ACPI ID 2: enabled
> SMP: Added CPU 4 (AP)
> MADT: Found CPU APIC ID 6 ACPI ID 3: enabled
> SMP: Added CPU 6 (AP)
> MADT: Found CPU APIC ID 8 ACPI ID 4: enabled
> SMP: Added CPU 8 (AP)
> MADT: Found CPU APIC ID 10 ACPI ID 5: enabled
> SMP: Added CPU 10 (AP)
> MADT: Found CPU APIC ID 12 ACPI ID 6: enabled
> SMP: Added CPU 12 (AP)
> MADT: Found CPU APIC ID 14 ACPI ID 7: enabled
> SMP: Added CPU 14 (AP)
> MADT: Found CPU APIC ID 32 ACPI ID 8: enabled
> SMP: Added CPU 32 (AP)
> MADT: Found CPU APIC ID 34 ACPI ID 9: enabled
> SMP: Added CPU 34 (AP)
> MADT: Found CPU APIC ID 36 ACPI ID 10: enabled
> SMP: Added CPU 36 (AP)
> MADT: Found CPU APIC ID 38 ACPI ID 11: enabled
> SMP: Added CPU 38 (AP)
> MADT: Found CPU APIC ID 40 ACPI ID 12: enabled
> SMP: Added CPU 40 (AP)
> MADT: Found CPU APIC ID 42 ACPI ID 13: enabled
> SMP: Added CPU 42 (AP)
> MADT: Found CPU APIC ID 44 ACPI ID 14: enabled
> SMP: Added CPU 44 (AP)
> MADT: Found CPU APIC ID 46 ACPI ID 15: enabled
> SMP: Added CPU 46 (AP)
> MADT: Found CPU APIC ID 1 ACPI ID 16: enabled
> SMP: Added CPU 1 (AP)
> MADT: Found CPU APIC ID 3 ACPI ID 17: enabled
> SMP: Added CPU 3 (AP)
> MADT: Found CPU APIC ID 5 ACPI ID 18: enabled
> SMP: Added CPU 5 (AP)
> MADT: Found CPU APIC ID 7 ACPI ID 19: enabled
> SMP: Added CPU 7 (AP)
> MADT: Found CPU APIC ID 9 ACPI ID 20: enabled
> SMP: Added CPU 9 (AP)
> MADT: Found CPU APIC ID 11 ACPI ID 21: enabled
> SMP: Added CPU 11 (AP)
> MADT: Found CPU APIC ID 13 ACPI ID 22: enabled
> SMP: Added CPU 13 (AP)
> MADT: Found CPU APIC ID 15 ACPI ID 23: enabled
> SMP: Added CPU 15 (AP)
> MADT: Found CPU APIC ID 33 ACPI ID 24: enabled
> SMP: Added CPU 33 (AP)
> MADT: Found CPU APIC ID 35 ACPI ID 25: enabled
> SMP: Added CPU 35 (AP)
> MADT: Found CPU APIC ID 37 ACPI ID 26: enabled
> SMP: Added CPU 37 (AP)
> MADT: Found CPU APIC ID 39 ACPI ID 27: enabled
> SMP: Added CPU 39 (AP)
> MADT: Found CPU APIC ID 41 ACPI ID 28: enabled
> SMP: Added CPU 41 (AP)
> MADT: Found CPU APIC ID 43 ACPI ID 29: enabled
> SMP: Added CPU 43 (AP)
> MADT: Found CPU APIC ID 45 ACPI ID 30: enabled
> SMP: Added CPU 45 (AP)
> MADT: Found CPU APIC ID 47 ACPI ID 31: enabled
> SMP: Added CPU 47 (AP)
> Copyright (c) 1992-2013 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> 	The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 10.0-CURRENT #0 r+7c97e5b: Fri May 17 02:38:29 UTC 2013
>     root@ip-10-148-212-216:/usr/obj/usr/src/sys/XENHVM amd64
> FreeBSD clang version 3.3 (trunk 178860) 20130405
> WARNING: WITNESS option enabled, expect reduced performance.
> XEN: Hypervisor version 3.4 detected.
> XEN: Disabling emulated block and network devices
> Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff81912000.
> Hypervisor: Origin = "XenVMMXenVMM"
> Calibrating TSC clock ... TSC clock: 2593801200 Hz
> CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2593.80-MHz K8-class CPU)
>   Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d  Stepping = 7
>   Features=0x1781fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT>
>   Features2=0x9c982201<SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,XSAVE,OSXSAVE,AVX,HV>
>   AMD Features=0x20100800<SYSCALL,NX,LM>
>   AMD Features2=0x1<LAHF>
> real memory  = 65011712000 (62000 MB)
> Physical memory chunk(s):
> 0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages)
> 0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages)
> 0x0000000001972000 - 0x00000000bfffffff, 3194544128 bytes (779918 pages)
> 0x0000000100000000 - 0x0000000efeb95fff, 60108136448 bytes (14674838 pages)
> avail memory = 60563271680 (57757 MB)
> Event timer "LAPIC" quality 400
> ACPI APIC Table: <Xen HVM>
> INTR: Adding local APIC 1 as a target
> INTR: Adding local APIC 2 as a target
> INTR: Adding local APIC 3 as a target
> INTR: Adding local APIC 4 as a target
> INTR: Adding local APIC 5 as a target
> INTR: Adding local APIC 6 as a target
> INTR: Adding local APIC 7 as a target
> INTR: Adding local APIC 8 as a target
> INTR: Adding local APIC 9 as a target
> INTR: Adding local APIC 10 as a target
> INTR: Adding local APIC 11 as a target
> INTR: Adding local APIC 12 as a target
> INTR: Adding local APIC 13 as a target
> INTR: Adding local APIC 14 as a target
> INTR: Adding local APIC 15 as a target
> INTR: Adding local APIC 32 as a target
> INTR: Adding local APIC 33 as a target
> INTR: Adding local APIC 34 as a target
> INTR: Adding local APIC 35 as a target
> INTR: Adding local APIC 36 as a target
> INTR: Adding local APIC 37 as a target
> INTR: Adding local APIC 38 as a target
> INTR: Adding local APIC 39 as a target
> INTR: Adding local APIC 40 as a target
> INTR: Adding local APIC 41 as a target
> INTR: Adding local APIC 42 as a target
> INTR: Adding local APIC 43 as a target
> INTR: Adding local APIC 44 as a target
> INTR: Adding local APIC 45 as a target
> INTR: Adding local APIC 46 as a target
> INTR: Adding local APIC 47 as a target
> FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
> FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 SMT threads
>  cpu0 (BSP): APIC ID:  0
>  cpu1 (AP): APIC ID:  1
>  cpu2 (AP): APIC ID:  2
>  cpu3 (AP): APIC ID:  3
>  cpu4 (AP): APIC ID:  4
>  cpu5 (AP): APIC ID:  5
>  cpu6 (AP): APIC ID:  6
>  cpu7 (AP): APIC ID:  7
>  cpu8 (AP): APIC ID:  8
>  cpu9 (AP): APIC ID:  9
>  cpu10 (AP): APIC ID: 10
>  cpu11 (AP): APIC ID: 11
>  cpu12 (AP): APIC ID: 12
>  cpu13 (AP): APIC ID: 13
>  cpu14 (AP): APIC ID: 14
>  cpu15 (AP): APIC ID: 15
>  cpu16 (AP): APIC ID: 32
>  cpu17 (AP): APIC ID: 33
>  cpu18 (AP): APIC ID: 34
>  cpu19 (AP): APIC ID: 35
>  cpu20 (AP): APIC ID: 36
>  cpu21 (AP): APIC ID: 37
>  cpu22 (AP): APIC ID: 38
>  cpu23 (AP): APIC ID: 39
>  cpu24 (AP): APIC ID: 40
>  cpu25 (AP): APIC ID: 41
>  cpu26 (AP): APIC ID: 42
>  cpu27 (AP): APIC ID: 43
>  cpu28 (AP): APIC ID: 44
>  cpu29 (AP): APIC ID: 45
>  cpu30 (AP): APIC ID: 46
>  cpu31 (AP): APIC ID: 47
> x86bios:  IVT 0x000000-0x0004ff at 0xfffffe0000000000
> x86bios: SSEG 0x001000-0x001fff at 0xffffff80003cc000
> x86bios: EBDA 0x09f000-0x09ffff at 0xfffffe000009f000
> x86bios:  ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000
> APIC: CPU 0 has ACPI ID 0
> APIC: CPU 1 has ACPI ID 16
> APIC: CPU 2 has ACPI ID 1
> APIC: CPU 3 has ACPI ID 17
> APIC: CPU 4 has ACPI ID 2
> APIC: CPU 5 has ACPI ID 18
> APIC: CPU 6 has ACPI ID 3
> APIC: CPU 7 has ACPI ID 19
> APIC: CPU 8 has ACPI ID 4
> APIC: CPU 9 has ACPI ID 20
> APIC: CPU 10 has ACPI ID 5
> APIC: CPU 11 has ACPI ID 21
> APIC: CPU 12 has ACPI ID 6
> APIC: CPU 13 has ACPI ID 22
> APIC: CPU 14 has ACPI ID 7
> APIC: CPU 15 has ACPI ID 23
> APIC: CPU 16 has ACPI ID 8
> APIC: CPU 17 has ACPI ID 24
> APIC: CPU 18 has ACPI ID 9
> APIC: CPU 19 has ACPI ID 25
> APIC: CPU 20 has ACPI ID 10
> APIC: CPU 21 has ACPI ID 26
> APIC: CPU 22 has ACPI ID 11
> APIC: CPU 23 has ACPI ID 27
> APIC: CPU 24 has ACPI ID 12
> APIC: CPU 25 has ACPI ID 28
> APIC: CPU 26 has ACPI ID 13
> APIC: CPU 27 has ACPI ID 29
> APIC: CPU 28 has ACPI ID 14
> APIC: CPU 29 has ACPI ID 30
> APIC: CPU 30 has ACPI ID 15
> APIC: CPU 31 has ACPI ID 31
> random device not loaded; using insecure entropy
> ULE: setup cpu 0
> ULE: setup cpu 1
> ULE: setup cpu 2
> ULE: setup cpu 3
> ULE: setup cpu 4
> ULE: setup cpu 5
> ULE: setup cpu 6
> ULE: setup cpu 7
> ULE: setup cpu 8
> ULE: setup cpu 9
> ULE: setup cpu 10
> ULE: setup cpu 11
> ULE: setup cpu 12
> ULE: setup cpu 13
> ULE: setup cpu 14
> ULE: setup cpu 15
> ULE: setup cpu 16
> ULE: setup cpu 17
> ULE: setup cpu 18
> ULE: setup cpu 19
> ULE: setup cpu 20
> ULE: setup cpu 21
> ULE: setup cpu 22
> ULE: setup cpu 23
> ULE: setup cpu 24
> ULE: setup cpu 25
> ULE: setup cpu 26
> ULE: setup cpu 27
> ULE: setup cpu 28
> ULE: setup cpu 29
> ULE: setup cpu 30
> ULE: setup cpu 31
> ACPI: RSDP 0xea020 00024 (v02    Xen)
> ACPI: XSDT 0xfc006430 0004C (v01    Xen      HVM 00000000 HVML 00000000)
> ACPI: FACP 0xfc005ee0 000F4 (v04    Xen      HVM 00000000 HVML 00000000)
> ACPI: DSDT 0xfc002c40 0321F (v02    Xen      HVM 00000000 INTL 20090220)
> ACPI: FACS 0xfc002c00 00040
> ACPI: APIC 0xfc005fe0 00160 (v02    Xen      HVM 00000000 HVML 00000000)
> ACPI: SRAT 0xfc006140 00280 (v01    Xen      HVM 00000000 HVML 00000000)
> ACPI: SLIT 0xfc0063c0 00030 (v01    Xen      HVM 00000000 HVML 00000000)
> ACPI: HPET 0xfc0063f0 00038 (v01    Xen      HVM 00000000 HVML 00000000)
> MADT: Found IO APIC ID 1, Interrupt 0 at 0xfec00000
> ioapic0: Changing APIC ID to 1
> ioapic0: Routing external 8259A's -> intpin 0
> MADT: Interrupt override: source 0, irq 2
> ioapic0: Routing IRQ 0 -> intpin 2
> MADT: Interrupt override: source 5, irq 5
> ioapic0: intpin 5 trigger: level
> ioapic0: intpin 5 polarity: low
> MADT: Interrupt override: source 10, irq 10
> ioapic0: intpin 10 trigger: level
> ioapic0: intpin 10 polarity: low
> MADT: Interrupt override: source 11, irq 11
> ioapic0: intpin 11 trigger: level
> ioapic0: intpin 11 polarity: low
> MADT: Forcing active-low polarity and level trigger for SCI
> ioapic0: intpin 9 polarity: low
> ioapic0: intpin 9 trigger: level
> ioapic0 <Version 1.1> irqs 0-47 on motherboard
> cpu0 BSP:
>      ID: 0x00000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
>   lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
>   timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
> Event-channel device installed.
> snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024]
> feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25
> wlan: <802.11 Link Layer>
> null: <null device, zero device>
> nfslock: pseudo-device
> random: <entropy source, Software, Yarrow>
> VESA: INT 0x10 vector 0xc000:0x836e
> VESA: information block
> 0000   56 45 53 41 00 02 f5 82 00 c0 00 00 00 00 40 00
> 0010   00 02 40 00 00 01 f5 82 00 c0 f5 82 00 c0 0e 83
> 0020   00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0040   01 01 10 01 11 01 12 01 03 01 13 01 14 01 15 01
> 0050   05 01 16 01 17 01 18 01 07 01 19 01 1a 01 ff ff
> 0060   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0070   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0080   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0090   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0100   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0110   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0120   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0130   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0140   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0150   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0160   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0170   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 01a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 01b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 01c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 01d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 01e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 01f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> VESA: 15 mode(s) found
> VESA: v2.0, 4096k memory, flags:0x0, mode table:0xffffff80003ff040 (2000040)
> VESA: VGABIOS Cirrus extension
> VESA: VGABIOS Cirrus extension VGABIOS Cirrus extension 1.0
> io: <I/O>
> kbd: new array size 4
> kbd1 at kbdmux0
> mem: <memory>
> hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2
> hpt27xx: RocketRAID 27xx controller driver v1.0
> xen_et0: vector callbacks unavailable
> acpi0: <Xen> on motherboard
> ACPI: All ACPI Tables successfully acquired
> ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48
> acpi0: Power Button (fixed)
> acpi0: reservation of 0, a0000 (3) failed
> cpu0: Processor \_PR_.PR00 (ACPI ID 0) -> APIC ID 0
> cpu0: <ACPI CPU> on acpi0
> cpu0: switching to generic Cx mode
> cpu1: Processor \_PR_.PR01 (ACPI ID 1) -> APIC ID 2
> cpu1: <ACPI CPU> on acpi0
> cpu2: Processor \_PR_.PR02 (ACPI ID 2) -> APIC ID 4
> cpu2: <ACPI CPU> on acpi0
> cpu3: Processor \_PR_.PR03 (ACPI ID 3) -> APIC ID 6
> cpu3: <ACPI CPU> on acpi0
> cpu4: Processor \_PR_.PR04 (ACPI ID 4) -> APIC ID 8
> cpu4: <ACPI CPU> on acpi0
> cpu5: Processor \_PR_.PR05 (ACPI ID 5) -> APIC ID 10
> cpu5: <ACPI CPU> on acpi0
> cpu6: Processor \_PR_.PR06 (ACPI ID 6) -> APIC ID 12
> cpu6: <ACPI CPU> on acpi0
> cpu7: Processor \_PR_.PR07 (ACPI ID 7) -> APIC ID 14
> cpu7: <ACPI CPU> on acpi0
> cpu8: Processor \_PR_.PR08 (ACPI ID 8) -> APIC ID 16
> cpu8: <ACPI CPU> on acpi0
> cpu9: Processor \_PR_.PR09 (ACPI ID 9) -> APIC ID 18
> cpu9: <ACPI CPU> on acpi0
> cpu10: Processor \_PR_.PR0A (ACPI ID 10) -> APIC ID 20
> cpu10: <ACPI CPU> on acpi0
> cpu11: Processor \_PR_.PR0B (ACPI ID 11) -> APIC ID 22
> cpu11: <ACPI CPU> on acpi0
> cpu12: Processor \_PR_.PR0C (ACPI ID 12) -> APIC ID 24
> cpu12: <ACPI CPU> on acpi0
> cpu13: Processor \_PR_.PR0D (ACPI ID 13) -> APIC ID 26
> cpu13: <ACPI CPU> on acpi0
> cpu14: Processor \_PR_.PR0E (ACPI ID 14) -> APIC ID 28
> cpu14: <ACPI CPU> on acpi0
> hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
> hpet0: vendor 0x8086, rev 0x1, 62500000Hz 64bit, 3 timers, legacy route
> hpet0:  t0: irqs 0x00f00000 (0), 64bit, periodic
> hpet0:  t1: irqs 0x00f00000 (0), 64bit, periodic
> hpet0:  t2: irqs 0x00f00000 (0), 64bit, periodic
> Timecounter "HPET" frequency 62500000 Hz quality 950
> attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
> Timecounter "i8254" frequency 1193182 Hz quality 0
> ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 0 vector 49
> Event timer "i8254" frequency 1193182 Hz quality 100
> atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
> atrtc0: registered as a time-of-day clock (resolution 1000000us, adjustment 0.500000000s)
> ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 50
> Event timer "RTC" frequency 32768 Hz quality 0
> ACPI timer: 1/7 1/5 1/8 1/9 1/7 1/7 1/7 1/7 1/19 1/8 -> 10
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x1f48-0x1f4b on acpi0
> pci_link0:        Index  IRQ  Rtd  Ref  IRQs
>   Initial Probe       0    5   N     0  5 10 11
>   Validation          0    5   N     0  5 10 11
>   After Disable       0  255   N     0  5 10 11
> pci_link1:        Index  IRQ  Rtd  Ref  IRQs
>   Initial Probe       0   10   N     0  5 10 11
>   Validation          0   10   N     0  5 10 11
>   After Disable       0  255   N     0  5 10 11
> pci_link2:        Index  IRQ  Rtd  Ref  IRQs
>   Initial Probe       0   11   N     0  5 10 11
>   Validation          0   11   N     0  5 10 11
>   After Disable       0  255   N     0  5 10 11
> pci_link3:        Index  IRQ  Rtd  Ref  IRQs
>   Initial Probe       0    5   N     0  5 10 11
>   Validation          0    5   N     0  5 10 11
>   After Disable       0  255   N     0  5 10 11
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pcib0: decoding 4 range 0-0xcf7
> pcib0: decoding 4 range 0xd00-0xffff
> pcib0: decoding 3 range 0xa0000-0xbffff
> pcib0: decoding 3 range 0xc0000000-0xf4ffffff
> pci0: <ACPI PCI bus> on pcib0
> pci0: domain=0, physical bus=0
> found->	vendor=0x8086, dev=0x1237, revid=0x02
> 	domain=0, bus=0, slot=0, func=0
> 	class=06-00-00, hdrtype=0x00, mfdev=0
> 	cmdreg=0x0004, statreg=0x0000, cachelnsz=0 (dwords)
> 	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
> found->	vendor=0x8086, dev=0x7000, revid=0x00
> 	domain=0, bus=0, slot=1, func=0
> 	class=06-01-00, hdrtype=0x00, mfdev=1
> 	cmdreg=0x0007, statreg=0x0200, cachelnsz=0 (dwords)
> 	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
> found->	vendor=0x8086, dev=0x7010, revid=0x00
> 	domain=0, bus=0, slot=1, func=1
> 	class=01-01-80, hdrtype=0x00, mfdev=0
> 	cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords)
> 	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
> pcib0: allocated type 4 (0x1f0-0x1f7) for rid 10 of pci0:0:1:1
> pcib0: allocated type 4 (0x3f6-0x3f6) for rid 14 of pci0:0:1:1
> pcib0: allocated type 4 (0x170-0x177) for rid 18 of pci0:0:1:1
> pcib0: allocated type 4 (0x376-0x376) for rid 1c of pci0:0:1:1
> 	map[20]: type I/O Port, range 32, base 0xc100, size  4, enabled
> pcib0: allocated type 4 (0xc100-0xc10f) for rid 20 of pci0:0:1:1
> found->	vendor=0x8086, dev=0x7113, revid=0x01
> 	domain=0, bus=0, slot=1, func=3
> 	class=06-80-00, hdrtype=0x00, mfdev=0
> 	cmdreg=0x0004, statreg=0x0000, cachelnsz=0 (dwords)
> 	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
> 	intpin=a, irq=10
> pcib0: matched entry for 0.1.INTA
> pcib0: slot 1 INTA hardwired to IRQ 20
> found->	vendor=0x1013, dev=0x00b8, revid=0x00
> 	domain=0, bus=0, slot=2, func=0
> 	class=03-00-00, hdrtype=0x00, mfdev=0
> 	cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords)
> 	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
> 	map[10]: type Prefetchable Memory, range 32, base 0xc0000000, size 25, enabled
> pcib0: allocated type 3 (0xc0000000-0xc1ffffff) for rid 10 of pci0:0:2:0
> 	map[14]: type Memory, range 32, base 0xc3000000, size 12, enabled
> pcib0: allocated type 3 (0xc3000000-0xc3000fff) for rid 14 of pci0:0:2:0
> found->	vendor=0x5853, dev=0x0001, revid=0x01
> 	domain=0, bus=0, slot=3, func=0
> 	class=ff-80-00, hdrtype=0x00, mfdev=0
> 	cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords)
> 	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
> 	intpin=a, irq=5
> 	map[10]: type I/O Port, range 32, base 0xc000, size  8, enabled
> pcib0: allocated type 4 (0xc000-0xc0ff) for rid 10 of pci0:0:3:0
> 	map[14]: type Prefetchable Memory, range 32, base 0xc2000000, size 24, enabled
> pcib0: allocated type 3 (0xc2000000-0xc2ffffff) for rid 14 of pci0:0:3:0
> pcib0: matched entry for 0.3.INTA
> pcib0: slot 3 INTA hardwired to IRQ 28
> isab0: <PCI-ISA bridge> at device 1.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc100-0xc10f at device 1.1 on pci0
> ata0: <ATA channel> at channel 0 on atapci0
> ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 0 vector 51
> ata1: <ATA channel> at channel 1 on atapci0
> ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 0 vector 52
> pci0: <bridge> at device 1.3 (no driver attached)
> vgapci0: <VGA-compatible display> mem 0xc0000000-0xc1ffffff,0xc3000000-0xc3000fff at device 2.0 on pci0
> xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xc2000000-0xc2ffffff irq 28 at device 3.0 on pci0
> ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 0 vector 53
> xenstore0: <XenStore> on xenpci0
> Grant table initialized
> psmcpnp0: <PS/2 mouse port> irq 12 on acpi0
> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> atkbd: the current kbd controller command byte 0061
> atkbd: keyboard ID 0x41ab (2)
> kbdc: RESET_KBD return code:00fa
> kbdc: RESET_KBD status:00aa
> kbd0 at atkbd0
> kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x1d0000
> ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 54
> atkbd0: [GIANT-LOCKED]
> psm0: current command byte:0061
> kbdc: TEST_AUX_PORT status:0000
> kbdc: RESET_AUX return code:00fa
> kbdc: RESET_AUX status:00aa
> kbdc: RESET_AUX ID:0000
> kbdc: RESET_AUX return code:00fa
> kbdc: RESET_AUX status:00aa
> kbdc: RESET_AUX ID:0000
> psm: status 00 02 64
> psm: status 00 00 64
> psm: status 00 03 64
> psm: status 00 03 64
> psm: data 08 00 00
> psm: status 00 02 64
> psm0: <PS/2 Mouse> irq 12 on atkbdc0
> ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 55
> psm0: [GIANT-LOCKED]
> psm0: model IntelliMouse Explorer, device ID 4-00, 5 buttons
> psm0: config:00000000, flags:00000008, packet size:4
> psm0: syncmask:08, syncbits:00
> fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
> fdc0: does not respond
> device_attach: fdc0 attach returned 6
> uart0: <Non-standard ns8250 class UART with FIFOs> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
> ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 56
> uart0: fast interrupt
> uart0: console (9600,n,8,1)
> ppc0: using extended I/O port range
> ACPI: Enabled 1 GPEs in block 00 to 1F
> acpi0: wakeup code va 0xffffff9096fb5000 pa 0x4000
> ex_isa_identify()
> pcib0: allocated type 3 (0xa0000-0xa07ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa0800-0xa0fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa1000-0xa17ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa1800-0xa1fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa2000-0xa27ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa2800-0xa2fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa3000-0xa37ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa3800-0xa3fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa4000-0xa47ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa4800-0xa4fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa5000-0xa57ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa5800-0xa5fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa6000-0xa67ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa6800-0xa6fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa7000-0xa77ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa7800-0xa7fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa8000-0xa87ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa8800-0xa8fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa9000-0xa97ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xa9800-0xa9fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xaa000-0xaa7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xaa800-0xaafff) for rid 0 of orm0
> pcib0: allocated type 3 (0xab000-0xab7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xab800-0xabfff) for rid 0 of orm0
> pcib0: allocated type 3 (0xac000-0xac7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xac800-0xacfff) for rid 0 of orm0
> pcib0: allocated type 3 (0xad000-0xad7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xad800-0xadfff) for rid 0 of orm0
> pcib0: allocated type 3 (0xae000-0xae7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xae800-0xaefff) for rid 0 of orm0
> pcib0: allocated type 3 (0xaf000-0xaf7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xaf800-0xaffff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0
> pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0
> pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0
> ahc_isa_identify 0: ioport 0xc00 alloc failed
> ahc_isa_identify 1: ioport 0x1c00 alloc failed
> ahc_isa_identify 2: ioport 0x2c00 alloc failed
> ahc_isa_identify 3: ioport 0x3c00 alloc failed
> ahc_isa_identify 4: ioport 0x4c00 alloc failed
> ahc_isa_identify 5: ioport 0x5c00 alloc failed
> ahc_isa_identify 6: ioport 0x6c00 alloc failed
> ahc_isa_identify 7: ioport 0x7c00 alloc failed
> ahc_isa_identify 8: ioport 0x8c00 alloc failed
> ahc_isa_identify 9: ioport 0x9c00 alloc failed
> ahc_isa_identify 10: ioport 0xac00 alloc failed
> ahc_isa_identify 11: ioport 0xbc00 alloc failed
> ahc_isa_identify 12: ioport 0xcc00 alloc failed
> ahc_isa_identify 13: ioport 0xdc00 alloc failed
> ahc_isa_identify 14: ioport 0xec00 alloc failed
> isa_probe_children: disabling PnP devices
> atkbdc: atkbdc0 already exists; skipping it
> atrtc: atrtc0 already exists; skipping it
> attimer: attimer0 already exists; skipping it
> sc: sc0 already exists; skipping it
> uart: uart0 already exists; skipping it
> isa_probe_children: probing non-PnP devices
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x100>
> sc0: fb0, kbd1, terminal emulator: scteken (teken terminal)
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0
> pcib0: allocated type 3 (0xa0000-0xbffff) for rid 0 of vga0
> fdc0: No FDOUT register!
> fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0
> ppc0: cannot reserve I/O port range
> ppc0 failed to probe at irq 7 on isa0
> pcib0: allocated type 4 (0x2f8-0x2ff) for rid 0 of uart1
> uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0
> wbwd0 failed to probe on isa0
> isa_probe_children: probing PnP devices
> Device configuration finished.
> procfs registered
> lapic: Divisor 2, Frequency 50000453 Hz
> Timecounters tick every 10.000 msec
> vlan: initialized, using hash tables with chaining
> tcp_init: net.inet.tcp.tcbhashsize auto tuned to 524288
> lo0: bpf attached
> hptrr: no controller detected.
> hpt27xx: no controller detected.
> xenbusb_front0: <Xen Frontend Devices> on xenstore0
> ata0: reset tp1 mask=03 ostat0=00 ostat1=00
> ata0: stat0=0x00 err=0x00 lsb=0x00 msb=0x00
> ata0: stat1=0x00 err=0x00 lsb=0x00 msb=0x00
> ata0: reset tp2 stat0=00 stat1=00 devices=0x0
> ata1: reset tp1 mask=03 ostat0=00 ostat1=00
> ata1: stat0=0x00 err=0x00 lsb=0x00 msb=0x00
> ata1: stat1=0x00 err=0x00 lsb=0x00 msb=0x00
> ata1: reset tp2 stat0=00 stat1=00 devices=0x0

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 17/05/13 05:07, Colin Percival wrote:
> On 05/16/13 17:43, Roger Pau Monné wrote:
>> Thanks for testing this on EC2, could you post the full dmesg? So I can
>> see the hypervisor version and if the PV timer is loaded or not.
> 
> Here's what I get on a cc2.8xlarge with boot_verbose=YES:

I've pushed a new branch to my repository, pvhvm_v7 that should work,
there was a bug with PCI event channel interrupt set up. I've tested
with 3.4 and seems OK, but of course it doesn't support the vector
callback injection.

Regards, Roger.

Re: FreeBSD PVHVM call for testing

[Colin Percival]

[-- Attachment #1: Type: text/plain, Size: 965 bytes --]

On 05/18/13 02:50, Roger Pau Monné wrote:
> On 17/05/13 05:07, Colin Percival wrote:
>> On 05/16/13 17:43, Roger Pau Monné wrote:
>>> Thanks for testing this on EC2, could you post the full dmesg? So I can
>>> see the hypervisor version and if the PV timer is loaded or not.
>>
>> Here's what I get on a cc2.8xlarge with boot_verbose=YES:
> 
> I've pushed a new branch to my repository, pvhvm_v7 that should work,
> there was a bug with PCI event channel interrupt set up. I've tested
> with 3.4 and seems OK, but of course it doesn't support the vector
> callback injection.

That seems to work.  dmesg is attached.  Are there any particular tests
you'd like me to run?

If anyone else wants to play with this, you can launch ami-e75c358e in the
EC2 us-east-1 region.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

[-- Attachment #2: dmesg.log --]
[-- Type: text/plain, Size: 38082 bytes --]

Table 'FACP' at 0xfc005ee0
Table 'APIC' at 0xfc005fe0
APIC: Found table at 0xfc005fe0
APIC: Using the MADT enumerator.
MADT: Found CPU APIC ID 0 ACPI ID 0: enabled
SMP: Added CPU 0 (AP)
MADT: Found CPU APIC ID 2 ACPI ID 1: enabled
SMP: Added CPU 2 (AP)
MADT: Found CPU APIC ID 4 ACPI ID 2: enabled
SMP: Added CPU 4 (AP)
MADT: Found CPU APIC ID 6 ACPI ID 3: enabled
SMP: Added CPU 6 (AP)
MADT: Found CPU APIC ID 8 ACPI ID 4: enabled
SMP: Added CPU 8 (AP)
MADT: Found CPU APIC ID 10 ACPI ID 5: enabled
SMP: Added CPU 10 (AP)
MADT: Found CPU APIC ID 12 ACPI ID 6: enabled
SMP: Added CPU 12 (AP)
MADT: Found CPU APIC ID 14 ACPI ID 7: enabled
SMP: Added CPU 14 (AP)
MADT: Found CPU APIC ID 32 ACPI ID 8: enabled
SMP: Added CPU 32 (AP)
MADT: Found CPU APIC ID 34 ACPI ID 9: enabled
SMP: Added CPU 34 (AP)
MADT: Found CPU APIC ID 36 ACPI ID 10: enabled
SMP: Added CPU 36 (AP)
MADT: Found CPU APIC ID 38 ACPI ID 11: enabled
SMP: Added CPU 38 (AP)
MADT: Found CPU APIC ID 40 ACPI ID 12: enabled
SMP: Added CPU 40 (AP)
MADT: Found CPU APIC ID 42 ACPI ID 13: enabled
SMP: Added CPU 42 (AP)
MADT: Found CPU APIC ID 44 ACPI ID 14: enabled
SMP: Added CPU 44 (AP)
MADT: Found CPU APIC ID 46 ACPI ID 15: enabled
SMP: Added CPU 46 (AP)
MADT: Found CPU APIC ID 1 ACPI ID 16: enabled
SMP: Added CPU 1 (AP)
MADT: Found CPU APIC ID 3 ACPI ID 17: enabled
SMP: Added CPU 3 (AP)
MADT: Found CPU APIC ID 5 ACPI ID 18: enabled
SMP: Added CPU 5 (AP)
MADT: Found CPU APIC ID 7 ACPI ID 19: enabled
SMP: Added CPU 7 (AP)
MADT: Found CPU APIC ID 9 ACPI ID 20: enabled
SMP: Added CPU 9 (AP)
MADT: Found CPU APIC ID 11 ACPI ID 21: enabled
SMP: Added CPU 11 (AP)
MADT: Found CPU APIC ID 13 ACPI ID 22: enabled
SMP: Added CPU 13 (AP)
MADT: Found CPU APIC ID 15 ACPI ID 23: enabled
SMP: Added CPU 15 (AP)
MADT: Found CPU APIC ID 33 ACPI ID 24: enabled
SMP: Added CPU 33 (AP)
MADT: Found CPU APIC ID 35 ACPI ID 25: enabled
SMP: Added CPU 35 (AP)
MADT: Found CPU APIC ID 37 ACPI ID 26: enabled
SMP: Added CPU 37 (AP)
MADT: Found CPU APIC ID 39 ACPI ID 27: enabled
SMP: Added CPU 39 (AP)
MADT: Found CPU APIC ID 41 ACPI ID 28: enabled
SMP: Added CPU 41 (AP)
MADT: Found CPU APIC ID 43 ACPI ID 29: enabled
SMP: Added CPU 43 (AP)
MADT: Found CPU APIC ID 45 ACPI ID 30: enabled
SMP: Added CPU 45 (AP)
MADT: Found CPU APIC ID 47 ACPI ID 31: enabled
SMP: Added CPU 47 (AP)
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #0 r+9b25356: Sat May 18 14:46:16 UTC 2013
    root@ip-10-140-132-115:/usr/obj/usr/src/sys/XENHVM amd64
FreeBSD clang version 3.3 (trunk 178860) 20130405
WARNING: WITNESS option enabled, expect reduced performance.
XEN: Hypervisor version 3.4 detected.
XEN: Disabling emulated block and network devices
Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff81912000.
Hypervisor: Origin = "XenVMMXenVMM"
Calibrating TSC clock ... TSC clock: 2593802768 Hz
CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2593.80-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206d6  Family = 0x6  Model = 0x2d  Stepping = 6
  Features=0x1781fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x9c982201<SSE3,SSSE3,CX16,SSE4.1,SSE4.2,POPCNT,XSAVE,OSXSAVE,AVX,HV>
  AMD Features=0x20100800<SYSCALL,NX,LM>
  AMD Features2=0x1<LAHF>
real memory  = 65011712000 (62000 MB)
Physical memory chunk(s):
0x0000000000001000 - 0x000000000009bfff, 634880 bytes (155 pages)
0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages)
0x0000000001972000 - 0x00000000bfffffff, 3194544128 bytes (779918 pages)
0x0000000100000000 - 0x0000000efeb95fff, 60108136448 bytes (14674838 pages)
avail memory = 60563271680 (57757 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <Xen HVM>
INTR: Adding local APIC 1 as a target
INTR: Adding local APIC 2 as a target
INTR: Adding local APIC 3 as a target
INTR: Adding local APIC 4 as a target
INTR: Adding local APIC 5 as a target
INTR: Adding local APIC 6 as a target
INTR: Adding local APIC 7 as a target
INTR: Adding local APIC 8 as a target
INTR: Adding local APIC 9 as a target
INTR: Adding local APIC 10 as a target
INTR: Adding local APIC 11 as a target
INTR: Adding local APIC 12 as a target
INTR: Adding local APIC 13 as a target
INTR: Adding local APIC 14 as a target
INTR: Adding local APIC 15 as a target
INTR: Adding local APIC 32 as a target
INTR: Adding local APIC 33 as a target
INTR: Adding local APIC 34 as a target
INTR: Adding local APIC 35 as a target
INTR: Adding local APIC 36 as a target
INTR: Adding local APIC 37 as a target
INTR: Adding local APIC 38 as a target
INTR: Adding local APIC 39 as a target
INTR: Adding local APIC 40 as a target
INTR: Adding local APIC 41 as a target
INTR: Adding local APIC 42 as a target
INTR: Adding local APIC 43 as a target
INTR: Adding local APIC 44 as a target
INTR: Adding local APIC 45 as a target
INTR: Adding local APIC 46 as a target
INTR: Adding local APIC 47 as a target
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 SMT threads
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
 cpu8 (AP): APIC ID:  8
 cpu9 (AP): APIC ID:  9
 cpu10 (AP): APIC ID: 10
 cpu11 (AP): APIC ID: 11
 cpu12 (AP): APIC ID: 12
 cpu13 (AP): APIC ID: 13
 cpu14 (AP): APIC ID: 14
 cpu15 (AP): APIC ID: 15
 cpu16 (AP): APIC ID: 32
 cpu17 (AP): APIC ID: 33
 cpu18 (AP): APIC ID: 34
 cpu19 (AP): APIC ID: 35
 cpu20 (AP): APIC ID: 36
 cpu21 (AP): APIC ID: 37
 cpu22 (AP): APIC ID: 38
 cpu23 (AP): APIC ID: 39
 cpu24 (AP): APIC ID: 40
 cpu25 (AP): APIC ID: 41
 cpu26 (AP): APIC ID: 42
 cpu27 (AP): APIC ID: 43
 cpu28 (AP): APIC ID: 44
 cpu29 (AP): APIC ID: 45
 cpu30 (AP): APIC ID: 46
 cpu31 (AP): APIC ID: 47
x86bios:  IVT 0x000000-0x0004ff at 0xfffffe0000000000
x86bios: SSEG 0x001000-0x001fff at 0xffffff80003cc000
x86bios: EBDA 0x09f000-0x09ffff at 0xfffffe000009f000
x86bios:  ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000
APIC: CPU 0 has ACPI ID 0
APIC: CPU 1 has ACPI ID 16
APIC: CPU 2 has ACPI ID 1
APIC: CPU 3 has ACPI ID 17
APIC: CPU 4 has ACPI ID 2
APIC: CPU 5 has ACPI ID 18
APIC: CPU 6 has ACPI ID 3
APIC: CPU 7 has ACPI ID 19
APIC: CPU 8 has ACPI ID 4
APIC: CPU 9 has ACPI ID 20
APIC: CPU 10 has ACPI ID 5
APIC: CPU 11 has ACPI ID 21
APIC: CPU 12 has ACPI ID 6
APIC: CPU 13 has ACPI ID 22
APIC: CPU 14 has ACPI ID 7
APIC: CPU 15 has ACPI ID 23
APIC: CPU 16 has ACPI ID 8
APIC: CPU 17 has ACPI ID 24
APIC: CPU 18 has ACPI ID 9
APIC: CPU 19 has ACPI ID 25
APIC: CPU 20 has ACPI ID 10
APIC: CPU 21 has ACPI ID 26
APIC: CPU 22 has ACPI ID 11
APIC: CPU 23 has ACPI ID 27
APIC: CPU 24 has ACPI ID 12
APIC: CPU 25 has ACPI ID 28
APIC: CPU 26 has ACPI ID 13
APIC: CPU 27 has ACPI ID 29
APIC: CPU 28 has ACPI ID 14
APIC: CPU 29 has ACPI ID 30
APIC: CPU 30 has ACPI ID 15
APIC: CPU 31 has ACPI ID 31
ULE: setup cpu 0
ULE: setup cpu 1
ULE: setup cpu 2
ULE: setup cpu 3
ULE: setup cpu 4
ULE: setup cpu 5
ULE: setup cpu 6
ULE: setup cpu 7
ULE: setup cpu 8
ULE: setup cpu 9
ULE: setup cpu 10
ULE: setup cpu 11
ULE: setup cpu 12
ULE: setup cpu 13
ULE: setup cpu 14
ULE: setup cpu 15
ULE: setup cpu 16
ULE: setup cpu 17
ULE: setup cpu 18
ULE: setup cpu 19
ULE: setup cpu 20
ULE: setup cpu 21
ULE: setup cpu 22
ULE: setup cpu 23
ULE: setup cpu 24
ULE: setup cpu 25
ULE: setup cpu 26
ULE: setup cpu 27
ULE: setup cpu 28
ULE: setup cpu 29
ULE: setup cpu 30
ULE: setup cpu 31
ACPI: RSDP 0xea020 00024 (v02    Xen)
ACPI: XSDT 0xfc006430 0004C (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: FACP 0xfc005ee0 000F4 (v04    Xen      HVM 00000000 HVML 00000000)
ACPI: DSDT 0xfc002c40 0321F (v02    Xen      HVM 00000000 INTL 20090220)
ACPI: FACS 0xfc002c00 00040
ACPI: APIC 0xfc005fe0 00160 (v02    Xen      HVM 00000000 HVML 00000000)
ACPI: SRAT 0xfc006140 00280 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: SLIT 0xfc0063c0 00030 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: HPET 0xfc0063f0 00038 (v01    Xen      HVM 00000000 HVML 00000000)
MADT: Found IO APIC ID 1, Interrupt 0 at 0xfec00000
ioapic0: Changing APIC ID to 1
ioapic0: Routing external 8259A's -> intpin 0
MADT: Interrupt override: source 0, irq 2
ioapic0: Routing IRQ 0 -> intpin 2
MADT: Interrupt override: source 5, irq 5
ioapic0: intpin 5 trigger: level
ioapic0: intpin 5 polarity: low
MADT: Interrupt override: source 10, irq 10
ioapic0: intpin 10 trigger: level
ioapic0: intpin 10 polarity: low
MADT: Interrupt override: source 11, irq 11
ioapic0: intpin 11 trigger: level
ioapic0: intpin 11 polarity: low
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0: intpin 9 polarity: low
ioapic0: intpin 9 trigger: level
ioapic0 <Version 1.1> irqs 0-47 on motherboard
cpu0 BSP:
     ID: 0x00000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
Event-channel device installed.
snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024]
feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25
wlan: <802.11 Link Layer>
null: <null device, zero device>
nfslock: pseudo-device
random: <entropy source, Software, Yarrow>
VESA: INT 0x10 vector 0xc000:0x836e
VESA: information block
0000   56 45 53 41 00 02 f5 82 00 c0 00 00 00 00 40 00
0010   00 02 40 00 00 01 f5 82 00 c0 f5 82 00 c0 0e 83
0020   00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0040   01 01 10 01 11 01 12 01 03 01 13 01 14 01 15 01
0050   05 01 16 01 17 01 18 01 07 01 19 01 1a 01 ff ff
0060   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0070   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0080   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0090   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0100   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0110   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0120   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0130   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0140   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0150   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0160   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0170   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
VESA: 15 mode(s) found
VESA: v2.0, 4096k memory, flags:0x0, mode table:0xffffff80003ff040 (2000040)
VESA: VGABIOS Cirrus extension
VESA: VGABIOS Cirrus extension VGABIOS Cirrus extension 1.0
io: <I/O>
kbd: new array size 4
kbd1 at kbdmux0
mem: <memory>
hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2
hpt27xx: RocketRAID 27xx controller driver v1.0
xen_et0: vector callbacks unavailable
acpi0: <Xen> on motherboard
ACPI: All ACPI Tables successfully acquired
ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48
acpi0: Power Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
cpu0: Processor \\_PR_.PR00 (ACPI ID 0) -> APIC ID 0
cpu0: <ACPI CPU> on acpi0
cpu0: switching to generic Cx mode
cpu1: Processor \\_PR_.PR01 (ACPI ID 1) -> APIC ID 2
cpu1: <ACPI CPU> on acpi0
cpu2: Processor \\_PR_.PR02 (ACPI ID 2) -> APIC ID 4
cpu2: <ACPI CPU> on acpi0
cpu3: Processor \\_PR_.PR03 (ACPI ID 3) -> APIC ID 6
cpu3: <ACPI CPU> on acpi0
cpu4: Processor \\_PR_.PR04 (ACPI ID 4) -> APIC ID 8
cpu4: <ACPI CPU> on acpi0
cpu5: Processor \\_PR_.PR05 (ACPI ID 5) -> APIC ID 10
cpu5: <ACPI CPU> on acpi0
cpu6: Processor \\_PR_.PR06 (ACPI ID 6) -> APIC ID 12
cpu6: <ACPI CPU> on acpi0
cpu7: Processor \\_PR_.PR07 (ACPI ID 7) -> APIC ID 14
cpu7: <ACPI CPU> on acpi0
cpu8: Processor \\_PR_.PR08 (ACPI ID 8) -> APIC ID 16
cpu8: <ACPI CPU> on acpi0
cpu9: Processor \\_PR_.PR09 (ACPI ID 9) -> APIC ID 18
cpu9: <ACPI CPU> on acpi0
cpu10: Processor \\_PR_.PR0A (ACPI ID 10) -> APIC ID 20
cpu10: <ACPI CPU> on acpi0
cpu11: Processor \\_PR_.PR0B (ACPI ID 11) -> APIC ID 22
cpu11: <ACPI CPU> on acpi0
cpu12: Processor \\_PR_.PR0C (ACPI ID 12) -> APIC ID 24
cpu12: <ACPI CPU> on acpi0
cpu13: Processor \\_PR_.PR0D (ACPI ID 13) -> APIC ID 26
cpu13: <ACPI CPU> on acpi0
cpu14: Processor \\_PR_.PR0E (ACPI ID 14) -> APIC ID 28
cpu14: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
hpet0: vendor 0x8086, rev 0x1, 62500000Hz 64bit, 3 timers, legacy route
hpet0:  t0: irqs 0x00f00000 (0), 64bit, periodic
hpet0:  t1: irqs 0x00f00000 (0), 64bit, periodic
hpet0:  t2: irqs 0x00f00000 (0), 64bit, periodic
Timecounter "HPET" frequency 62500000 Hz quality 950
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 0 vector 49
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
atrtc0: registered as a time-of-day clock (resolution 1000000us, adjustment 0.500000000s)
ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 50
Event timer "RTC" frequency 32768 Hz quality 0
ACPI timer: 1/18 1/8 1/7 1/8 1/8 1/7 1/5 1/8 1/16 1/8 -> 10
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x1f48-0x1f4b on acpi0
pci_link0:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0    5   N     0  5 10 11
  Validation          0    5   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link1:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0   10   N     0  5 10 11
  Validation          0   10   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link2:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0   11   N     0  5 10 11
  Validation          0   11   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link3:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0    5   N     0  5 10 11
  Validation          0    5   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pcib0: decoding 4 range 0-0xcf7
pcib0: decoding 4 range 0xd00-0xffff
pcib0: decoding 3 range 0xa0000-0xbffff
pcib0: decoding 3 range 0xc0000000-0xf4ffffff
pci0: <ACPI PCI bus> on pcib0
pci0: domain=0, physical bus=0
found->	vendor=0x8086, dev=0x1237, revid=0x02
	domain=0, bus=0, slot=0, func=0
	class=06-00-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0004, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
found->	vendor=0x8086, dev=0x7000, revid=0x00
	domain=0, bus=0, slot=1, func=0
	class=06-01-00, hdrtype=0x00, mfdev=1
	cmdreg=0x0007, statreg=0x0200, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
found->	vendor=0x8086, dev=0x7010, revid=0x00
	domain=0, bus=0, slot=1, func=1
	class=01-01-80, hdrtype=0x00, mfdev=0
	cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
pcib0: allocated type 4 (0x1f0-0x1f7) for rid 10 of pci0:0:1:1
pcib0: allocated type 4 (0x3f6-0x3f6) for rid 14 of pci0:0:1:1
pcib0: allocated type 4 (0x170-0x177) for rid 18 of pci0:0:1:1
pcib0: allocated type 4 (0x376-0x376) for rid 1c of pci0:0:1:1
	map[20]: type I/O Port, range 32, base 0xc100, size  4, enabled
pcib0: allocated type 4 (0xc100-0xc10f) for rid 20 of pci0:0:1:1
found->	vendor=0x8086, dev=0x7113, revid=0x01
	domain=0, bus=0, slot=1, func=3
	class=06-80-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0004, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	intpin=a, irq=10
pcib0: matched entry for 0.1.INTA
pcib0: slot 1 INTA hardwired to IRQ 20
found->	vendor=0x1013, dev=0x00b8, revid=0x00
	domain=0, bus=0, slot=2, func=0
	class=03-00-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	map[10]: type Prefetchable Memory, range 32, base 0xc0000000, size 25, enabled
pcib0: allocated type 3 (0xc0000000-0xc1ffffff) for rid 10 of pci0:0:2:0
	map[14]: type Memory, range 32, base 0xc3000000, size 12, enabled
pcib0: allocated type 3 (0xc3000000-0xc3000fff) for rid 14 of pci0:0:2:0
found->	vendor=0x5853, dev=0x0001, revid=0x01
	domain=0, bus=0, slot=3, func=0
	class=ff-80-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	intpin=a, irq=5
	map[10]: type I/O Port, range 32, base 0xc000, size  8, enabled
pcib0: allocated type 4 (0xc000-0xc0ff) for rid 10 of pci0:0:3:0
	map[14]: type Prefetchable Memory, range 32, base 0xc2000000, size 24, enabled
pcib0: allocated type 3 (0xc2000000-0xc2ffffff) for rid 14 of pci0:0:3:0
pcib0: matched entry for 0.3.INTA
pcib0: slot 3 INTA hardwired to IRQ 28
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc100-0xc10f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 0 vector 51
ata1: <ATA channel> at channel 1 on atapci0
ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 0 vector 52
pci0: <bridge> at device 1.3 (no driver attached)
vgapci0: <VGA-compatible display> mem 0xc0000000-0xc1ffffff,0xc3000000-0xc3000fff at device 2.0 on pci0
xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xc2000000-0xc2ffffff irq 28 at device 3.0 on pci0
ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 0 vector 53
xenstore0: <XenStore> on xenpci0
Grant table initialized
psmcpnp0: <PS/2 mouse port> irq 12 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
atkbd: the current kbd controller command byte 0061
atkbd: keyboard ID 0x41ab (2)
kbd0 at atkbd0
kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x1d0000
ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 54
atkbd0: [GIANT-LOCKED]
psm0: current command byte:0061
psm0: <PS/2 Mouse> irq 12 on atkbdc0
ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 55
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4-00, 5 buttons
psm0: config:00000000, flags:00000008, packet size:4
psm0: syncmask:08, syncbits:00
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
uart0: <Non-standard ns8250 class UART with FIFOs> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 56
uart0: fast interrupt
uart0: console (9600,n,8,1)
ppc0: using extended I/O port range
ACPI: Enabled 1 GPEs in block 00 to 1F
acpi0: wakeup code va 0xffffff9096fb5000 pa 0x4000
ex_isa_identify()
pcib0: allocated type 3 (0xa0000-0xa07ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa0800-0xa0fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa1000-0xa17ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa1800-0xa1fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa2000-0xa27ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa2800-0xa2fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa3000-0xa37ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa3800-0xa3fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa4000-0xa47ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa4800-0xa4fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa5000-0xa57ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa5800-0xa5fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa6000-0xa67ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa6800-0xa6fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa7000-0xa77ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa7800-0xa7fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa8000-0xa87ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa8800-0xa8fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa9000-0xa97ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa9800-0xa9fff) for rid 0 of orm0
pcib0: allocated type 3 (0xaa000-0xaa7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xaa800-0xaafff) for rid 0 of orm0
pcib0: allocated type 3 (0xab000-0xab7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xab800-0xabfff) for rid 0 of orm0
pcib0: allocated type 3 (0xac000-0xac7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xac800-0xacfff) for rid 0 of orm0
pcib0: allocated type 3 (0xad000-0xad7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xad800-0xadfff) for rid 0 of orm0
pcib0: allocated type 3 (0xae000-0xae7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xae800-0xaefff) for rid 0 of orm0
pcib0: allocated type 3 (0xaf000-0xaf7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xaf800-0xaffff) for rid 0 of orm0
pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0
pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0
pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0
pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0
ahc_isa_identify 0: ioport 0xc00 alloc failed
ahc_isa_identify 1: ioport 0x1c00 alloc failed
ahc_isa_identify 2: ioport 0x2c00 alloc failed
ahc_isa_identify 3: ioport 0x3c00 alloc failed
ahc_isa_identify 4: ioport 0x4c00 alloc failed
ahc_isa_identify 5: ioport 0x5c00 alloc failed
ahc_isa_identify 6: ioport 0x6c00 alloc failed
ahc_isa_identify 7: ioport 0x7c00 alloc failed
ahc_isa_identify 8: ioport 0x8c00 alloc failed
ahc_isa_identify 9: ioport 0x9c00 alloc failed
ahc_isa_identify 10: ioport 0xac00 alloc failed
ahc_isa_identify 11: ioport 0xbc00 alloc failed
ahc_isa_identify 12: ioport 0xcc00 alloc failed
ahc_isa_identify 13: ioport 0xdc00 alloc failed
ahc_isa_identify 14: ioport 0xec00 alloc failed
isa_probe_children: disabling PnP devices
atkbdc: atkbdc0 already exists; skipping it
atrtc: atrtc0 already exists; skipping it
attimer: attimer0 already exists; skipping it
sc: sc0 already exists; skipping it
uart: uart0 already exists; skipping it
isa_probe_children: probing non-PnP devices
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
sc0: fb0, kbd1, terminal emulator: scteken (teken terminal)
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0
pcib0: allocated type 3 (0xa0000-0xbffff) for rid 0 of vga0
fdc0: No FDOUT register!
fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0
ppc0: cannot reserve I/O port range
ppc0 failed to probe at irq 7 on isa0
pcib0: allocated type 4 (0x2f8-0x2ff) for rid 0 of uart1
uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0
wbwd0 failed to probe on isa0
isa_probe_children: probing PnP devices
Device configuration finished.
procfs registered
lapic: Divisor 2, Frequency 50000487 Hz
Timecounters tick every 10.000 msec
vlan: initialized, using hash tables with chaining
tcp_init: net.inet.tcp.tcbhashsize auto tuned to 524288
lo0: bpf attached
hptrr: no controller detected.
hpt27xx: no controller detected.
xenbusb_front0: <Xen Frontend Devices> on xenstore0
ata0: reset tp1 mask=03 ostat0=00 ostat1=00
ata0: stat0=0x00 err=0x00 lsb=0x00 msb=0x00
ata0: stat1=0x00 err=0x00 lsb=0x00 msb=0x00
ata0: reset tp2 stat0=00 stat1=00 devices=0x0
ata1: reset tp1 mask=03 ostat0=00 ostat1=00
ata1: stat0=0x00 err=0x00 lsb=0x00 msb=0x00
ata1: stat1=0x00 err=0x00 lsb=0x00 msb=0x00
ata1: reset tp2 stat0=00 stat1=00 devices=0x0
xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
xn0: bpf attached
xn0: Ethernet address: 12:31:39:37:18:49
xenbusb_back0: <Xen Backend Devices> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
xn0: backend features: feature-sg feature-gso-tcp4
xbd0: 10240MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
xbd0: attaching as ad0
xbd0: disk supports cache flush using: barriers
GEOM: new disk ad0
xbd1: 860095MB <Virtual Block Device> at device/vbd/51728 on xenbusb_front0
xbd1: disk supports cache flush using: barriers
xbd2: 860095MB <Virtual Block Device> at device/vbd/51744 on xenbusb_front0
xbd2: disk supports cache flush using: barriers
xbd3: 860095MB <Virtual Block Device> at device/vbd/51760 on xenbusb_front0
xbd3: disk supports cache flush using: barriers
xbd4: 860095MB <Virtual Block Device> at device/vbd/51776 on xenbusb_front0
xbd4: disk supports cache flush using: barriers
SMP: AP CPU #1 Launched!
cpu1 AP:
     ID: 0x01000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #20 Launched!
cpu20 AP:
     ID: 0x24000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #21 Launched!
cpu21 AP:
     ID: 0x25000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #13 Launched!
cpu13 AP:
     ID: 0x0d000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #31 Launched!
cpu31 AP:
     ID: 0x2f000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #27 Launched!
cpu27 AP:
     ID: 0x2b000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #29 Launched!
cpu29 AP:
     ID: 0x2d000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #11 Launched!
cpu11 AP:
     ID: 0x0b000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #28 Launched!
cpu28 AP:
     ID: 0x2c000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #14 Launched!
cpu14 AP:
     ID: 0x0e000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #18 Launched!
cpu18 AP:
     ID: 0x22000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #12 Launched!
cpu12 AP:
     ID: 0x0c000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #8 Launched!
cpu8 AP:
     ID: 0x08000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #10 Launched!
cpu10 AP:
     ID: 0x0a000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #24 Launched!
cpu24 AP:
     ID: 0x28000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #15 Launched!
cpu15 AP:
     ID: 0x0f000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #3 Launched!
cpu3 AP:
     ID: 0x03000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #4 Launched!
cpu4 AP:
     ID: 0x04000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #25 Launched!
cpu25 AP:
     ID: 0x29000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #19 Launched!
cpu19 AP:
     ID: 0x23000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #16 Launched!
cpu16 AP:
     ID: 0x20000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #22 Launched!
cpu22 AP:
     ID: 0x26000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #7 Launched!
cpu7 AP:
     ID: 0x07000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #30 Launched!
cpu30 AP:
     ID: 0x2e000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #26 Launched!
cpu26 AP:
     ID: 0x2a000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #5 Launched!
cpu5 AP:
     ID: 0x05000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #2 Launched!
cpu2 AP:
     ID: 0x02000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #17 Launched!
cpu17 AP:
     ID: 0x21000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #23 Launched!
cpu23 AP:
     ID: 0x27000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #6 Launched!
cpu6 AP:
     ID: 0x06000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #9 Launched!
cpu9 AP:
     ID: 0x09000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 1 vector 48
ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 2 vector 48
ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 3 vector 48
ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 4 vector 48
ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 5 vector 48
ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 6 vector 48
ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 7 vector 48
TSC timecounter discards lower 1 bit(s)
Timecounter "TSC-low" frequency 1296901384 Hz quality -100
WARNING: WITNESS option enabled, expect reduced performance.
GEOM: new disk xbd1
GEOM: new disk xbd2
GEOM: new disk xbd3
GEOM: new disk xbd4
Trying to mount root from ufs:/dev/ad0a [rw]...
start_init: trying /sbin/init
Setting hostuuid: 00000000-0000-0000-0000-ec2195337621.
Setting hostid: 0xd9cb810e.
No suitable dump device was found.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
/dev/ad0a: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ad0a: clean, 1121193 free (3953 frags, 139655 blocks, 0.2% fragmentation)
Mounting local file systems:.
xn0: 2 link states coalesced
Starting Network: lo0 xn0.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128 
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 
	inet 127.0.0.1 netmask 0xff000000 
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
xn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=503<RXCSUM,TXCSUM,TSO4,LRO>
	ether 12:31:39:37:18:49
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet manual
	status: active
Starting devd.
Starting dhclient.
DHCPREQUEST on xn0 to 255.255.255.255 port 67
DHCPNAK from 169.254.1.0
DHCPDISCOVER on xn0 to 255.255.255.255 port 67 interval 6
DHCPOFFER from 169.254.1.0
DHCPREQUEST on xn0 to 255.255.255.255 port 67
DHCPACK from 169.254.1.0
bound to 10.58.166.183 -- renewal in 43200 seconds.
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
add net fe80::: gateway ::1
add net ff02::: gateway ::1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
32-bit compatibility ldconfig path: /usr/lib32
Creating and/or trimming log files.
Starting syslogd.
Attempting to create a 15494 MB swap area on /dev/xbd1... done.
Attempting to create a 15494 MB swap area on /dev/xbd2... done.
Attempting to create a 15494 MB swap area on /dev/xbd3... done.
Attempting to create a 15494 MB swap area on /dev/xbd4... done.
Enabling swapping to /dev/xbd1s1b
Enabling swapping to /dev/xbd2s1b
Enabling swapping to /dev/xbd3s1b
Enabling swapping to /dev/xbd4s1b
Enabling crash dumps to /dev/xbd1s1b
No core dumps found.
Clearing /tmp (X related).
Updating motd:.
Fetching SSH public key for ec2-user
Starting sshd.
ec2: #############################################################
ec2: -----BEGIN SSH HOST KEY FINGERPRINTS-----
ec2: 1024 65:01:f6:12:a2:34:4d:90:0f:a1:4f:b5:7a:e2:45:44 /etc/ssh/ssh_host_dsa_key.pub (DSA)
ec2: 256 cb:17:bf:04:f0:2a:9e:64:b0:d5:a9:a7:31:98:1b:84 /etc/ssh/ssh_host_ecdsa_key.pub (ECDSA)
ec2: 1024 13:18:35:e1:ea:37:58:c6:41:f5:97:97:69:f7:1b:7a /etc/ssh/ssh_host_key.pub (RSA1)
ec2: 2048 4b:06:3d:8a:6f:49:c6:81:f2:1d:bb:78:70:26:b6:70 /etc/ssh/ssh_host_rsa_key.pub (RSA)
ec2: -----END SSH HOST KEY FINGERPRINTS-----
ec2: #############################################################
Configuring syscons: blanktime.
Starting cron.
Starting background file system checks in 60 seconds.

Sat May 18 15:40:45 UTC 2013
May 18 15:40:53 ip-10-58-166-183 su: ec2-user to root on /dev/pts/0

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 18/05/13 17:44, Colin Percival wrote:
> On 05/18/13 02:50, Roger Pau Monné wrote:
>> On 17/05/13 05:07, Colin Percival wrote:
>>> On 05/16/13 17:43, Roger Pau Monné wrote:
>>>> Thanks for testing this on EC2, could you post the full dmesg? So I can
>>>> see the hypervisor version and if the PV timer is loaded or not.
>>>
>>> Here's what I get on a cc2.8xlarge with boot_verbose=YES:
>>
>> I've pushed a new branch to my repository, pvhvm_v7 that should work,
>> there was a bug with PCI event channel interrupt set up. I've tested
>> with 3.4 and seems OK, but of course it doesn't support the vector
>> callback injection.
> 
> That seems to work.  dmesg is attached.  Are there any particular tests
> you'd like me to run?

I have not tested ZFS, that might be a good one. If you are running this
on Xen 3.4 the behaviour should be the same as without this patches, so
there shouldn't be many differences.

If you could try that on Xen 4.0 at least (if I remember correctly
that's when the vector callback was introduced), you should see the PV
timer getting attached, and a performance increase.

> If anyone else wants to play with this, you can launch ami-e75c358e in the
> EC2 us-east-1 region.
> 

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/22/13 04:45, Roger Pau Monné wrote:
> On 18/05/13 17:44, Colin Percival wrote:
>> That seems to work.  dmesg is attached.  Are there any particular tests
>> you'd like me to run?
> 
> I have not tested ZFS, that might be a good one. If you are running this
> on Xen 3.4 the behaviour should be the same as without this patches, so
> there shouldn't be many differences.

I don't use ZFS personally, so I'm not sure exactly what tests to run on it;
hopefully someone else can take care of that.

> If you could try that on Xen 4.0 at least (if I remember correctly
> that's when the vector callback was introduced), you should see the PV
> timer getting attached, and a performance increase.

Testing on a cr1.8xlarge EC2 instance, I get Xen 4.2, but it ends up with
a panic -- console output below.  I can get a backtrace and possibly even
a dump if those would help.

Booting [/boot/kernel/kernel]...
-\|/-\|GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
SMAP type=01 base=0000000000000000 len=000000000009e000
SMAP type=02 base=000000000009e000 len=0000000000002000
SMAP type=02 base=00000000000e0000 len=0000000000020000
SMAP type=01 base=0000000000100000 len=00000000eff00000
SMAP type=02 base=00000000fc000000 len=0000000004000000
SMAP type=01 base=0000000100000000 len=0000003c19000000
Table 'FACP' at 0xfc014980
Table 'APIC' at 0xfc014a80
APIC: Found table at 0xfc014a80
APIC: Using the MADT enumerator.
MADT: Found CPU APIC ID 0 ACPI ID 0: enabled
SMP: Added CPU 0 (AP)
MADT: Found CPU APIC ID 2 ACPI ID 1: enabled
SMP: Added CPU 2 (AP)
MADT: Found CPU APIC ID 4 ACPI ID 2: enabled
SMP: Added CPU 4 (AP)
MADT: Found CPU APIC ID 6 ACPI ID 3: enabled
SMP: Added CPU 6 (AP)
MADT: Found CPU APIC ID 8 ACPI ID 4: enabled
SMP: Added CPU 8 (AP)
MADT: Found CPU APIC ID 10 ACPI ID 5: enabled
SMP: Added CPU 10 (AP)
MADT: Found CPU APIC ID 12 ACPI ID 6: enabled
SMP: Added CPU 12 (AP)
MADT: Found CPU APIC ID 14 ACPI ID 7: enabled
SMP: Added CPU 14 (AP)
MADT: Found CPU APIC ID 32 ACPI ID 8: enabled
SMP: Added CPU 32 (AP)
MADT: Found CPU APIC ID 34 ACPI ID 9: enabled
SMP: Added CPU 34 (AP)
MADT: Found CPU APIC ID 36 ACPI ID 10: enabled
SMP: Added CPU 36 (AP)
MADT: Found CPU APIC ID 38 ACPI ID 11: enabled
SMP: Added CPU 38 (AP)
MADT: Found CPU APIC ID 40 ACPI ID 12: enabled
SMP: Added CPU 40 (AP)
MADT: Found CPU APIC ID 42 ACPI ID 13: enabled
SMP: Added CPU 42 (AP)
MADT: Found CPU APIC ID 44 ACPI ID 14: enabled
SMP: Added CPU 44 (AP)
MADT: Found CPU APIC ID 46 ACPI ID 15: enabled
SMP: Added CPU 46 (AP)
MADT: Found CPU APIC ID 1 ACPI ID 16: enabled
SMP: Added CPU 1 (AP)
MADT: Found CPU APIC ID 3 ACPI ID 17: enabled
SMP: Added CPU 3 (AP)
MADT: Found CPU APIC ID 5 ACPI ID 18: enabled
SMP: Added CPU 5 (AP)
MADT: Found CPU APIC ID 7 ACPI ID 19: enabled
SMP: Added CPU 7 (AP)
MADT: Found CPU APIC ID 9 ACPI ID 20: enabled
SMP: Added CPU 9 (AP)
MADT: Found CPU APIC ID 11 ACPI ID 21: enabled
SMP: Added CPU 11 (AP)
MADT: Found CPU APIC ID 13 ACPI ID 22: enabled
SMP: Added CPU 13 (AP)
MADT: Found CPU APIC ID 15 ACPI ID 23: enabled
SMP: Added CPU 15 (AP)
MADT: Found CPU APIC ID 33 ACPI ID 24: enabled
SMP: Added CPU 33 (AP)
MADT: Found CPU APIC ID 35 ACPI ID 25: enabled
SMP: Added CPU 35 (AP)
MADT: Found CPU APIC ID 37 ACPI ID 26: enabled
SMP: Added CPU 37 (AP)
MADT: Found CPU APIC ID 39 ACPI ID 27: enabled
SMP: Added CPU 39 (AP)
MADT: Found CPU APIC ID 41 ACPI ID 28: enabled
SMP: Added CPU 41 (AP)
MADT: Found CPU APIC ID 43 ACPI ID 29: enabled
SMP: Added CPU 43 (AP)
MADT: Found CPU APIC ID 45 ACPI ID 30: enabled
SMP: Added CPU 45 (AP)
MADT: Found CPU APIC ID 47 ACPI ID 31: enabled
SMP: Added CPU 47 (AP)
MADT: Found CPU APIC ID 0 ACPI ID 32: disabled
MADT: Found CPU APIC ID 0 ACPI ID 33: disabled
MADT: Found CPU APIC ID 0 ACPI ID 34: disabled
MADT: Found CPU APIC ID 0 ACPI ID 35: disabled
MADT: Found CPU APIC ID 0 ACPI ID 36: disabled
MADT: Found CPU APIC ID 0 ACPI ID 37: disabled
MADT: Found CPU APIC ID 0 ACPI ID 38: disabled
MADT: Found CPU APIC ID 0 ACPI ID 39: disabled
MADT: Found CPU APIC ID 0 ACPI ID 40: disabled
MADT: Found CPU APIC ID 0 ACPI ID 41: disabled
MADT: Found CPU APIC ID 0 ACPI ID 42: disabled
MADT: Found CPU APIC ID 0 ACPI ID 43: disabled
MADT: Found CPU APIC ID 0 ACPI ID 44: disabled
MADT: Found CPU APIC ID 0 ACPI ID 45: disabled
MADT: Found CPU APIC ID 0 ACPI ID 46: disabled
MADT: Found CPU APIC ID 0 ACPI ID 47: disabled
MADT: Found CPU APIC ID 0 ACPI ID 48: disabled
MADT: Found CPU APIC ID 0 ACPI ID 49: disabled
MADT: Found CPU APIC ID 0 ACPI ID 50: disabled
MADT: Found CPU APIC ID 0 ACPI ID 51: disabled
MADT: Found CPU APIC ID 0 ACPI ID 52: disabled
MADT: Found CPU APIC ID 0 ACPI ID 53: disabled
MADT: Found CPU APIC ID 0 ACPI ID 54: disabled
MADT: Found CPU APIC ID 0 ACPI ID 55: disabled
MADT: Found CPU APIC ID 0 ACPI ID 56: disabled
MADT: Found CPU APIC ID 0 ACPI ID 57: disabled
MADT: Found CPU APIC ID 0 ACPI ID 58: disabled
MADT: Found CPU APIC ID 0 ACPI ID 59: disabled
MADT: Found CPU APIC ID 0 ACPI ID 60: disabled
MADT: Found CPU APIC ID 0 ACPI ID 61: disabled
MADT: Found CPU APIC ID 0 ACPI ID 62: disabled
MADT: Found CPU APIC ID 0 ACPI ID 63: disabled
MADT: Found CPU APIC ID 0 ACPI ID 64: disabled
MADT: Found CPU APIC ID 0 ACPI ID 65: disabled
MADT: Found CPU APIC ID 0 ACPI ID 66: disabled
MADT: Found CPU APIC ID 0 ACPI ID 67: disabled
MADT: Found CPU APIC ID 0 ACPI ID 68: disabled
MADT: Found CPU APIC ID 0 ACPI ID 69: disabled
MADT: Found CPU APIC ID 0 ACPI ID 70: disabled
MADT: Found CPU APIC ID 0 ACPI ID 71: disabled
MADT: Found CPU APIC ID 0 ACPI ID 72: disabled
MADT: Found CPU APIC ID 0 ACPI ID 73: disabled
MADT: Found CPU APIC ID 0 ACPI ID 74: disabled
MADT: Found CPU APIC ID 0 ACPI ID 75: disabled
MADT: Found CPU APIC ID 0 ACPI ID 76: disabled
MADT: Found CPU APIC ID 0 ACPI ID 77: disabled
MADT: Found CPU APIC ID 0 ACPI ID 78: disabled
MADT: Found CPU APIC ID 0 ACPI ID 79: disabled
MADT: Found CPU APIC ID 0 ACPI ID 80: disabled
MADT: Found CPU APIC ID 0 ACPI ID 81: disabled
MADT: Found CPU APIC ID 0 ACPI ID 82: disabled
MADT: Found CPU APIC ID 0 ACPI ID 83: disabled
MADT: Found CPU APIC ID 0 ACPI ID 84: disabled
MADT: Found CPU APIC ID 0 ACPI ID 85: disabled
MADT: Found CPU APIC ID 0 ACPI ID 86: disabled
MADT: Found CPU APIC ID 0 ACPI ID 87: disabled
MADT: Found CPU APIC ID 0 ACPI ID 88: disabled
MADT: Found CPU APIC ID 0 ACPI ID 89: disabled
MADT: Found CPU APIC ID 0 ACPI ID 90: disabled
MADT: Found CPU APIC ID 0 ACPI ID 91: disabled
MADT: Found CPU APIC ID 0 ACPI ID 92: disabled
MADT: Found CPU APIC ID 0 ACPI ID 93: disabled
MADT: Found CPU APIC ID 0 ACPI ID 94: disabled
MADT: Found CPU APIC ID 0 ACPI ID 95: disabled
MADT: Found CPU APIC ID 0 ACPI ID 96: disabled
MADT: Found CPU APIC ID 0 ACPI ID 97: disabled
MADT: Found CPU APIC ID 0 ACPI ID 98: disabled
MADT: Found CPU APIC ID 0 ACPI ID 99: disabled
MADT: Found CPU APIC ID 0 ACPI ID 100: disabled
MADT: Found CPU APIC ID 0 ACPI ID 101: disabled
MADT: Found CPU APIC ID 0 ACPI ID 102: disabled
MADT: Found CPU APIC ID 0 ACPI ID 103: disabled
MADT: Found CPU APIC ID 0 ACPI ID 104: disabled
MADT: Found CPU APIC ID 0 ACPI ID 105: disabled
MADT: Found CPU APIC ID 0 ACPI ID 106: disabled
MADT: Found CPU APIC ID 0 ACPI ID 107: disabled
MADT: Found CPU APIC ID 0 ACPI ID 108: disabled
MADT: Found CPU APIC ID 0 ACPI ID 109: disabled
MADT: Found CPU APIC ID 0 ACPI ID 110: disabled
MADT: Found CPU APIC ID 0 ACPI ID 111: disabled
MADT: Found CPU APIC ID 0 ACPI ID 112: disabled
MADT: Found CPU APIC ID 0 ACPI ID 113: disabled
MADT: Found CPU APIC ID 0 ACPI ID 114: disabled
MADT: Found CPU APIC ID 0 ACPI ID 115: disabled
MADT: Found CPU APIC ID 0 ACPI ID 116: disabled
MADT: Found CPU APIC ID 0 ACPI ID 117: disabled
MADT: Found CPU APIC ID 0 ACPI ID 118: disabled
MADT: Found CPU APIC ID 0 ACPI ID 119: disabled
MADT: Found CPU APIC ID 0 ACPI ID 120: disabled
MADT: Found CPU APIC ID 0 ACPI ID 121: disabled
MADT: Found CPU APIC ID 0 ACPI ID 122: disabled
MADT: Found CPU APIC ID 0 ACPI ID 123: disabled
MADT: Found CPU APIC ID 0 ACPI ID 124: disabled
MADT: Found CPU APIC ID 0 ACPI ID 125: disabled
MADT: Found CPU APIC ID 0 ACPI ID 126: disabled
MADT: Found CPU APIC ID 0 ACPI ID 127: disabled
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #0 r+9b25356: Sat May 18 14:46:16 UTC 2013
    root@ip-10-140-132-115:/usr/obj/usr/src/sys/XENHVM amd64
FreeBSD clang version 3.3 (trunk 178860) 20130405
WARNING: WITNESS option enabled, expect reduced performance.
XEN: Hypervisor version 4.2 detected.
XEN: Disabling emulated block and network devices
Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff81912000.
Hypervisor: Origin = "XenVMMXenVMM"
Calibrating TSC clock ... TSC clock: 2600048761 Hz
CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2600.05-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d  Stepping = 7

Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>

Features2=0x9fba2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,HV>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
real memory  = 262144000000 (250000 MB)
Physical memory chunk(s):
0x0000000000001000 - 0x0000000000099fff, 626688 bytes (153 pages)
0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages)
0x0000000001a2a000 - 0x00000000efffffff, 3999096832 bytes (976342 pages)
0x0000000100000000 - 0x0000003b8ae37fff, 251438268416 bytes (61386296 pages)
avail memory = 244393709568 (233072 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <Xen HVM>
INTR: Adding local APIC 1 as a target
INTR: Adding local APIC 2 as a target
INTR: Adding local APIC 3 as a target
INTR: Adding local APIC 4 as a target
INTR: Adding local APIC 5 as a target
INTR: Adding local APIC 6 as a target
INTR: Adding local APIC 7 as a target
INTR: Adding local APIC 8 as a target
INTR: Adding local APIC 9 as a target
INTR: Adding local APIC 10 as a target
INTR: Adding local APIC 11 as a target
INTR: Adding local APIC 12 as a target
INTR: Adding local APIC 13 as a target
INTR: Adding local APIC 14 as a target
INTR: Adding local APIC 15 as a target
INTR: Adding local APIC 32 as a target
INTR: Adding local APIC 33 as a target
INTR: Adding local APIC 34 as a target
INTR: Adding local APIC 35 as a target
INTR: Adding local APIC 36 as a target
INTR: Adding local APIC 37 as a target
INTR: Adding local APIC 38 as a target
INTR: Adding local APIC 39 as a target
INTR: Adding local APIC 40 as a target
INTR: Adding local APIC 41 as a target
INTR: Adding local APIC 42 as a target
INTR: Adding local APIC 43 as a target
INTR: Adding local APIC 44 as a target
INTR: Adding local APIC 45 as a target
INTR: Adding local APIC 46 as a target
INTR: Adding local APIC 47 as a target
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 SMT threads
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
 cpu8 (AP): APIC ID:  8
 cpu9 (AP): APIC ID:  9
 cpu10 (AP): APIC ID: 10
 cpu11 (AP): APIC ID: 11
 cpu12 (AP): APIC ID: 12
 cpu13 (AP): APIC ID: 13
 cpu14 (AP): APIC ID: 14
 cpu15 (AP): APIC ID: 15
 cpu16 (AP): APIC ID: 32
 cpu17 (AP): APIC ID: 33
 cpu18 (AP): APIC ID: 34
 cpu19 (AP): APIC ID: 35
 cpu20 (AP): APIC ID: 36
 cpu21 (AP): APIC ID: 37
 cpu22 (AP): APIC ID: 38
 cpu23 (AP): APIC ID: 39
 cpu24 (AP): APIC ID: 40
 cpu25 (AP): APIC ID: 41
 cpu26 (AP): APIC ID: 42
 cpu27 (AP): APIC ID: 43
 cpu28 (AP): APIC ID: 44
 cpu29 (AP): APIC ID: 45
 cpu30 (AP): APIC ID: 46
 cpu31 (AP): APIC ID: 47
x86bios:  IVT 0x000000-0x0004ff at 0xfffffe0000000000
x86bios: SSEG 0x001000-0x001fff at 0xffffff80005e3000
x86bios: EBDA 0x09f000-0x09ffff at 0xfffffe000009f000
x86bios:  ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000
APIC: CPU 0 has ACPI ID 0
APIC: CPU 1 has ACPI ID 16
APIC: CPU 2 has ACPI ID 1
APIC: CPU 3 has ACPI ID 17
APIC: CPU 4 has ACPI ID 2
APIC: CPU 5 has ACPI ID 18
APIC: CPU 6 has ACPI ID 3
APIC: CPU 7 has ACPI ID 19
APIC: CPU 8 has ACPI ID 4
APIC: CPU 9 has ACPI ID 20
APIC: CPU 10 has ACPI ID 5
APIC: CPU 11 has ACPI ID 21
APIC: CPU 12 has ACPI ID 6
APIC: CPU 13 has ACPI ID 22
APIC: CPU 14 has ACPI ID 7
APIC: CPU 15 has ACPI ID 23
APIC: CPU 16 has ACPI ID 8
APIC: CPU 17 has ACPI ID 24
APIC: CPU 18 has ACPI ID 9
APIC: CPU 19 has ACPI ID 25
APIC: CPU 20 has ACPI ID 10
APIC: CPU 21 has ACPI ID 26
APIC: CPU 22 has ACPI ID 11
APIC: CPU 23 has ACPI ID 27
APIC: CPU 24 has ACPI ID 12
APIC: CPU 25 has ACPI ID 28
APIC: CPU 26 has ACPI ID 13
APIC: CPU 27 has ACPI ID 29
APIC: CPU 28 has ACPI ID 14
APIC: CPU 29 has ACPI ID 30
APIC: CPU 30 has ACPI ID 15
APIC: CPU 31 has ACPI ID 31
random device not loaded; using insecure entropy
ULE: setup cpu 0
ULE: setup cpu 1
ULE: setup cpu 2
ULE: setup cpu 3
ULE: setup cpu 4
ULE: setup cpu 5
ULE: setup cpu 6
ULE: setup cpu 7
ULE: setup cpu 8
ULE: setup cpu 9
ULE: setup cpu 10
ULE: setup cpu 11
ULE: setup cpu 12
ULE: setup cpu 13
ULE: setup cpu 14
ULE: setup cpu 15
ULE: setup cpu 16
ULE: setup cpu 17
ULE: setup cpu 18
ULE: setup cpu 19
ULE: setup cpu 20
ULE: setup cpu 21
ULE: setup cpu 22
ULE: setup cpu 23
ULE: setup cpu 24
ULE: setup cpu 25
ULE: setup cpu 26
ULE: setup cpu 27
ULE: setup cpu 28
ULE: setup cpu 29
ULE: setup cpu 30
ULE: setup cpu 31
ACPI: RSDP 0xea020 00024 (v02    Xen)
ACPI: XSDT 0xfc015920 0005C (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: FACP 0xfc014980 000F4 (v04    Xen      HVM 00000000 HVML 00000000)
ACPI: DSDT 0xfc0035e0 11315 (v02    Xen      HVM 00000000 INTL 20090123)
ACPI: FACS 0xfc0035a0 00040
ACPI: APIC 0xfc014a80 00460 (v02    Xen      HVM 00000000 HVML 00000000)
ACPI: SRAT 0xfc014f60 008A8 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: HPET 0xfc015830 00038 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: WAET 0xfc015870 00028 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: SSDT 0xfc0158a0 00031 (v02    Xen      HVM 00000000 INTL 20090123)
ACPI: SSDT 0xfc0158e0 00031 (v02    Xen      HVM 00000000 INTL 20090123)
MADT: Found IO APIC ID 1, Interrupt 0 at 0xfec00000
ioapic0: Changing APIC ID to 1
ioapic0: Routing external 8259A's -> intpin 0
MADT: Interrupt override: source 0, irq 2
ioapic0: Routing IRQ 0 -> intpin 2
MADT: Interrupt override: source 5, irq 5
ioapic0: intpin 5 trigger: level
ioapic0: intpin 5 polarity: low
MADT: Interrupt override: source 10, irq 10
ioapic0: intpin 10 trigger: level
ioapic0: intpin 10 polarity: low
MADT: Interrupt override: source 11, irq 11
ioapic0: intpin 11 trigger: level
ioapic0: intpin 11 polarity: low
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0: intpin 9 polarity: low
ioapic0: intpin 9 trigger: level
ioapic0 <Version 1.1> irqs 0-47 on motherboard
cpu0 BSP:
     ID: 0x00000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
Event-channel device installed.
snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024]
feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1
feeder_rate_max=2016000 feeder_rate_round=25
wlan: <802.11 Link Layer>
null: <null device, zero device>
nfslock: pseudo-device
random: <entropy source, Software, Yarrow>
VESA: INT 0x10 vector 0xc000:0x836e
VESA: information block
0000   56 45 53 41 00 02 f5 82 00 c0 00 00 00 00 40 00
0010   00 02 40 00 00 01 f5 82 00 c0 f5 82 00 c0 0e 83
0020   00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0040   01 01 10 01 11 01 12 01 03 01 13 01 14 01 15 01
0050   05 01 16 01 17 01 18 01 07 01 19 01 1a 01 ff ff
0060   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0070   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0080   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0090   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0100   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0110   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0120   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0130   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0140   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0150   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0160   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0170   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
VESA: 15 mode(s) found
VESA: v2.0, 4096k memory, flags:0x0, mode table:0xffffff80005fd040 (2000040)
VESA: VGABIOS Cirrus extension
VESA: VGABIOS Cirrus extension VGABIOS Cirrus extension 1.0
io: <I/O>
kbd: new array size 4
kbd1 at kbdmux0
mem: <memory>
hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2
hpt27xx: RocketRAID 27xx controller driver v1.0
xen_et0: <Xen PV Clock> on motherboard
Event timer "XENTIMER" frequency 1000000000 Hz quality 950
Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
xen_et0: registered as a time-of-day clock (resolution 10000000us, adjustment
5.000000000s)
acpi0: <Xen> on motherboard
ACPI: All ACPI Tables successfully acquired
ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
cpu0: Processor \_SB_.PR00 (ACPI ID 0) -> APIC ID 0
cpu0: <ACPI CPU> on acpi0
cpu0: switching to generic Cx mode
cpu1: Processor \_SB_.PR01 (ACPI ID 1) -> APIC ID 2
cpu1: <ACPI CPU> on acpi0
cpu2: Processor \_SB_.PR02 (ACPI ID 2) -> APIC ID 4
cpu2: <ACPI CPU> on acpi0
cpu3: Processor \_SB_.PR03 (ACPI ID 3) -> APIC ID 6
cpu3: <ACPI CPU> on acpi0
cpu4: Processor \_SB_.PR04 (ACPI ID 4) -> APIC ID 8
cpu4: <ACPI CPU> on acpi0
cpu5: Processor \_SB_.PR05 (ACPI ID 5) -> APIC ID 10
cpu5: <ACPI CPU> on acpi0
cpu6: Processor \_SB_.PR06 (ACPI ID 6) -> APIC ID 12
cpu6: <ACPI CPU> on acpi0
cpu7: Processor \_SB_.PR07 (ACPI ID 7) -> APIC ID 14
cpu7: <ACPI CPU> on acpi0
cpu8: Processor \_SB_.PR08 (ACPI ID 8) -> APIC ID 16
cpu8: <ACPI CPU> on acpi0
cpu9: Processor \_SB_.PR09 (ACPI ID 9) -> APIC ID 18
cpu9: <ACPI CPU> on acpi0
cpu10: Processor \_SB_.PR0A (ACPI ID 10) -> APIC ID 20
cpu10: <ACPI CPU> on acpi0
cpu11: Processor \_SB_.PR0B (ACPI ID 11) -> APIC ID 22
cpu11: <ACPI CPU> on acpi0
cpu12: Processor \_SB_.PR0C (ACPI ID 12) -> APIC ID 24
cpu12: <ACPI CPU> on acpi0
cpu13: Processor \_SB_.PR0D (ACPI ID 13) -> APIC ID 26
cpu13: <ACPI CPU> on acpi0
cpu14: Processor \_SB_.PR0E (ACPI ID 14) -> APIC ID 28
cpu14: <ACPI CPU> on acpi0
cpu15: Processor \_SB_.PR0F (ACPI ID 15) -> APIC ID 30
cpu15: <ACPI CPU> on acpi0
cpu16: Processor \_SB_.PR10 (ACPI ID 16) -> APIC ID 1
cpu16: <ACPI CPU> on acpi0
cpu17: Processor \_SB_.PR11 (ACPI ID 17) -> APIC ID 3
cpu17: <ACPI CPU> on acpi0
cpu18: Processor \_SB_.PR12 (ACPI ID 18) -> APIC ID 5
cpu18: <ACPI CPU> on acpi0
cpu19: Processor \_SB_.PR13 (ACPI ID 19) -> APIC ID 7
cpu19: <ACPI CPU> on acpi0
cpu20: Processor \_SB_.PR14 (ACPI ID 20) -> APIC ID 9
cpu20: <ACPI CPU> on acpi0
cpu21: Processor \_SB_.PR15 (ACPI ID 21) -> APIC ID 11
cpu21: <ACPI CPU> on acpi0
cpu22: Processor \_SB_.PR16 (ACPI ID 22) -> APIC ID 13
cpu22: <ACPI CPU> on acpi0
cpu23: Processor \_SB_.PR17 (ACPI ID 23) -> APIC ID 15
cpu23: <ACPI CPU> on acpi0
cpu24: Processor \_SB_.PR18 (ACPI ID 24) -> APIC ID 17
cpu24: <ACPI CPU> on acpi0
cpu25: Processor \_SB_.PR19 (ACPI ID 25) -> APIC ID 19
cpu25: <ACPI CPU> on acpi0
cpu26: Processor \_SB_.PR1A (ACPI ID 26) -> APIC ID 21
cpu26: <ACPI CPU> on acpi0
cpu27: Processor \_SB_.PR1B (ACPI ID 27) -> APIC ID 23
cpu27: <ACPI CPU> on acpi0
cpu28: Processor \_SB_.PR1C (ACPI ID 28) -> APIC ID 25
cpu28: <ACPI CPU> on acpi0
cpu29: Processor \_SB_.PR1D (ACPI ID 29) -> APIC ID 27
cpu29: <ACPI CPU> on acpi0
cpu30: Processor \_SB_.PR1E (ACPI ID 30) -> APIC ID 29
cpu30: <ACPI CPU> on acpi0
cpu31: Processor \_SB_.PR1F (ACPI ID 31) -> APIC ID 31
cpu31: <ACPI CPU> on acpi0
ACPI: Processor \_SB_.PR20 (ACPI ID 32) ignored
ACPI: Processor \_SB_.PR21 (ACPI ID 33) ignored
ACPI: Processor \_SB_.PR22 (ACPI ID 34) ignored
ACPI: Processor \_SB_.PR23 (ACPI ID 35) ignored
ACPI: Processor \_SB_.PR24 (ACPI ID 36) ignored
ACPI: Processor \_SB_.PR25 (ACPI ID 37) ignored
ACPI: Processor \_SB_.PR26 (ACPI ID 38) ignored
ACPI: Processor \_SB_.PR27 (ACPI ID 39) ignored
ACPI: Processor \_SB_.PR28 (ACPI ID 40) ignored
ACPI: Processor \_SB_.PR29 (ACPI ID 41) ignored
ACPI: Processor \_SB_.PR2A (ACPI ID 42) ignored
ACPI: Processor \_SB_.PR2B (ACPI ID 43) ignored
ACPI: Processor \_SB_.PR2C (ACPI ID 44) ignored
ACPI: Processor \_SB_.PR2D (ACPI ID 45) ignored
ACPI: Processor \_SB_.PR2E (ACPI ID 46) ignored
ACPI: Processor \_SB_.PR2F (ACPI ID 47) ignored
ACPI: Processor \_SB_.PR30 (ACPI ID 48) ignored
ACPI: Processor \_SB_.PR31 (ACPI ID 49) ignored
ACPI: Processor \_SB_.PR32 (ACPI ID 50) ignored
ACPI: Processor \_SB_.PR33 (ACPI ID 51) ignored
ACPI: Processor \_SB_.PR34 (ACPI ID 52) ignored
ACPI: Processor \_SB_.PR35 (ACPI ID 53) ignored
ACPI: Processor \_SB_.PR36 (ACPI ID 54) ignored
ACPI: Processor \_SB_.PR37 (ACPI ID 55) ignored
ACPI: Processor \_SB_.PR38 (ACPI ID 56) ignored
ACPI: Processor \_SB_.PR39 (ACPI ID 57) ignored
ACPI: Processor \_SB_.PR3A (ACPI ID 58) ignored
ACPI: Processor \_SB_.PR3B (ACPI ID 59) ignored
ACPI: Processor \_SB_.PR3C (ACPI ID 60) ignored
ACPI: Processor \_SB_.PR3D (ACPI ID 61) ignored
ACPI: Processor \_SB_.PR3E (ACPI ID 62) ignored
ACPI: Processor \_SB_.PR3F (ACPI ID 63) ignored
ACPI: Processor \_SB_.PR40 (ACPI ID 64) ignored
ACPI: Processor \_SB_.PR41 (ACPI ID 65) ignored
ACPI: Processor \_SB_.PR42 (ACPI ID 66) ignored
ACPI: Processor \_SB_.PR43 (ACPI ID 67) ignored
ACPI: Processor \_SB_.PR44 (ACPI ID 68) ignored
ACPI: Processor \_SB_.PR45 (ACPI ID 69) ignored
ACPI: Processor \_SB_.PR46 (ACPI ID 70) ignored
ACPI: Processor \_SB_.PR47 (ACPI ID 71) ignored
ACPI: Processor \_SB_.PR48 (ACPI ID 72) ignored
ACPI: Processor \_SB_.PR49 (ACPI ID 73) ignored
ACPI: Processor \_SB_.PR4A (ACPI ID 74) ignored
ACPI: Processor \_SB_.PR4B (ACPI ID 75) ignored
ACPI: Processor \_SB_.PR4C (ACPI ID 76) ignored
ACPI: Processor \_SB_.PR4D (ACPI ID 77) ignored
ACPI: Processor \_SB_.PR4E (ACPI ID 78) ignored
ACPI: Processor \_SB_.PR4F (ACPI ID 79) ignored
ACPI: Processor \_SB_.PR50 (ACPI ID 80) ignored
ACPI: Processor \_SB_.PR51 (ACPI ID 81) ignored
ACPI: Processor \_SB_.PR52 (ACPI ID 82) ignored
ACPI: Processor \_SB_.PR53 (ACPI ID 83) ignored
ACPI: Processor \_SB_.PR54 (ACPI ID 84) ignored
ACPI: Processor \_SB_.PR55 (ACPI ID 85) ignored
ACPI: Processor \_SB_.PR56 (ACPI ID 86) ignored
ACPI: Processor \_SB_.PR57 (ACPI ID 87) ignored
ACPI: Processor \_SB_.PR58 (ACPI ID 88) ignored
ACPI: Processor \_SB_.PR59 (ACPI ID 89) ignored
ACPI: Processor \_SB_.PR5A (ACPI ID 90) ignored
ACPI: Processor \_SB_.PR5B (ACPI ID 91) ignored
ACPI: Processor \_SB_.PR5C (ACPI ID 92) ignored
ACPI: Processor \_SB_.PR5D (ACPI ID 93) ignored
ACPI: Processor \_SB_.PR5E (ACPI ID 94) ignored
ACPI: Processor \_SB_.PR5F (ACPI ID 95) ignored
ACPI: Processor \_SB_.PR60 (ACPI ID 96) ignored
ACPI: Processor \_SB_.PR61 (ACPI ID 97) ignored
ACPI: Processor \_SB_.PR62 (ACPI ID 98) ignored
ACPI: Processor \_SB_.PR63 (ACPI ID 99) ignored
ACPI: Processor \_SB_.PR64 (ACPI ID 100) ignored
ACPI: Processor \_SB_.PR65 (ACPI ID 101) ignored
ACPI: Processor \_SB_.PR66 (ACPI ID 102) ignored
ACPI: Processor \_SB_.PR67 (ACPI ID 103) ignored
ACPI: Processor \_SB_.PR68 (ACPI ID 104) ignored
ACPI: Processor \_SB_.PR69 (ACPI ID 105) ignored
ACPI: Processor \_SB_.PR6A (ACPI ID 106) ignored
ACPI: Processor \_SB_.PR6B (ACPI ID 107) ignored
ACPI: Processor \_SB_.PR6C (ACPI ID 108) ignored
ACPI: Processor \_SB_.PR6D (ACPI ID 109) ignored
ACPI: Processor \_SB_.PR6E (ACPI ID 110) ignored
ACPI: Processor \_SB_.PR6F (ACPI ID 111) ignored
ACPI: Processor \_SB_.PR70 (ACPI ID 112) ignored
ACPI: Processor \_SB_.PR71 (ACPI ID 113) ignored
ACPI: Processor \_SB_.PR72 (ACPI ID 114) ignored
ACPI: Processor \_SB_.PR73 (ACPI ID 115) ignored
ACPI: Processor \_SB_.PR74 (ACPI ID 116) ignored
ACPI: Processor \_SB_.PR75 (ACPI ID 117) ignored
ACPI: Processor \_SB_.PR76 (ACPI ID 118) ignored
ACPI: Processor \_SB_.PR77 (ACPI ID 119) ignored
ACPI: Processor \_SB_.PR78 (ACPI ID 120) ignored
ACPI: Processor \_SB_.PR79 (ACPI ID 121) ignored
ACPI: Processor \_SB_.PR7A (ACPI ID 122) ignored
ACPI: Processor \_SB_.PR7B (ACPI ID 123) ignored
ACPI: Processor \_SB_.PR7C (ACPI ID 124) ignored
ACPI: Processor \_SB_.PR7D (ACPI ID 125) ignored
ACPI: Processor \_SB_.PR7E (ACPI ID 126) ignored
ACPI: Processor \_SB_.PR7F (ACPI ID 127) ignored
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
hpet0: vendor 0x8086, rev 0x1, 62500000Hz 64bit, 3 timers, legacy route
hpet0:  t0: irqs 0x00f00000 (0), 64bit, periodic
hpet0:  t1: irqs 0x00f00000 (0), 64bit, periodic
hpet0:  t2: irqs 0x00f00000 (0), 64bit, periodic
Timecounter "HPET" frequency 62500000 Hz quality 950
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 0 vector 49
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
atrtc0: not installed as time-of-day clock: clock xen_et has higher resolution
ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 50
Event timer "RTC" frequency 32768 Hz quality 0
ACPI timer: 1/18 1/17 1/21 1/18 1/18 1/16 1/20 1/22 1/21 1/22 -> 10
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0xb008-0xb00b on acpi0
pci_link0:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0    5   N     0  5 10 11
  Validation          0    5   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link1:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0   10   N     0  5 10 11
  Validation          0   10   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link2:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0   11   N     0  5 10 11
  Validation          0   11   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link3:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0    5   N     0  5 10 11
  Validation          0    5   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pcib0: decoding 4 range 0-0xcf7
pcib0: decoding 4 range 0xd00-0xffff
pcib0: decoding 3 range 0xa0000-0xbffff
pcib0: decoding 3 range 0xf0000000-0xfbffffff
pci0: <ACPI PCI bus> on pcib0
pci0: domain=0, physical bus=0
found->	vendor=0x8086, dev=0x1237, revid=0x02
	domain=0, bus=0, slot=0, func=0
	class=06-00-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0004, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
found->	vendor=0x8086, dev=0x7000, revid=0x00
	domain=0, bus=0, slot=1, func=0
	class=06-01-00, hdrtype=0x00, mfdev=1
	cmdreg=0x0007, statreg=0x0200, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
found->	vendor=0x8086, dev=0x7010, revid=0x00
	domain=0, bus=0, slot=1, func=1
	class=01-01-80, hdrtype=0x00, mfdev=0
	cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
pcib0: allocated type 4 (0x1f0-0x1f7) for rid 10 of pci0:0:1:1
pcib0: allocated type 4 (0x3f6-0x3f6) for rid 14 of pci0:0:1:1
pcib0: allocated type 4 (0x170-0x177) for rid 18 of pci0:0:1:1
pcib0: allocated type 4 (0x376-0x376) for rid 1c of pci0:0:1:1
	map[20]: type I/O Port, range 32, base 0xc100, size  4, enabled
pcib0: allocated type 4 (0xc100-0xc10f) for rid 20 of pci0:0:1:1
found->	vendor=0x8086, dev=0x7113, revid=0x01
	domain=0, bus=0, slot=1, func=3
	class=06-80-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0004, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	intpin=a, irq=10
pcib0: matched entry for 0.1.INTA
pcib0: slot 1 INTA hardwired to IRQ 20
found->	vendor=0x1013, dev=0x00b8, revid=0x00
	domain=0, bus=0, slot=2, func=0
	class=03-00-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	map[10]: type Prefetchable Memory, range 32, base 0xf0000000, size 25, enabled
pcib0: allocated type 3 (0xf0000000-0xf1ffffff) for rid 10 of pci0:0:2:0
	map[14]: type Memory, range 32, base 0xf3000000, size 12, enabled
pcib0: allocated type 3 (0xf3000000-0xf3000fff) for rid 14 of pci0:0:2:0
found->	vendor=0x5853, dev=0x0001, revid=0x01
	domain=0, bus=0, slot=3, func=0
	class=ff-80-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	intpin=a, irq=5
	map[10]: type I/O Port, range 32, base 0xc000, size  8, enabled
pcib0: allocated type 4 (0xc000-0xc0ff) for rid 10 of pci0:0:3:0
	map[14]: type Prefetchable Memory, range 32, base 0xf2000000, size 24, enabled
pcib0: allocated type 3 (0xf2000000-0xf2ffffff) for rid 14 of pci0:0:3:0
pcib0: matched entry for 0.3.INTA
pcib0: slot 3 INTA hardwired to IRQ 28
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc100-0xc10f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 0 vector 51
ata1: <ATA channel> at channel 1 on atapci0
ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 0 vector 52
pci0: <bridge> at device 1.3 (no driver attached)
vgapci0: <VGA-compatible display> mem
0xf0000000-0xf1ffffff,0xf3000000-0xf3000fff at device 2.0 on pci0
xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xf2000000-0xf2ffffff irq
28 at device 3.0 on pci0
ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 0 vector 53
xenstore0: <XenStore> on xenpci0
Grant table initialized
psmcpnp0: <PS/2 mouse port> irq 12 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
atkbd: the current kbd controller command byte 0061
atkbd: keyboard ID 0x41ab (2)
kbdc: RESET_KBD return code:00fa
kbdc: RESET_KBD status:00aa
kbd0 at atkbd0
kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x1d0000
ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 54
atkbd0: [GIANT-LOCKED]
psm0: current command byte:0061
kbdc: TEST_AUX_PORT status:0000
kbdc: RESET_AUX return code:00fa
kbdc: RESET_AUX status:00aa
kbdc: RESET_AUX ID:0000
kbdc: RESET_AUX return code:00fa
kbdc: RESET_AUX status:00aa
kbdc: RESET_AUX ID:0000
psm: status 00 02 64
psm: status 00 00 64
psm: status 00 03 64
psm: status 00 03 64
psm: data 08 00 00
psm: status 00 02 64
psm0: <PS/2 Mouse> irq 12 on atkbdc0
ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 55
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4-00, 5 buttons
psm0: config:00000000, flags:00000008, packet size:4
psm0: syncmask:08, syncbits:00
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 56
uart0: fast interrupt
uart0: console (9600,n,8,1)
ACPI: Enabled 2 GPEs in block 00 to 0F
acpi0: wakeup code va 0xffffffc2cdff7000 pa 0x4000
ex_isa_identify()
pcib0: allocated type 3 (0xa0000-0xa07ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa0800-0xa0fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa1000-0xa17ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa1800-0xa1fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa2000-0xa27ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa2800-0xa2fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa3000-0xa37ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa3800-0xa3fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa4000-0xa47ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa4800-0xa4fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa5000-0xa57ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa5800-0xa5fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa6000-0xa67ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa6800-0xa6fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa7000-0xa77ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa7800-0xa7fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa8000-0xa87ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa8800-0xa8fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa9000-0xa97ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa9800-0xa9fff) for rid 0 of orm0
pcib0: allocated type 3 (0xaa000-0xaa7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xaa800-0xaafff) for rid 0 of orm0
pcib0: allocated type 3 (0xab000-0xab7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xab800-0xabfff) for rid 0 of orm0
pcib0: allocated type 3 (0xac000-0xac7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xac800-0xacfff) for rid 0 of orm0
pcib0: allocated type 3 (0xad000-0xad7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xad800-0xadfff) for rid 0 of orm0
pcib0: allocated type 3 (0xae000-0xae7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xae800-0xaefff) for rid 0 of orm0
pcib0: allocated type 3 (0xaf000-0xaf7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xaf800-0xaffff) for rid 0 of orm0
pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0
pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0
pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0
pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0
ahc_isa_identify 0: ioport 0xc00 alloc failed
ahc_isa_identify 1: ioport 0x1c00 alloc failed
ahc_isa_identify 2: ioport 0x2c00 alloc failed
ahc_isa_identify 3: ioport 0x3c00 alloc failed
ahc_isa_identify 4: ioport 0x4c00 alloc failed
ahc_isa_identify 5: ioport 0x5c00 alloc failed
ahc_isa_identify 6: ioport 0x6c00 alloc failed
ahc_isa_identify 7: ioport 0x7c00 alloc failed
ahc_isa_identify 8: ioport 0x8c00 alloc failed
ahc_isa_identify 9: ioport 0x9c00 alloc failed
ahc_isa_identify 10: ioport 0xac00 alloc failed
ahc_isa_identify 11: ioport 0xbc00 alloc failed
ahc_isa_identify 12: ioport 0xcc00 alloc failed
ahc_isa_identify 13: ioport 0xdc00 alloc failed
ahc_isa_identify 14: ioport 0xec00 alloc failed
isa_probe_children: disabling PnP devices
atkbdc: atkbdc0 already exists; skipping it
atrtc: atrtc0 already exists; skipping it
attimer: attimer0 already exists; skipping it
sc: sc0 already exists; skipping it
uart: uart0 already exists; skipping it
isa_probe_children: probing non-PnP devices
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
sc0: fb0, kbd1, terminal emulator: scteken (teken terminal)
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0
pcib0: allocated type 3 (0xa0000-0xbffff) for rid 0 of vga0
fdc0: No FDOUT register!
fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0
ppc0: cannot reserve I/O port range
ppc0 failed to probe at irq 7 on isa0
pcib0: allocated type 4 (0x2f8-0x2ff) for rid 0 of uart1
uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0
wbwd0 failed to probe on isa0
isa_probe_children: probing PnP devices
Device configuration finished.
procfs registered
Timecounters tick every 10.000 msec
vlan: initialized, using hash tables with chaining
tcp_init: net.inet.tcp.tcbhashsize auto tuned to 2097152
lo0: bpf attached
hptrr: no controller detected.
hpt27xx: no controller detected.
xenbusb_front0: <Xen Frontend Devices> on xenstore0
ata0: reset tp1 mask=03 ostat0=00 ostat1=00
ata0: stat0=0x00 err=0x00 lsb=0x00 msb=0x00
ata0: stat1=0x00 err=0x00 lsb=0x00 msb=0x00
ata0: reset tp2 stat0=00 stat1=00 devices=0x0
ata1: reset tp1 mask=03 ostat0=00 ostat1=00
ata1: stat0=0x00 err=0x00 lsb=0x00 msb=0x00
ata1: stat1=0x00 err=0x00 lsb=0x00 msb=0x00
ata1: reset tp2 stat0=00 stat1=00 devices=0x0
xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
xn0: bpf attached
xn0: Ethernet address: 22:00:0a:94:b4:ba
xenbusb_back0: <Xen Backend Devices> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
xn0: backend features: feature-sg feature-gso-tcp4
xbd0: 10240MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
xbd0: attaching as ad0
xbd0: disk supports cache flush using: barriers
GEOM: new disk ad0
xbd1: 122866MB <Virtual Block Device> at device/vbd/51728 on xenbusb_front0
xbd1: disk supports cache flush using: flush
xbd2: 122866MB <Virtual Block Device> at device/vbd/51744 on xenbusb_front0
xbd2: disk supports cache flush using: flush
SMP: AP CPU #29 Launched!
cpu29 AP:
     ID: 0x2d000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #24 Launched!
cpu24 AP:
     ID: 0x28000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #21 Launched!
cpu21 AP:
     ID: 0x25000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #3 Launched!
cpu3 AP:
     ID: 0x03000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #2 Launched!
cpu2 AP:
     ID: 0x02000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #6 Launched!
cpu6 AP:
     ID: 0x06000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #7 Launched!
cpu7 AP:
     ID: 0x07000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #23 Launched!
cpu23 AP:
     ID: 0x27000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #1 Launched!
cpu1 AP:
     ID: 0x01000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #18 Launched!
cpu18 AP:
     ID: 0x22000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #25 Launched!
cpu25 AP:
     ID: 0x29000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #16 Launched!
cpu16 AP:
     ID: 0x20000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #28 Launched!
cpu28 AP:
     ID: 0x2c000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #10 Launched!
cpu10 AP:
     ID: 0x0a000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #27 Launched!
cpu27 AP:
     ID: 0x2b000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #12 Launched!
cpu12 AP:
     ID: 0x0c000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #30 Launched!
cpu30 AP:
     ID: 0x2e000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #31 Launched!
cpu31 AP:
     ID: 0x2f000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #19 Launched!
cpu19 AP:
     ID: 0x23000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #15 Launched!
cpu15 AP:
     ID: 0x0f000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #17 Launched!
cpu17 AP:
     ID: 0x21000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #14 Launched!
cpu14 AP:
     ID: 0x0e000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #20 Launched!
cpu20 AP:
     ID: 0x24000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #22 Launched!
cpu22 AP:
     ID: 0x26000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #4 Launched!
cpu4 AP:
     ID: 0x04000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #8 Launched!
cpu8 AP:
     ID: 0x08000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #13 Launched!
cpu13 AP:
     ID: 0x0d000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #11 Launched!
cpu11 AP:
     ID: 0x0b000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #5 Launched!
cpu5 AP:
     ID: 0x05000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #26 Launched!
cpu26 AP:
     ID: 0x2a000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #9 Launched!
cpu9 AP:
     ID: 0x09000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
panic: xen_et0: Error -22 setting singleshot timer to 766317887512893

cpuid = 8
KDB: enter: panic
[ thread pid 11 tid 100011 ]
Stopped at      kdb_enter+0x3e: movq    $0,kdb_why
db>

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

[-- Attachment #1: Type: text/plain, Size: 12230 bytes --]

On 22/05/13 22:03, Colin Percival wrote:
> On 05/22/13 04:45, Roger Pau Monné wrote:
>> On 18/05/13 17:44, Colin Percival wrote:
>>> That seems to work.  dmesg is attached.  Are there any particular tests
>>> you'd like me to run?
>>
>> I have not tested ZFS, that might be a good one. If you are running this
>> on Xen 3.4 the behaviour should be the same as without this patches, so
>> there shouldn't be many differences.
> 
> I don't use ZFS personally, so I'm not sure exactly what tests to run on it;
> hopefully someone else can take care of that.
> 
>> If you could try that on Xen 4.0 at least (if I remember correctly
>> that's when the vector callback was introduced), you should see the PV
>> timer getting attached, and a performance increase.
> 
> Testing on a cr1.8xlarge EC2 instance, I get Xen 4.2, but it ends up with
> a panic -- console output below.  I can get a backtrace and possibly even
> a dump if those would help.

Hello Colin,

Thanks for the test, I've been using Xen 4.2 (and 4.3) without problems 
so far. By looking at the Xen code, the only reason the timer setup 
could return -22 (EINVAL), is that we try to set the timer for a 
different vCPU than the one we are running on.

I've been able to boot a 32 vCPU DomU on my 8way box using Xen 4.2.1 
(using both qemu-xen and qemu-xen-traditional device models), so I'm 
unsure if this could be due to some patch Amazon applies to Xen. Could 
you try the following patch and post the error message? I would like to 
see if the cpuid reported by kdb and the vCPU that we are trying to set 
the timer are the same.

Booting...
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #68: Wed May 22 19:00:14 CEST 2013
    root@:/usr/obj/usr/src/sys/XENHVM amd64
FreeBSD clang version 3.3 (trunk 178860) 20130405
WARNING: WITNESS option enabled, expect reduced performance.
XEN: Hypervisor version 4.2 detected.
CPU: Intel(R) Xeon(R) CPU           W3550  @ 3.07GHz (3066.83-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x106a5  Family = 0x6  Model = 0x1a  Stepping = 5
  Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x81b82201<SSE3,SSSE3,CX16,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,HV>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
real memory  = 4286578688 (4088 MB)
avail memory = 3961323520 (3777 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <Xen HVM>
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
FreeBSD/SMP: 2 package(s) x 16 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  2
 cpu2 (AP): APIC ID:  4
 cpu3 (AP): APIC ID:  6
 cpu4 (AP): APIC ID:  8
 cpu5 (AP): APIC ID: 10
 cpu6 (AP): APIC ID: 12
 cpu7 (AP): APIC ID: 14
 cpu8 (AP): APIC ID: 16
 cpu9 (AP): APIC ID: 18
 cpu10 (AP): APIC ID: 20
 cpu11 (AP): APIC ID: 22
 cpu12 (AP): APIC ID: 24
 cpu13 (AP): APIC ID: 26
 cpu14 (AP): APIC ID: 28
 cpu15 (AP): APIC ID: 30
 cpu16 (AP): APIC ID: 32
 cpu17 (AP): APIC ID: 34
 cpu18 (AP): APIC ID: 36
 cpu19 (AP): APIC ID: 38
 cpu20 (AP): APIC ID: 40
 cpu21 (AP): APIC ID: 42
 cpu22 (AP): APIC ID: 44
 cpu23 (AP): APIC ID: 46
 cpu24 (AP): APIC ID: 48
 cpu25 (AP): APIC ID: 50
 cpu26 (AP): APIC ID: 52
 cpu27 (AP): APIC ID: 54
 cpu28 (AP): APIC ID: 56
 cpu29 (AP): APIC ID: 58
 cpu30 (AP): APIC ID: 60
 cpu31 (AP): APIC ID: 62
random device not loaded; using insecure entropy
ioapic0: Changing APIC ID to 1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-47 on motherboard
kbd1 at kbdmux0
xen_et0: <Xen PV Clock> on motherboard
Event timer "XENTIMER" frequency 1000000000 Hz quality 950
Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
acpi0: <Xen> on motherboard
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
cpu4: <ACPI CPU> on acpi0
cpu5: <ACPI CPU> on acpi0
cpu6: <ACPI CPU> on acpi0
cpu7: <ACPI CPU> on acpi0
cpu8: <ACPI CPU> on acpi0
cpu9: <ACPI CPU> on acpi0
cpu10: <ACPI CPU> on acpi0
cpu11: <ACPI CPU> on acpi0
cpu12: <ACPI CPU> on acpi0
cpu13: <ACPI CPU> on acpi0
cpu14: <ACPI CPU> on acpi0
cpu15: <ACPI CPU> on acpi0
cpu16: <ACPI CPU> on acpi0
cpu17: <ACPI CPU> on acpi0
cpu18: <ACPI CPU> on acpi0
cpu19: <ACPI CPU> on acpi0
cpu20: <ACPI CPU> on acpi0
cpu21: <ACPI CPU> on acpi0
cpu22: <ACPI CPU> on acpi0
cpu23: <ACPI CPU> on acpi0
cpu24: <ACPI CPU> on acpi0
cpu25: <ACPI CPU> on acpi0
cpu26: <ACPI CPU> on acpi0
cpu27: <ACPI CPU> on acpi0
cpu28: <ACPI CPU> on acpi0
cpu29: <ACPI CPU> on acpi0
cpu30: <ACPI CPU> on acpi0
cpu31: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 62500000 Hz quality 950
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0xb008-0xb00b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc300-0xc30f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
pci0: <bridge> at device 1.3 (no driver attached)
vgapci0: <VGA-compatible display> mem 0xf0000000-0xf1ffffff,0xf3030000-0xf3030fff at device 2.0 on pci0
xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xf2000000-0xf2ffffff irq 28 at device 3.0 on pci0
xenstore0: <XenStore> on xenpci0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (9600,n,8,1)
ppc0: <Parallel port> port 0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
qpi0: <QPI system bus> on motherboard
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
fdc0: No FDOUT register!
Timecounters tick every 10.000 msec
xenbusb_front0: <Xen Frontend Devices> on xenstore0
xenbusb_add_device: Device device/suspend/event-channel ignored. State 6
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: <QEMU QEMU DVD-ROM 1.0.> Removable CD-ROM SCSI-0 device
cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
xn0: Ethernet address: 00:16:3e:47:d4:52
xn1: <Virtual Network Interface> at device/vif/1 on xenbusb_front0
xn1: Ethernet address: 00:16:3e:47:d4:53
xenbusb_back0: <Xen Backend Devices> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
xn0: backend features: feature-sg feature-gso-tcp4
xn1: backend features: feature-sg feature-gso-tcp4
xbd0: 20480MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
xbd0: attaching as ad0
xbd0: disk supports cache flush using: flush
SMP: AP CPU #25 Launched!
SMP: AP CPU #23 Launched!
SMP: AP CPU #27 Launched!
SMP: AP CPU #15 Launched!
SMP: AP CPU #9 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #30 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #22 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #29 Launched!
SMP: AP CPU #10 Launched!
SMP: AP CPU #19 Launched!
SMP: AP CPU #13 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #17 Launched!
SMP: AP CPU #18 Launched!
SMP: AP CPU #11 Launched!
SMP: AP CPU #28 Launched!
SMP: AP CPU #12 Launched!
SMP: AP CPU #31 Launched!
SMP: AP CPU #20 Launched!
SMP: AP CPU #24 Launched!
SMP: AP CPU #7 Launched!
SMP: AP CPU #16 Launched!
SMP: AP CPU #14 Launched!
SMP: AP CPU #6 Launched!
SMP: AP CPU #8 Launched!
SMP: AP CPU #26 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #21 Launched!
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/ad0p2 [rw]...
Setting hostuuid: c9230f36-1a54-489e-877c-1d15b8f463e9.
Setting hostid: 0xd52252c7.
Entropy harvesting: interrupts ethernet point_to_point kickstart.
Starting file system checks:
/dev/ad0p2: FILE SYSTEM CLEAN; SKIPPING CHECKS
/dev/ad0p2: clean, 1690868 free (11852 frags, 209877 blocks, 0.2% fragmentation)
Mounting local file systems:.
Writing entropy file:.
xn0: link state changed to DOWN
xn0: link state changed to UP
Starting Network: lo0 xn0 xn1.
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
	options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
	inet6 ::1 prefixlen 128
	inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
	inet 127.0.0.1 netmask 0xff000000
	nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
xn0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=503<RXCSUM,TXCSUM,TSO4,LRO>
	ether 00:16:3e:47:d4:52
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet manual
	status: active
xn1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=503<RXCSUM,TXCSUM,TSO4,LRO>
	ether 00:16:3e:47:d4:53
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet manual
	status: active
Starting devd.
Starting Network: xn1.
xn1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
	options=503<RXCSUM,TXCSUM,TSO4,LRO>
	ether 00:16:3e:47:d4:53
	nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
	media: Ethernet manual
	status: active
Starting dhclient.
DHCPREQUEST on xn0 to 255.255.255.255 port 67
DHCPACK from 192.168.1.1
bound to 192.168.1.107 -- renewal in 43200 seconds.
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
add net fe80::: gateway ::1
add net ff02::: gateway ::1
ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib
32-bit compatibility ldconfig path: /usr/lib32
Creating and/or trimming log files.
Starting syslogd.
Starting watchdogd.
watchdogd: patting the dog: Operation not supported
/etc/rc: WARNING: failed to start watchdogd
No core dumps found.
lock order reversal:
 1st 0xffffff80f7627d80 bufwait (bufwait) @ /usr/src/sys/kern/vfs_bio.c:3070
 2nd 0xfffffe0027c18000 dirhash (dirhash) @ /usr/src/sys/ufs/ufs/ufs_dirhash.c:284
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xffffff8121720450
kdb_backtrace() at kdb_backtrace+0x39/frame 0xffffff8121720500
witness_checkorder() at witness_checkorder+0xc3f/frame 0xffffff8121720580
_sx_xlock() at _sx_xlock+0x75/frame 0xffffff81217205c0
ufsdirhash_add() at ufsdirhash_add+0x3b/frame 0xffffff8121720600
ufs_direnter() at ufs_direnter+0x688/frame 0xffffff81217206c0
ufs_mkdir() at ufs_mkdir+0x863/frame 0xffffff81217208c0
VOP_MKDIR_APV() at VOP_MKDIR_APV+0xf0/frame 0xffffff81217208f0
kern_mkdirat() at kern_mkdirat+0x21a/frame 0xffffff8121720ae0
amd64_syscall() at amd64_syscall+0x265/frame 0xffffff8121720bf0
Xfast_syscall() at Xfast_syscall+0xfb/frame 0xffffff8121720bf0
--- syscall (136, FreeBSD ELF64, sys_mkdir), rip = 0x80092faaa, rsp = 0x7fffffffd788, rbp = 0x7fffffffdc70 ---
Clearing /tmp (X related).
Updating motd:.
Configuring syscons: keymap blanktime.
Performing sanity check on sshd configuration.
Starting sshd.
Starting cron.
Starting background file system checks in 60 seconds.

Thu May 23 10:08:04 CEST 2013

FreeBSD/amd64 (Amnesiac) (ttyu0)

[-- Attachment #2: timer_panic.patch --]
[-- Type: text/plain, Size: 572 bytes --]

diff --git a/sys/dev/xen/timer/timer.c b/sys/dev/xen/timer/timer.c
index 7083e46..4cb49b0 100644
--- a/sys/dev/xen/timer/timer.c
+++ b/sys/dev/xen/timer/timer.c
@@ -418,8 +418,8 @@ xentimer_et_start(struct eventtimer *et,
 	} while (error == -ETIME);
 
 	if (error)
-		panic("%s: Error %d setting singleshot timer to %"PRIu64"\n",
-		    device_get_nameunit(sc->dev), error, next_time);
+		panic("%s: Error %d setting singleshot timer to %"PRIu64" for vCPU#%d\n",
+		    device_get_nameunit(sc->dev), error, next_time, cpu);
 
 	pcpu->timer = next_time;
 	return (error);

[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/23/13 02:06, Roger Pau Monné wrote:
> On 22/05/13 22:03, Colin Percival wrote:
>> Testing on a cr1.8xlarge EC2 instance, I get Xen 4.2, but it ends up with
>> a panic -- console output below.  I can get a backtrace and possibly even
>> a dump if those would help.
> 
> Thanks for the test, I've been using Xen 4.2 (and 4.3) without problems 
> so far. By looking at the Xen code, the only reason the timer setup 
> could return -22 (EINVAL), is that we try to set the timer for a 
> different vCPU than the one we are running on.
> 
> I've been able to boot a 32 vCPU DomU on my 8way box using Xen 4.2.1 
> (using both qemu-xen and qemu-xen-traditional device models), so I'm 
> unsure if this could be due to some patch Amazon applies to Xen. Could 
> you try the following patch and post the error message? I would like to 
> see if the cpuid reported by kdb and the vCPU that we are trying to set 
> the timer are the same.

Looks like there's agreement about the cpuids here.  Anything else I should
try testing?

SMAP type=01 base=0000000000000000 len=000000000009e000
SMAP type=02 base=000000000009e000 len=0000000000002000
SMAP type=02 base=00000000000e0000 len=0000000000020000
SMAP type=01 base=0000000000100000 len=00000000eff00000
SMAP type=02 base=00000000fc000000 len=0000000004000000
SMAP type=01 base=0000000100000000 len=0000003c19000000
Table 'FACP' at 0xfc014980
Table 'APIC' at 0xfc014a80
APIC: Found table at 0xfc014a80
APIC: Using the MADT enumerator.
MADT: Found CPU APIC ID 0 ACPI ID 0: enabled
SMP: Added CPU 0 (AP)
MADT: Found CPU APIC ID 2 ACPI ID 1: enabled
SMP: Added CPU 2 (AP)
MADT: Found CPU APIC ID 4 ACPI ID 2: enabled
SMP: Added CPU 4 (AP)
MADT: Found CPU APIC ID 6 ACPI ID 3: enabled
SMP: Added CPU 6 (AP)
MADT: Found CPU APIC ID 8 ACPI ID 4: enabled
SMP: Added CPU 8 (AP)
MADT: Found CPU APIC ID 10 ACPI ID 5: enabled
SMP: Added CPU 10 (AP)
MADT: Found CPU APIC ID 12 ACPI ID 6: enabled
SMP: Added CPU 12 (AP)
MADT: Found CPU APIC ID 14 ACPI ID 7: enabled
SMP: Added CPU 14 (AP)
MADT: Found CPU APIC ID 32 ACPI ID 8: enabled
SMP: Added CPU 32 (AP)
MADT: Found CPU APIC ID 34 ACPI ID 9: enabled
SMP: Added CPU 34 (AP)
MADT: Found CPU APIC ID 36 ACPI ID 10: enabled
SMP: Added CPU 36 (AP)
MADT: Found CPU APIC ID 38 ACPI ID 11: enabled
SMP: Added CPU 38 (AP)
MADT: Found CPU APIC ID 40 ACPI ID 12: enabled
SMP: Added CPU 40 (AP)
MADT: Found CPU APIC ID 42 ACPI ID 13: enabled
SMP: Added CPU 42 (AP)
MADT: Found CPU APIC ID 44 ACPI ID 14: enabled
SMP: Added CPU 44 (AP)
MADT: Found CPU APIC ID 46 ACPI ID 15: enabled
SMP: Added CPU 46 (AP)
MADT: Found CPU APIC ID 1 ACPI ID 16: enabled
SMP: Added CPU 1 (AP)
MADT: Found CPU APIC ID 3 ACPI ID 17: enabled
SMP: Added CPU 3 (AP)
MADT: Found CPU APIC ID 5 ACPI ID 18: enabled
SMP: Added CPU 5 (AP)
MADT: Found CPU APIC ID 7 ACPI ID 19: enabled
SMP: Added CPU 7 (AP)
MADT: Found CPU APIC ID 9 ACPI ID 20: enabled
SMP: Added CPU 9 (AP)
MADT: Found CPU APIC ID 11 ACPI ID 21: enabled
SMP: Added CPU 11 (AP)
MADT: Found CPU APIC ID 13 ACPI ID 22: enabled
SMP: Added CPU 13 (AP)
MADT: Found CPU APIC ID 15 ACPI ID 23: enabled
SMP: Added CPU 15 (AP)
MADT: Found CPU APIC ID 33 ACPI ID 24: enabled
SMP: Added CPU 33 (AP)
MADT: Found CPU APIC ID 35 ACPI ID 25: enabled
SMP: Added CPU 35 (AP)
MADT: Found CPU APIC ID 37 ACPI ID 26: enabled
SMP: Added CPU 37 (AP)
MADT: Found CPU APIC ID 39 ACPI ID 27: enabled
SMP: Added CPU 39 (AP)
MADT: Found CPU APIC ID 41 ACPI ID 28: enabled
SMP: Added CPU 41 (AP)
MADT: Found CPU APIC ID 43 ACPI ID 29: enabled
SMP: Added CPU 43 (AP)
MADT: Found CPU APIC ID 45 ACPI ID 30: enabled
SMP: Added CPU 45 (AP)
MADT: Found CPU APIC ID 47 ACPI ID 31: enabled
SMP: Added CPU 47 (AP)
MADT: Found CPU APIC ID 0 ACPI ID 32: disabled
MADT: Found CPU APIC ID 0 ACPI ID 33: disabled
MADT: Found CPU APIC ID 0 ACPI ID 34: disabled
MADT: Found CPU APIC ID 0 ACPI ID 35: disabled
MADT: Found CPU APIC ID 0 ACPI ID 36: disabled
MADT: Found CPU APIC ID 0 ACPI ID 37: disabled
MADT: Found CPU APIC ID 0 ACPI ID 38: disabled
MADT: Found CPU APIC ID 0 ACPI ID 39: disabled
MADT: Found CPU APIC ID 0 ACPI ID 40: disabled
MADT: Found CPU APIC ID 0 ACPI ID 41: disabled
MADT: Found CPU APIC ID 0 ACPI ID 42: disabled
MADT: Found CPU APIC ID 0 ACPI ID 43: disabled
MADT: Found CPU APIC ID 0 ACPI ID 44: disabled
MADT: Found CPU APIC ID 0 ACPI ID 45: disabled
MADT: Found CPU APIC ID 0 ACPI ID 46: disabled
MADT: Found CPU APIC ID 0 ACPI ID 47: disabled
MADT: Found CPU APIC ID 0 ACPI ID 48: disabled
MADT: Found CPU APIC ID 0 ACPI ID 49: disabled
MADT: Found CPU APIC ID 0 ACPI ID 50: disabled
MADT: Found CPU APIC ID 0 ACPI ID 51: disabled
MADT: Found CPU APIC ID 0 ACPI ID 52: disabled
MADT: Found CPU APIC ID 0 ACPI ID 53: disabled
MADT: Found CPU APIC ID 0 ACPI ID 54: disabled
MADT: Found CPU APIC ID 0 ACPI ID 55: disabled
MADT: Found CPU APIC ID 0 ACPI ID 56: disabled
MADT: Found CPU APIC ID 0 ACPI ID 57: disabled
MADT: Found CPU APIC ID 0 ACPI ID 58: disabled
MADT: Found CPU APIC ID 0 ACPI ID 59: disabled
MADT: Found CPU APIC ID 0 ACPI ID 60: disabled
MADT: Found CPU APIC ID 0 ACPI ID 61: disabled
MADT: Found CPU APIC ID 0 ACPI ID 62: disabled
MADT: Found CPU APIC ID 0 ACPI ID 63: disabled
MADT: Found CPU APIC ID 0 ACPI ID 64: disabled
MADT: Found CPU APIC ID 0 ACPI ID 65: disabled
MADT: Found CPU APIC ID 0 ACPI ID 66: disabled
MADT: Found CPU APIC ID 0 ACPI ID 67: disabled
MADT: Found CPU APIC ID 0 ACPI ID 68: disabled
MADT: Found CPU APIC ID 0 ACPI ID 69: disabled
MADT: Found CPU APIC ID 0 ACPI ID 70: disabled
MADT: Found CPU APIC ID 0 ACPI ID 71: disabled
MADT: Found CPU APIC ID 0 ACPI ID 72: disabled
MADT: Found CPU APIC ID 0 ACPI ID 73: disabled
MADT: Found CPU APIC ID 0 ACPI ID 74: disabled
MADT: Found CPU APIC ID 0 ACPI ID 75: disabled
MADT: Found CPU APIC ID 0 ACPI ID 76: disabled
MADT: Found CPU APIC ID 0 ACPI ID 77: disabled
MADT: Found CPU APIC ID 0 ACPI ID 78: disabled
MADT: Found CPU APIC ID 0 ACPI ID 79: disabled
MADT: Found CPU APIC ID 0 ACPI ID 80: disabled
MADT: Found CPU APIC ID 0 ACPI ID 81: disabled
MADT: Found CPU APIC ID 0 ACPI ID 82: disabled
MADT: Found CPU APIC ID 0 ACPI ID 83: disabled
MADT: Found CPU APIC ID 0 ACPI ID 84: disabled
MADT: Found CPU APIC ID 0 ACPI ID 85: disabled
MADT: Found CPU APIC ID 0 ACPI ID 86: disabled
MADT: Found CPU APIC ID 0 ACPI ID 87: disabled
MADT: Found CPU APIC ID 0 ACPI ID 88: disabled
MADT: Found CPU APIC ID 0 ACPI ID 89: disabled
MADT: Found CPU APIC ID 0 ACPI ID 90: disabled
MADT: Found CPU APIC ID 0 ACPI ID 91: disabled
MADT: Found CPU APIC ID 0 ACPI ID 92: disabled
MADT: Found CPU APIC ID 0 ACPI ID 93: disabled
MADT: Found CPU APIC ID 0 ACPI ID 94: disabled
MADT: Found CPU APIC ID 0 ACPI ID 95: disabled
MADT: Found CPU APIC ID 0 ACPI ID 96: disabled
MADT: Found CPU APIC ID 0 ACPI ID 97: disabled
MADT: Found CPU APIC ID 0 ACPI ID 98: disabled
MADT: Found CPU APIC ID 0 ACPI ID 99: disabled
MADT: Found CPU APIC ID 0 ACPI ID 100: disabled
MADT: Found CPU APIC ID 0 ACPI ID 101: disabled
MADT: Found CPU APIC ID 0 ACPI ID 102: disabled
MADT: Found CPU APIC ID 0 ACPI ID 103: disabled
MADT: Found CPU APIC ID 0 ACPI ID 104: disabled
MADT: Found CPU APIC ID 0 ACPI ID 105: disabled
MADT: Found CPU APIC ID 0 ACPI ID 106: disabled
MADT: Found CPU APIC ID 0 ACPI ID 107: disabled
MADT: Found CPU APIC ID 0 ACPI ID 108: disabled
MADT: Found CPU APIC ID 0 ACPI ID 109: disabled
MADT: Found CPU APIC ID 0 ACPI ID 110: disabled
MADT: Found CPU APIC ID 0 ACPI ID 111: disabled
MADT: Found CPU APIC ID 0 ACPI ID 112: disabled
MADT: Found CPU APIC ID 0 ACPI ID 113: disabled
MADT: Found CPU APIC ID 0 ACPI ID 114: disabled
MADT: Found CPU APIC ID 0 ACPI ID 115: disabled
MADT: Found CPU APIC ID 0 ACPI ID 116: disabled
MADT: Found CPU APIC ID 0 ACPI ID 117: disabled
MADT: Found CPU APIC ID 0 ACPI ID 118: disabled
MADT: Found CPU APIC ID 0 ACPI ID 119: disabled
MADT: Found CPU APIC ID 0 ACPI ID 120: disabled
MADT: Found CPU APIC ID 0 ACPI ID 121: disabled
MADT: Found CPU APIC ID 0 ACPI ID 122: disabled
MADT: Found CPU APIC ID 0 ACPI ID 123: disabled
MADT: Found CPU APIC ID 0 ACPI ID 124: disabled
MADT: Found CPU APIC ID 0 ACPI ID 125: disabled
MADT: Found CPU APIC ID 0 ACPI ID 126: disabled
MADT: Found CPU APIC ID 0 ACPI ID 127: disabled
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #2 r+9b25356-dirty: Thu May 23 18:49:22 UTC 2013
    root@ip-10-148-177-216:/usr/obj/usr/src/sys/XENHVM amd64
FreeBSD clang version 3.3 (trunk 178860) 20130405
WARNING: WITNESS option enabled, expect reduced performance.
XEN: Hypervisor version 4.2 detected.
XEN: Disabling emulated block and network devices
Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff818ca000.
Hypervisor: Origin = "XenVMMXenVMM"
Calibrating TSC clock ... TSC clock: 2600046368 Hz
CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz (2600.05-MHz K8-class CPU)
  Origin = "GenuineIntel"  Id = 0x206d7  Family = 0x6  Model = 0x2d  Stepping = 7

Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>

Features2=0x9fba2203<SSE3,PCLMULQDQ,SSSE3,CX16,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,HV>
  AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
  AMD Features2=0x1<LAHF>
real memory  = 262144000000 (250000 MB)
Physical memory chunk(s):
0x0000000000001000 - 0x0000000000099fff, 626688 bytes (153 pages)
0x0000000000100000 - 0x00000000001fffff, 1048576 bytes (256 pages)
0x00000000019e1000 - 0x00000000efffffff, 3999395840 bytes (976415 pages)
0x0000000100000000 - 0x0000003b8ae37fff, 251438268416 bytes (61386296 pages)
avail memory = 244393992192 (233072 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <Xen HVM>
INTR: Adding local APIC 1 as a target
INTR: Adding local APIC 2 as a target
INTR: Adding local APIC 3 as a target
INTR: Adding local APIC 4 as a target
INTR: Adding local APIC 5 as a target
INTR: Adding local APIC 6 as a target
INTR: Adding local APIC 7 as a target
INTR: Adding local APIC 8 as a target
INTR: Adding local APIC 9 as a target
INTR: Adding local APIC 10 as a target
INTR: Adding local APIC 11 as a target
INTR: Adding local APIC 12 as a target
INTR: Adding local APIC 13 as a target
INTR: Adding local APIC 14 as a target
INTR: Adding local APIC 15 as a target
INTR: Adding local APIC 32 as a target
INTR: Adding local APIC 33 as a target
INTR: Adding local APIC 34 as a target
INTR: Adding local APIC 35 as a target
INTR: Adding local APIC 36 as a target
INTR: Adding local APIC 37 as a target
INTR: Adding local APIC 38 as a target
INTR: Adding local APIC 39 as a target
INTR: Adding local APIC 40 as a target
INTR: Adding local APIC 41 as a target
INTR: Adding local APIC 42 as a target
INTR: Adding local APIC 43 as a target
INTR: Adding local APIC 44 as a target
INTR: Adding local APIC 45 as a target
INTR: Adding local APIC 46 as a target
INTR: Adding local APIC 47 as a target
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 SMT threads
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
 cpu2 (AP): APIC ID:  2
 cpu3 (AP): APIC ID:  3
 cpu4 (AP): APIC ID:  4
 cpu5 (AP): APIC ID:  5
 cpu6 (AP): APIC ID:  6
 cpu7 (AP): APIC ID:  7
 cpu8 (AP): APIC ID:  8
 cpu9 (AP): APIC ID:  9
 cpu10 (AP): APIC ID: 10
 cpu11 (AP): APIC ID: 11
 cpu12 (AP): APIC ID: 12
 cpu13 (AP): APIC ID: 13
 cpu14 (AP): APIC ID: 14
 cpu15 (AP): APIC ID: 15
 cpu16 (AP): APIC ID: 32
 cpu17 (AP): APIC ID: 33
 cpu18 (AP): APIC ID: 34
 cpu19 (AP): APIC ID: 35
 cpu20 (AP): APIC ID: 36
 cpu21 (AP): APIC ID: 37
 cpu22 (AP): APIC ID: 38
 cpu23 (AP): APIC ID: 39
 cpu24 (AP): APIC ID: 40
 cpu25 (AP): APIC ID: 41
 cpu26 (AP): APIC ID: 42
 cpu27 (AP): APIC ID: 43
 cpu28 (AP): APIC ID: 44
 cpu29 (AP): APIC ID: 45
 cpu30 (AP): APIC ID: 46
 cpu31 (AP): APIC ID: 47
APIC: CPU 0 has ACPI ID 0
APIC: CPU 1 has ACPI ID 16
APIC: CPU 2 has ACPI ID 1
APIC: CPU 3 has ACPI ID 17
APIC: CPU 4 has ACPI ID 2
APIC: CPU 5 has ACPI ID 18
APIC: CPU 6 has ACPI ID 3
APIC: CPU 7 has ACPI ID 19
APIC: CPU 8 has ACPI ID 4
APIC: CPU 9 has ACPI ID 20
APIC: CPU 10 has ACPI ID 5
APIC: CPU 11 has ACPI ID 21
APIC: CPU 12 has ACPI ID 6
APIC: CPU 13 has ACPI ID 22
APIC: CPU 14 has ACPI ID 7
APIC: CPU 15 has ACPI ID 23
APIC: CPU 16 has ACPI ID 8
APIC: CPU 17 has ACPI ID 24
APIC: CPU 18 has ACPI ID 9
APIC: CPU 19 has ACPI ID 25
APIC: CPU 20 has ACPI ID 10
APIC: CPU 21 has ACPI ID 26
APIC: CPU 22 has ACPI ID 11
APIC: CPU 23 has ACPI ID 27
APIC: CPU 24 has ACPI ID 12
APIC: CPU 25 has ACPI ID 28
APIC: CPU 26 has ACPI ID 13
APIC: CPU 27 has ACPI ID 29
APIC: CPU 28 has ACPI ID 14
APIC: CPU 29 has ACPI ID 30
APIC: CPU 30 has ACPI ID 15
APIC: CPU 31 has ACPI ID 31
x86bios:  IVT 0x000000-0x0004ff at 0xfffffe0000000000
x86bios: SSEG 0x001000-0x001fff at 0xffffff80005e3000
x86bios: EBDA 0x09f000-0x09ffff at 0xfffffe000009f000
x86bios:  ROM 0x0a0000-0x0fefff at 0xfffffe00000a0000
random device not loaded; using insecure entropy
ULE: setup cpu 0
ULE: setup cpu 1
ULE: setup cpu 2
ULE: setup cpu 3
ULE: setup cpu 4
ULE: setup cpu 5
ULE: setup cpu 6
ULE: setup cpu 7
ULE: setup cpu 8
ULE: setup cpu 9
ULE: setup cpu 10
ULE: setup cpu 11
ULE: setup cpu 12
ULE: setup cpu 13
ULE: setup cpu 14
ULE: setup cpu 15
ULE: setup cpu 16
ULE: setup cpu 17
ULE: setup cpu 18
ULE: setup cpu 19
ULE: setup cpu 20
ULE: setup cpu 21
ULE: setup cpu 22
ULE: setup cpu 23
ULE: setup cpu 24
ULE: setup cpu 25
ULE: setup cpu 26
ULE: setup cpu 27
ULE: setup cpu 28
ULE: setup cpu 29
ULE: setup cpu 30
ULE: setup cpu 31
ACPI: RSDP 0xea020 00024 (v02    Xen)
ACPI: XSDT 0xfc015920 0005C (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: FACP 0xfc014980 000F4 (v04    Xen      HVM 00000000 HVML 00000000)
ACPI: DSDT 0xfc0035e0 11315 (v02    Xen      HVM 00000000 INTL 20090123)
ACPI: FACS 0xfc0035a0 00040
ACPI: APIC 0xfc014a80 00460 (v02    Xen      HVM 00000000 HVML 00000000)
ACPI: SRAT 0xfc014f60 008A8 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: HPET 0xfc015830 00038 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: WAET 0xfc015870 00028 (v01    Xen      HVM 00000000 HVML 00000000)
ACPI: SSDT 0xfc0158a0 00031 (v02    Xen      HVM 00000000 INTL 20090123)
ACPI: SSDT 0xfc0158e0 00031 (v02    Xen      HVM 00000000 INTL 20090123)
MADT: Found IO APIC ID 1, Interrupt 0 at 0xfec00000
ioapic0: Changing APIC ID to 1
ioapic0: Routing external 8259A's -> intpin 0
MADT: Interrupt override: source 0, irq 2
ioapic0: Routing IRQ 0 -> intpin 2
MADT: Interrupt override: source 5, irq 5
ioapic0: intpin 5 trigger: level
ioapic0: intpin 5 polarity: low
MADT: Interrupt override: source 10, irq 10
ioapic0: intpin 10 trigger: level
ioapic0: intpin 10 polarity: low
MADT: Interrupt override: source 11, irq 11
ioapic0: intpin 11 trigger: level
ioapic0: intpin 11 polarity: low
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0: intpin 9 polarity: low
ioapic0: intpin 9 trigger: level
ioapic0 <Version 1.1> irqs 0-47 on motherboard
cpu0 BSP:
     ID: 0x00000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
wlan: <802.11 Link Layer>
Event-channel device installed.
snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024]
feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1
feeder_rate_max=2016000 feeder_rate_round=25
kbd: new array size 4
kbd1 at kbdmux0
mem: <memory>
nfslock: pseudo-device
null: <null device, zero device>
random: <entropy source, Software, Yarrow>
VESA: INT 0x10 vector 0xc000:0x836e
VESA: information block
0000   56 45 53 41 00 02 f5 82 00 c0 00 00 00 00 40 00
0010   00 02 40 00 00 01 f5 82 00 c0 f5 82 00 c0 0e 83
0020   00 c0 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0030   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0040   01 01 10 01 11 01 12 01 03 01 13 01 14 01 15 01
0050   05 01 16 01 17 01 18 01 07 01 19 01 1a 01 ff ff
0060   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0070   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0080   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0090   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0100   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0110   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0120   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0130   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0140   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0150   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0160   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0170   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0180   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0190   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01a0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01b0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01c0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01d0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
VESA: 15 mode(s) found
VESA: v2.0, 4096k memory, flags:0x0, mode table:0xffffff80005fd040 (2000040)
VESA: VGABIOS Cirrus extension
VESA: VGABIOS Cirrus extension VGABIOS Cirrus extension 1.0
io: <I/O>
hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2
hpt27xx: RocketRAID 27xx controller driver v1.0
xen_et0: <Xen PV Clock> on motherboard
Event timer "XENTIMER" frequency 1000000000 Hz quality 950
Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
xen_et0: registered as a time-of-day clock (resolution 10000000us, adjustment
5.000000000s)
acpi0: <Xen> on motherboard
ACPI: All ACPI Tables successfully acquired
ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
cpu0: Processor \_SB_.PR00 (ACPI ID 0) -> APIC ID 0
cpu0: <ACPI CPU> on acpi0
cpu0: switching to generic Cx mode
cpu1: Processor \_SB_.PR01 (ACPI ID 1) -> APIC ID 2
cpu1: <ACPI CPU> on acpi0
cpu2: Processor \_SB_.PR02 (ACPI ID 2) -> APIC ID 4
cpu2: <ACPI CPU> on acpi0
cpu3: Processor \_SB_.PR03 (ACPI ID 3) -> APIC ID 6
cpu3: <ACPI CPU> on acpi0
cpu4: Processor \_SB_.PR04 (ACPI ID 4) -> APIC ID 8
cpu4: <ACPI CPU> on acpi0
cpu5: Processor \_SB_.PR05 (ACPI ID 5) -> APIC ID 10
cpu5: <ACPI CPU> on acpi0
cpu6: Processor \_SB_.PR06 (ACPI ID 6) -> APIC ID 12
cpu6: <ACPI CPU> on acpi0
cpu7: Processor \_SB_.PR07 (ACPI ID 7) -> APIC ID 14
cpu7: <ACPI CPU> on acpi0
cpu8: Processor \_SB_.PR08 (ACPI ID 8) -> APIC ID 16
cpu8: <ACPI CPU> on acpi0
cpu9: Processor \_SB_.PR09 (ACPI ID 9) -> APIC ID 18
cpu9: <ACPI CPU> on acpi0
cpu10: Processor \_SB_.PR0A (ACPI ID 10) -> APIC ID 20
cpu10: <ACPI CPU> on acpi0
cpu11: Processor \_SB_.PR0B (ACPI ID 11) -> APIC ID 22
cpu11: <ACPI CPU> on acpi0
cpu12: Processor \_SB_.PR0C (ACPI ID 12) -> APIC ID 24
cpu12: <ACPI CPU> on acpi0
cpu13: Processor \_SB_.PR0D (ACPI ID 13) -> APIC ID 26
cpu13: <ACPI CPU> on acpi0
cpu14: Processor \_SB_.PR0E (ACPI ID 14) -> APIC ID 28
cpu14: <ACPI CPU> on acpi0
cpu15: Processor \_SB_.PR0F (ACPI ID 15) -> APIC ID 30
cpu15: <ACPI CPU> on acpi0
cpu16: Processor \_SB_.PR10 (ACPI ID 16) -> APIC ID 1
cpu16: <ACPI CPU> on acpi0
cpu17: Processor \_SB_.PR11 (ACPI ID 17) -> APIC ID 3
cpu17: <ACPI CPU> on acpi0
cpu18: Processor \_SB_.PR12 (ACPI ID 18) -> APIC ID 5
cpu18: <ACPI CPU> on acpi0
cpu19: Processor \_SB_.PR13 (ACPI ID 19) -> APIC ID 7
cpu19: <ACPI CPU> on acpi0
cpu20: Processor \_SB_.PR14 (ACPI ID 20) -> APIC ID 9
cpu20: <ACPI CPU> on acpi0
cpu21: Processor \_SB_.PR15 (ACPI ID 21) -> APIC ID 11
cpu21: <ACPI CPU> on acpi0
cpu22: Processor \_SB_.PR16 (ACPI ID 22) -> APIC ID 13
cpu22: <ACPI CPU> on acpi0
cpu23: Processor \_SB_.PR17 (ACPI ID 23) -> APIC ID 15
cpu23: <ACPI CPU> on acpi0
cpu24: Processor \_SB_.PR18 (ACPI ID 24) -> APIC ID 17
cpu24: <ACPI CPU> on acpi0
cpu25: Processor \_SB_.PR19 (ACPI ID 25) -> APIC ID 19
cpu25: <ACPI CPU> on acpi0
cpu26: Processor \_SB_.PR1A (ACPI ID 26) -> APIC ID 21
cpu26: <ACPI CPU> on acpi0
cpu27: Processor \_SB_.PR1B (ACPI ID 27) -> APIC ID 23
cpu27: <ACPI CPU> on acpi0
cpu28: Processor \_SB_.PR1C (ACPI ID 28) -> APIC ID 25
cpu28: <ACPI CPU> on acpi0
cpu29: Processor \_SB_.PR1D (ACPI ID 29) -> APIC ID 27
cpu29: <ACPI CPU> on acpi0
cpu30: Processor \_SB_.PR1E (ACPI ID 30) -> APIC ID 29
cpu30: <ACPI CPU> on acpi0
cpu31: Processor \_SB_.PR1F (ACPI ID 31) -> APIC ID 31
cpu31: <ACPI CPU> on acpi0
ACPI: Processor \_SB_.PR20 (ACPI ID 32) ignored
ACPI: Processor \_SB_.PR21 (ACPI ID 33) ignored
ACPI: Processor \_SB_.PR22 (ACPI ID 34) ignored
ACPI: Processor \_SB_.PR23 (ACPI ID 35) ignored
ACPI: Processor \_SB_.PR24 (ACPI ID 36) ignored
ACPI: Processor \_SB_.PR25 (ACPI ID 37) ignored
ACPI: Processor \_SB_.PR26 (ACPI ID 38) ignored
ACPI: Processor \_SB_.PR27 (ACPI ID 39) ignored
ACPI: Processor \_SB_.PR28 (ACPI ID 40) ignored
ACPI: Processor \_SB_.PR29 (ACPI ID 41) ignored
ACPI: Processor \_SB_.PR2A (ACPI ID 42) ignored
ACPI: Processor \_SB_.PR2B (ACPI ID 43) ignored
ACPI: Processor \_SB_.PR2C (ACPI ID 44) ignored
ACPI: Processor \_SB_.PR2D (ACPI ID 45) ignored
ACPI: Processor \_SB_.PR2E (ACPI ID 46) ignored
ACPI: Processor \_SB_.PR2F (ACPI ID 47) ignored
ACPI: Processor \_SB_.PR30 (ACPI ID 48) ignored
ACPI: Processor \_SB_.PR31 (ACPI ID 49) ignored
ACPI: Processor \_SB_.PR32 (ACPI ID 50) ignored
ACPI: Processor \_SB_.PR33 (ACPI ID 51) ignored
ACPI: Processor \_SB_.PR34 (ACPI ID 52) ignored
ACPI: Processor \_SB_.PR35 (ACPI ID 53) ignored
ACPI: Processor \_SB_.PR36 (ACPI ID 54) ignored
ACPI: Processor \_SB_.PR37 (ACPI ID 55) ignored
ACPI: Processor \_SB_.PR38 (ACPI ID 56) ignored
ACPI: Processor \_SB_.PR39 (ACPI ID 57) ignored
ACPI: Processor \_SB_.PR3A (ACPI ID 58) ignored
ACPI: Processor \_SB_.PR3B (ACPI ID 59) ignored
ACPI: Processor \_SB_.PR3C (ACPI ID 60) ignored
ACPI: Processor \_SB_.PR3D (ACPI ID 61) ignored
ACPI: Processor \_SB_.PR3E (ACPI ID 62) ignored
ACPI: Processor \_SB_.PR3F (ACPI ID 63) ignored
ACPI: Processor \_SB_.PR40 (ACPI ID 64) ignored
ACPI: Processor \_SB_.PR41 (ACPI ID 65) ignored
ACPI: Processor \_SB_.PR42 (ACPI ID 66) ignored
ACPI: Processor \_SB_.PR43 (ACPI ID 67) ignored
ACPI: Processor \_SB_.PR44 (ACPI ID 68) ignored
ACPI: Processor \_SB_.PR45 (ACPI ID 69) ignored
ACPI: Processor \_SB_.PR46 (ACPI ID 70) ignored
ACPI: Processor \_SB_.PR47 (ACPI ID 71) ignored
ACPI: Processor \_SB_.PR48 (ACPI ID 72) ignored
ACPI: Processor \_SB_.PR49 (ACPI ID 73) ignored
ACPI: Processor \_SB_.PR4A (ACPI ID 74) ignored
ACPI: Processor \_SB_.PR4B (ACPI ID 75) ignored
ACPI: Processor \_SB_.PR4C (ACPI ID 76) ignored
ACPI: Processor \_SB_.PR4D (ACPI ID 77) ignored
ACPI: Processor \_SB_.PR4E (ACPI ID 78) ignored
ACPI: Processor \_SB_.PR4F (ACPI ID 79) ignored
ACPI: Processor \_SB_.PR50 (ACPI ID 80) ignored
ACPI: Processor \_SB_.PR51 (ACPI ID 81) ignored
ACPI: Processor \_SB_.PR52 (ACPI ID 82) ignored
ACPI: Processor \_SB_.PR53 (ACPI ID 83) ignored
ACPI: Processor \_SB_.PR54 (ACPI ID 84) ignored
ACPI: Processor \_SB_.PR55 (ACPI ID 85) ignored
ACPI: Processor \_SB_.PR56 (ACPI ID 86) ignored
ACPI: Processor \_SB_.PR57 (ACPI ID 87) ignored
ACPI: Processor \_SB_.PR58 (ACPI ID 88) ignored
ACPI: Processor \_SB_.PR59 (ACPI ID 89) ignored
ACPI: Processor \_SB_.PR5A (ACPI ID 90) ignored
ACPI: Processor \_SB_.PR5B (ACPI ID 91) ignored
ACPI: Processor \_SB_.PR5C (ACPI ID 92) ignored
ACPI: Processor \_SB_.PR5D (ACPI ID 93) ignored
ACPI: Processor \_SB_.PR5E (ACPI ID 94) ignored
ACPI: Processor \_SB_.PR5F (ACPI ID 95) ignored
ACPI: Processor \_SB_.PR60 (ACPI ID 96) ignored
ACPI: Processor \_SB_.PR61 (ACPI ID 97) ignored
ACPI: Processor \_SB_.PR62 (ACPI ID 98) ignored
ACPI: Processor \_SB_.PR63 (ACPI ID 99) ignored
ACPI: Processor \_SB_.PR64 (ACPI ID 100) ignored
ACPI: Processor \_SB_.PR65 (ACPI ID 101) ignored
ACPI: Processor \_SB_.PR66 (ACPI ID 102) ignored
ACPI: Processor \_SB_.PR67 (ACPI ID 103) ignored
ACPI: Processor \_SB_.PR68 (ACPI ID 104) ignored
ACPI: Processor \_SB_.PR69 (ACPI ID 105) ignored
ACPI: Processor \_SB_.PR6A (ACPI ID 106) ignored
ACPI: Processor \_SB_.PR6B (ACPI ID 107) ignored
ACPI: Processor \_SB_.PR6C (ACPI ID 108) ignored
ACPI: Processor \_SB_.PR6D (ACPI ID 109) ignored
ACPI: Processor \_SB_.PR6E (ACPI ID 110) ignored
ACPI: Processor \_SB_.PR6F (ACPI ID 111) ignored
ACPI: Processor \_SB_.PR70 (ACPI ID 112) ignored
ACPI: Processor \_SB_.PR71 (ACPI ID 113) ignored
ACPI: Processor \_SB_.PR72 (ACPI ID 114) ignored
ACPI: Processor \_SB_.PR73 (ACPI ID 115) ignored
ACPI: Processor \_SB_.PR74 (ACPI ID 116) ignored
ACPI: Processor \_SB_.PR75 (ACPI ID 117) ignored
ACPI: Processor \_SB_.PR76 (ACPI ID 118) ignored
ACPI: Processor \_SB_.PR77 (ACPI ID 119) ignored
ACPI: Processor \_SB_.PR78 (ACPI ID 120) ignored
ACPI: Processor \_SB_.PR79 (ACPI ID 121) ignored
ACPI: Processor \_SB_.PR7A (ACPI ID 122) ignored
ACPI: Processor \_SB_.PR7B (ACPI ID 123) ignored
ACPI: Processor \_SB_.PR7C (ACPI ID 124) ignored
ACPI: Processor \_SB_.PR7D (ACPI ID 125) ignored
ACPI: Processor \_SB_.PR7E (ACPI ID 126) ignored
ACPI: Processor \_SB_.PR7F (ACPI ID 127) ignored
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
hpet0: vendor 0x8086, rev 0x1, 62500000Hz 64bit, 3 timers, legacy route
hpet0:  t0: irqs 0x00f00000 (0), 64bit, periodic
hpet0:  t1: irqs 0x00f00000 (0), 64bit, periodic
hpet0:  t2: irqs 0x00f00000 (0), 64bit, periodic
Timecounter "HPET" frequency 62500000 Hz quality 950
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
ioapic0: routing intpin 2 (ISA IRQ 0) to lapic 0 vector 49
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
atrtc0: not installed as time-of-day clock: clock xen_et has higher resolution
ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 50
Event timer "RTC" frequency 32768 Hz quality 0
ACPI timer: 1/20 1/19 1/19 1/20 1/19 1/23 1/20 1/19 1/18 1/17 -> 10
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0xb008-0xb00b on acpi0
pci_link0:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0    5   N     0  5 10 11
  Validation          0    5   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link1:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0   10   N     0  5 10 11
  Validation          0   10   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link2:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0   11   N     0  5 10 11
  Validation          0   11   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pci_link3:        Index  IRQ  Rtd  Ref  IRQs
  Initial Probe       0    5   N     0  5 10 11
  Validation          0    5   N     0  5 10 11
  After Disable       0  255   N     0  5 10 11
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pcib0: decoding 4 range 0-0xcf7
pcib0: decoding 4 range 0xd00-0xffff
pcib0: decoding 3 range 0xa0000-0xbffff
pcib0: decoding 3 range 0xf0000000-0xfbffffff
pci0: <ACPI PCI bus> on pcib0
pci0: domain=0, physical bus=0
found->	vendor=0x8086, dev=0x1237, revid=0x02
	domain=0, bus=0, slot=0, func=0
	class=06-00-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0004, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
found->	vendor=0x8086, dev=0x7000, revid=0x00
	domain=0, bus=0, slot=1, func=0
	class=06-01-00, hdrtype=0x00, mfdev=1
	cmdreg=0x0007, statreg=0x0200, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
found->	vendor=0x8086, dev=0x7010, revid=0x00
	domain=0, bus=0, slot=1, func=1
	class=01-01-80, hdrtype=0x00, mfdev=0
	cmdreg=0x0005, statreg=0x0280, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
pcib0: allocated type 4 (0x1f0-0x1f7) for rid 10 of pci0:0:1:1
pcib0: allocated type 4 (0x3f6-0x3f6) for rid 14 of pci0:0:1:1
pcib0: allocated type 4 (0x170-0x177) for rid 18 of pci0:0:1:1
pcib0: allocated type 4 (0x376-0x376) for rid 1c of pci0:0:1:1
	map[20]: type I/O Port, range 32, base 0xc100, size  4, enabled
pcib0: allocated type 4 (0xc100-0xc10f) for rid 20 of pci0:0:1:1
found->	vendor=0x8086, dev=0x7113, revid=0x01
	domain=0, bus=0, slot=1, func=3
	class=06-80-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0004, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	intpin=a, irq=10
pcib0: matched entry for 0.1.INTA
pcib0: slot 1 INTA hardwired to IRQ 20
found->	vendor=0x1013, dev=0x00b8, revid=0x00
	domain=0, bus=0, slot=2, func=0
	class=03-00-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	map[10]: type Prefetchable Memory, range 32, base 0xf0000000, size 25, enabled
pcib0: allocated type 3 (0xf0000000-0xf1ffffff) for rid 10 of pci0:0:2:0
	map[14]: type Memory, range 32, base 0xf3000000, size 12, enabled
pcib0: allocated type 3 (0xf3000000-0xf3000fff) for rid 14 of pci0:0:2:0
found->	vendor=0x5853, dev=0x0001, revid=0x01
	domain=0, bus=0, slot=3, func=0
	class=ff-80-00, hdrtype=0x00, mfdev=0
	cmdreg=0x0007, statreg=0x0000, cachelnsz=0 (dwords)
	lattimer=0x00 (0 ns), mingnt=0x00 (0 ns), maxlat=0x00 (0 ns)
	intpin=a, irq=5
	map[10]: type I/O Port, range 32, base 0xc000, size  8, enabled
pcib0: allocated type 4 (0xc000-0xc0ff) for rid 10 of pci0:0:3:0
	map[14]: type Prefetchable Memory, range 32, base 0xf2000000, size 24, enabled
pcib0: allocated type 3 (0xf2000000-0xf2ffffff) for rid 14 of pci0:0:3:0
pcib0: matched entry for 0.3.INTA
pcib0: slot 3 INTA hardwired to IRQ 28
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc100-0xc10f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 0 vector 51
ata1: <ATA channel> at channel 1 on atapci0
ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 0 vector 52
pci0: <bridge> at device 1.3 (no driver attached)
vgapci0: <VGA-compatible display> mem
0xf0000000-0xf1ffffff,0xf3000000-0xf3000fff at device 2.0 on pci0
xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xf2000000-0xf2ffffff irq
28 at device 3.0 on pci0
ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 0 vector 53
xenstore0: <XenStore> on xenpci0
Grant table initialized
psmcpnp0: <PS/2 mouse port> irq 12 on acpi0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
atkbd: the current kbd controller command byte 0061
atkbd: keyboard ID 0x41ab (2)
kbdc: RESET_KBD return code:00fa
kbdc: RESET_KBD status:00aa
kbd0 at atkbd0
kbd0: atkbd0, AT 101/102 (2), config:0x0, flags:0x1d0000
ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 0 vector 54
atkbd0: [GIANT-LOCKED]
psm0: current command byte:0061
kbdc: TEST_AUX_PORT status:0000
kbdc: RESET_AUX return code:00fa
kbdc: RESET_AUX status:00aa
kbdc: RESET_AUX ID:0000
kbdc: RESET_AUX return code:00fa
kbdc: RESET_AUX status:00aa
kbdc: RESET_AUX ID:0000
psm: status 00 02 64
psm: status 00 00 64
psm: status 00 03 64
psm: status 00 03 64
psm: data 08 00 00
psm: status 00 02 64
psm0: <PS/2 Mouse> irq 12 on atkbdc0
ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 0 vector 55
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4-00, 5 buttons
psm0: config:00000000, flags:00000008, packet size:4
psm0: syncmask:08, syncbits:00
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 0 vector 56
uart0: fast interrupt
uart0: console (9600,n,8,1)
ACPI: Enabled 2 GPEs in block 00 to 0F
acpi0: wakeup code va 0xffffffc2ce049000 pa 0x30000
ahc_isa_identify 0: ioport 0xc00 alloc failed
ahc_isa_identify 1: ioport 0x1c00 alloc failed
ahc_isa_identify 2: ioport 0x2c00 alloc failed
ahc_isa_identify 3: ioport 0x3c00 alloc failed
ahc_isa_identify 4: ioport 0x4c00 alloc failed
ahc_isa_identify 5: ioport 0x5c00 alloc failed
ahc_isa_identify 6: ioport 0x6c00 alloc failed
ahc_isa_identify 7: ioport 0x7c00 alloc failed
ahc_isa_identify 8: ioport 0x8c00 alloc failed
ahc_isa_identify 9: ioport 0x9c00 alloc failed
ahc_isa_identify 10: ioport 0xac00 alloc failed
ahc_isa_identify 11: ioport 0xbc00 alloc failed
ahc_isa_identify 12: ioport 0xcc00 alloc failed
ahc_isa_identify 13: ioport 0xdc00 alloc failed
ahc_isa_identify 14: ioport 0xec00 alloc failed
pcib0: allocated type 3 (0xa0000-0xa07ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa0800-0xa0fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa1000-0xa17ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa1800-0xa1fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa2000-0xa27ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa2800-0xa2fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa3000-0xa37ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa3800-0xa3fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa4000-0xa47ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa4800-0xa4fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa5000-0xa57ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa5800-0xa5fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa6000-0xa67ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa6800-0xa6fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa7000-0xa77ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa7800-0xa7fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa8000-0xa87ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa8800-0xa8fff) for rid 0 of orm0
pcib0: allocated type 3 (0xa9000-0xa97ff) for rid 0 of orm0
pcib0: allocated type 3 (0xa9800-0xa9fff) for rid 0 of orm0
pcib0: allocated type 3 (0xaa000-0xaa7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xaa800-0xaafff) for rid 0 of orm0
pcib0: allocated type 3 (0xab000-0xab7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xab800-0xabfff) for rid 0 of orm0
pcib0: allocated type 3 (0xac000-0xac7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xac800-0xacfff) for rid 0 of orm0
pcib0: allocated type 3 (0xad000-0xad7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xad800-0xadfff) for rid 0 of orm0
pcib0: allocated type 3 (0xae000-0xae7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xae800-0xaefff) for rid 0 of orm0
pcib0: allocated type 3 (0xaf000-0xaf7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xaf800-0xaffff) for rid 0 of orm0
pcib0: allocated type 3 (0xb0000-0xb07ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb0800-0xb0fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb1000-0xb17ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb1800-0xb1fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb2000-0xb27ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb2800-0xb2fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb3000-0xb37ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb3800-0xb3fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb4000-0xb47ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb4800-0xb4fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb5000-0xb57ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb5800-0xb5fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb6000-0xb67ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb6800-0xb6fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb7000-0xb77ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb7800-0xb7fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb8000-0xb87ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb8800-0xb8fff) for rid 0 of orm0
pcib0: allocated type 3 (0xb9000-0xb97ff) for rid 0 of orm0
pcib0: allocated type 3 (0xb9800-0xb9fff) for rid 0 of orm0
pcib0: allocated type 3 (0xba000-0xba7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xba800-0xbafff) for rid 0 of orm0
pcib0: allocated type 3 (0xbb000-0xbb7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbb800-0xbbfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbc000-0xbc7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbc800-0xbcfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbd000-0xbd7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbd800-0xbdfff) for rid 0 of orm0
pcib0: allocated type 3 (0xbe000-0xbe7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbe800-0xbefff) for rid 0 of orm0
pcib0: allocated type 3 (0xbf000-0xbf7ff) for rid 0 of orm0
pcib0: allocated type 3 (0xbf800-0xbffff) for rid 0 of orm0
ex_isa_identify()
isa_probe_children: disabling PnP devices
atkbdc: atkbdc0 already exists; skipping it
atrtc: atrtc0 already exists; skipping it
attimer: attimer0 already exists; skipping it
sc: sc0 already exists; skipping it
uart: uart0 already exists; skipping it
isa_probe_children: probing non-PnP devices
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x100>
sc0: fb0, kbd1, terminal emulator: scteken (teken terminal)
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
pcib0: allocated type 4 (0x3c0-0x3df) for rid 0 of vga0
pcib0: allocated type 3 (0xa0000-0xbffff) for rid 0 of vga0
fdc0: No FDOUT register!
fdc0 failed to probe at port 0x3f0 irq 6 drq 2 on isa0
ppc0: cannot reserve I/O port range
ppc0 failed to probe at irq 7 on isa0
pcib0: allocated type 4 (0x2f8-0x2ff) for rid 0 of uart1
uart1 failed to probe at port 0x2f8-0x2ff irq 3 on isa0
wbwd0 failed to probe on isa0
isa_probe_children: probing PnP devices
Device configuration finished.
procfs registered
Timecounters tick every 10.000 msec
vlan: initialized, using hash tables with chaining
tcp_init: net.inet.tcp.tcbhashsize auto tuned to 2097152
lo0: bpf attached
hptrr: no controller detected.
hpt27xx: no controller detected.
xctrl0: <Xen Control Device> on xenstore0
ata0: reset tp1 mask=03 ostat0=00 ostat1=00
xenbusb_front0: <Xen Frontend Devices> on xenstore0
ata0: stat0=0x00 err=0x00 lsb=0x00 msb=0x00
ata0: stat1=0x00 err=0x00 lsb=0x00 msb=0x00
ata0: reset tp2 stat0=00 stat1=00 devices=0x0
ata1: reset tp1 mask=03 ostat0=00 ostat1=00
ata1: stat0=0x00 err=0x00 lsb=0x00 msb=0x00
ata1: stat1=0x00 err=0x00 lsb=0x00 msb=0x00
ata1: reset tp2 stat0=00 stat1=00 devices=0x0
xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
xn0: bpf attached
xn0: Ethernet address: 22:00:0a:94:b5:05
xenbusb_back0: <Xen Backend Devices> on xenstore0
xn0: backend features: feature-sg feature-gso-tcp4
xbd0: 10240MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
xbd0: attaching as ad0
xbd0: disk supports cache flush using: barriers
GEOM: new disk ad0
xbd1: 122866MB <Virtual Block Device> at device/vbd/51728 on xenbusb_front0
xbd1: disk supports cache flush using: flush
xbd2: 122866MB <Virtual Block Device> at device/vbd/51744 on xenbusb_front0
xbd2: disk supports cache flush using: flush
SMP: AP CPU #4 Launched!
cpu4 AP:
     ID: 0x04000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #1 Launched!
cpu1 AP:
     ID: 0x01000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #10 Launched!
cpu10 AP:
     ID: 0x0a000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #11 Launched!
cpu11 AP:
     ID: 0x0b000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #16 Launched!
cpu16 AP:
     ID: 0x20000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #22 Launched!
cpu22 AP:
     ID: 0x26000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #23 Launched!
cpu23 AP:
     ID: 0x27000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #19 Launched!
cpu19 AP:
     ID: 0x23000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #17 Launched!
cpu17 AP:
     ID: 0x21000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #7 Launched!
cpu7 AP:
     ID: 0x07000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #27 Launched!
cpu27 AP:
     ID: 0x2b000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #26 Launched!
cpu26 AP:
     ID: 0x2a000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #25 Launched!
cpu25 AP:
     ID: 0x29000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #3 Launched!
cpu3 AP:
     ID: 0x03000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #2 Launched!
cpu2 AP:
     ID: 0x02000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #21 Launched!
cpu21 AP:
     ID: 0x25000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #20 Launched!
cpu20 AP:
     ID: 0x24000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #29 Launched!
cpu29 AP:
     ID: 0x2d000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #31 Launched!
cpu31 AP:
     ID: 0x2f000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #18 Launched!
cpu18 AP:
     ID: 0x22000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #30 Launched!
cpu30 AP:
     ID: 0x2e000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #13 Launched!
cpu13 AP:
     ID: 0x0d000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #24 Launched!
cpu24 AP:
     ID: 0x28000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #28 Launched!
cpu28 AP:
     ID: 0x2c000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #8 Launched!
cpu8 AP:
     ID: 0x08000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #14 Launched!
cpu14 AP:
     ID: 0x0e000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #5 Launched!
cpu5 AP:
     ID: 0x05000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #12 Launched!
cpu12 AP:
     ID: 0x0c000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #15 Launched!
cpu15 AP:
     ID: 0x0f000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #6 Launched!
cpu6 AP:
     ID: 0x06000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
SMP: AP CPU #9 Launched!
cpu9 AP:
     ID: 0x09000000   VER: 0x00050014 LDR: 0x00000000 DFR: 0xffffffff
  lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
  timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 1 vector 48
panic: xen_et0: Error -22 setting singleshot timer to 506167555095805 for vCPU#1

cpuid = 1
KDB: stack backtrace:
#0 0xffffffff808e6ed0 at kdb_backtrace+0x60
#1 0xffffffff808b22b6 at vpanic+0x126
#2 0xffffffff808b2343 at panic+0x43
#3 0xffffffff807a445a at xentimer_et_start+0xca
#4 0xffffffff80cb365d at loadtimer+0xfd
#5 0xffffffff80cb38d7 at cpu_new_callout+0xc7
#6 0xffffffff808c5ca1 at callout_process+0x2c1
#7 0xffffffff80cb25d5 at handleevents+0x185
#8 0xffffffff80cb324b at cpu_initclocks_ap+0xcb
#9 0xffffffff80c2fc84 at init_secondary+0x474
Uptime: 3s
Automatic reboot in 15 seconds - press a key on the console to abort
--> Press a key on the console to reboot,
--> or switch off the system now.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 23/05/13 21:09, Colin Percival wrote:
> On 05/23/13 02:06, Roger Pau Monné wrote:
>> On 22/05/13 22:03, Colin Percival wrote:
>>> Testing on a cr1.8xlarge EC2 instance, I get Xen 4.2, but it ends up with
>>> a panic -- console output below.  I can get a backtrace and possibly even
>>> a dump if those would help.
>>
>> Thanks for the test, I've been using Xen 4.2 (and 4.3) without problems 
>> so far. By looking at the Xen code, the only reason the timer setup 
>> could return -22 (EINVAL), is that we try to set the timer for a 
>> different vCPU than the one we are running on.
>>
>> I've been able to boot a 32 vCPU DomU on my 8way box using Xen 4.2.1 
>> (using both qemu-xen and qemu-xen-traditional device models), so I'm 
>> unsure if this could be due to some patch Amazon applies to Xen. Could 
>> you try the following patch and post the error message? I would like to 
>> see if the cpuid reported by kdb and the vCPU that we are trying to set 
>> the timer are the same.
> 
> Looks like there's agreement about the cpuids here.  Anything else I should
> try testing?

Thanks for the test, this is what I expected. I'm a little bit out of
ideas since I'm not able to reproduce this on upstream Xen 4.2. Without
knowing what's happening inside the hypervisor it's hard to tell what's
wrong. It would be interesting to try if the same happens with a Linux
PVHVM (not PV) running on the same instance type.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 24/05/13 12:11, Roger Pau Monné wrote:
> On 23/05/13 21:09, Colin Percival wrote:
>> On 05/23/13 02:06, Roger Pau Monné wrote:
>>> On 22/05/13 22:03, Colin Percival wrote:
>>>> Testing on a cr1.8xlarge EC2 instance, I get Xen 4.2, but it ends up with
>>>> a panic -- console output below.  I can get a backtrace and possibly even
>>>> a dump if those would help.
>>>
>>> Thanks for the test, I've been using Xen 4.2 (and 4.3) without problems 
>>> so far. By looking at the Xen code, the only reason the timer setup 
>>> could return -22 (EINVAL), is that we try to set the timer for a 
>>> different vCPU than the one we are running on.
>>>
>>> I've been able to boot a 32 vCPU DomU on my 8way box using Xen 4.2.1 
>>> (using both qemu-xen and qemu-xen-traditional device models), so I'm 
>>> unsure if this could be due to some patch Amazon applies to Xen. Could 
>>> you try the following patch and post the error message? I would like to 
>>> see if the cpuid reported by kdb and the vCPU that we are trying to set 
>>> the timer are the same.
>>
>> Looks like there's agreement about the cpuids here.  Anything else I should
>> try testing?
> 
> Thanks for the test, this is what I expected. I'm a little bit out of
> ideas since I'm not able to reproduce this on upstream Xen 4.2. Without
> knowing what's happening inside the hypervisor it's hard to tell what's
> wrong. It would be interesting to try if the same happens with a Linux
> PVHVM (not PV) running on the same instance type.

Hello Matt,

Colin has found an issue on the FreeBSD PVHVM port that I haven't been
able to reproduce using open source Xen, even when using the same
version as the one reported by EC2. Is there anyway you could provide
some help debugging this? Without seeing the Xen code that causes
VCPUOP_set_singleshot_timer to return EINVAL it is quite hard to figure
out what's happening inside the hypervisor.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Matt Wilson]

On Tue, May 28, 2013 at 06:15:00PM +0200, Roger Pau Monné wrote:
> On 24/05/13 12:11, Roger Pau Monné wrote:
> > 
> > Thanks for the test, this is what I expected. I'm a little bit out of
> > ideas since I'm not able to reproduce this on upstream Xen 4.2. Without
> > knowing what's happening inside the hypervisor it's hard to tell what's
> > wrong. It would be interesting to try if the same happens with a Linux
> > PVHVM (not PV) running on the same instance type.
> 
> Hello Matt,
> 
> Colin has found an issue on the FreeBSD PVHVM port that I haven't been
> able to reproduce using open source Xen, even when using the same
> version as the one reported by EC2. Is there anyway you could provide
> some help debugging this? Without seeing the Xen code that causes
> VCPUOP_set_singleshot_timer to return EINVAL it is quite hard to figure
> out what's happening inside the hypervisor.

Hi Roger,

VCPUOP_set_singleshot_timer returns -EINVAL when:

1) the specified vCPU ID is out of range (<0 or >MAX_VIRT_CPUS)
2) the specified vCPU ID doesn't match the running vCPU.

It seems that there is a confusion between the logical vCPU ID and the
local APIC physical ID.

I added some debugging to case 2):

diff --git a/xen/common/domain.c b/xen/common/domain.c
index e728819..e3efb8c 100644
--- a/xen/common/domain.c
+++ b/xen/common/domain.c
@@ -901,7 +901,12 @@ long do_vcpu_op(int cmd, int vcpuid, XEN_GUEST_HANDLE(void) arg)
         struct vcpu_set_singleshot_timer set;
 
         if ( v != current )
+        {
+            printk("Domain %d (vcpu#%d) VCPUOP_set_singleshot_timer specified vcpuid %d\n",
+                   d->domain_id, current->vcpu_id, vcpuid);
+            
             return -EINVAL;
+        }
 
         if ( copy_from_guest(&set, arg, 1) )
             return -EFAULT;


The output from booting ami-e75c358e on a cr1.8xlarge:

(XEN) Domain 1 (vcpu#16) VCPUOP_set_singleshot_timer specified vcpuid 1
(XEN) Domain 1 (vcpu#7) VCPUOP_set_singleshot_timer specified vcpuid 14
(XEN) Domain 1 (vcpu#23) VCPUOP_set_singleshot_timer specified vcpuid 15
(XEN) Domain 1 (vcpu#11) VCPUOP_set_singleshot_timer specified vcpuid 22
(XEN) Domain 1 (vcpu#27) VCPUOP_set_singleshot_timer specified vcpuid 23
(XEN) Domain 1 (vcpu#18) VCPUOP_set_singleshot_timer specified vcpuid 5
(XEN) Domain 1 (vcpu#2) VCPUOP_set_singleshot_timer specified vcpuid 4
(XEN) Domain 1 (vcpu#9) VCPUOP_set_singleshot_timer specified vcpuid 18
(XEN) Domain 1 (vcpu#25) VCPUOP_set_singleshot_timer specified vcpuid 19
(XEN) Domain 1 (vcpu#1) VCPUOP_set_singleshot_timer specified vcpuid 2
(XEN) Domain 1 (vcpu#6) VCPUOP_set_singleshot_timer specified vcpuid 12
(XEN) Domain 1 (vcpu#22) VCPUOP_set_singleshot_timer specified vcpuid 13
(XEN) Domain 1 (vcpu#26) VCPUOP_set_singleshot_timer specified vcpuid 21
(XEN) Domain 1 (vcpu#10) VCPUOP_set_singleshot_timer specified vcpuid 20
(XEN) Domain 1 (vcpu#14) VCPUOP_set_singleshot_timer specified vcpuid 28
(XEN) Domain 1 (vcpu#30) VCPUOP_set_singleshot_timer specified vcpuid 29
(XEN) Domain 1 (vcpu#3) VCPUOP_set_singleshot_timer specified vcpuid 6
(XEN) Domain 1 (vcpu#19) VCPUOP_set_singleshot_timer specified vcpuid 7
(XEN) Domain 1 (vcpu#12) VCPUOP_set_singleshot_timer specified vcpuid 24
(XEN) Domain 1 (vcpu#28) VCPUOP_set_singleshot_timer specified vcpuid 25
(XEN) Domain 1 (vcpu#5) VCPUOP_set_singleshot_timer specified vcpuid 10
(XEN) Domain 1 (vcpu#21) VCPUOP_set_singleshot_timer specified vcpuid 11
(XEN) Domain 1 (vcpu#24) VCPUOP_set_singleshot_timer specified vcpuid 17
(XEN) Domain 1 (vcpu#8) VCPUOP_set_singleshot_timer specified vcpuid 16
(XEN) Domain 1 (vcpu#17) VCPUOP_set_singleshot_timer specified vcpuid 3
(XEN) Domain 1 (vcpu#20) VCPUOP_set_singleshot_timer specified vcpuid 9
(XEN) Domain 1 (vcpu#4) VCPUOP_set_singleshot_timer specified vcpuid 8
(XEN) Domain 1 (vcpu#13) VCPUOP_set_singleshot_timer specified vcpuid 26
(XEN) Domain 1 (vcpu#29) VCPUOP_set_singleshot_timer specified vcpuid 27
(XEN) Domain 1 (vcpu#15) VCPUOP_set_singleshot_timer specified vcpuid 30

Note from the FreeBSD boot output:
APIC: CPU 0 has ACPI ID 0
APIC: CPU 1 has ACPI ID 16
APIC: CPU 2 has ACPI ID 1
APIC: CPU 3 has ACPI ID 17
APIC: CPU 4 has ACPI ID 2
APIC: CPU 5 has ACPI ID 18
APIC: CPU 6 has ACPI ID 3
APIC: CPU 7 has ACPI ID 19
APIC: CPU 8 has ACPI ID 4
APIC: CPU 9 has ACPI ID 20
APIC: CPU 10 has ACPI ID 5
APIC: CPU 11 has ACPI ID 21
APIC: CPU 12 has ACPI ID 6
APIC: CPU 13 has ACPI ID 22
APIC: CPU 14 has ACPI ID 7
APIC: CPU 15 has ACPI ID 23
APIC: CPU 16 has ACPI ID 8
APIC: CPU 17 has ACPI ID 24
APIC: CPU 18 has ACPI ID 9
APIC: CPU 19 has ACPI ID 25
APIC: CPU 20 has ACPI ID 10
APIC: CPU 21 has ACPI ID 26
APIC: CPU 22 has ACPI ID 11
APIC: CPU 23 has ACPI ID 27
APIC: CPU 24 has ACPI ID 12
APIC: CPU 25 has ACPI ID 28
APIC: CPU 26 has ACPI ID 13
APIC: CPU 27 has ACPI ID 29
APIC: CPU 28 has ACPI ID 14
APIC: CPU 29 has ACPI ID 30
APIC: CPU 30 has ACPI ID 15
APIC: CPU 31 has ACPI ID 31

--msw

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/28/13 12:18, Matt Wilson wrote:
> VCPUOP_set_singleshot_timer returns -EINVAL when:
> 
> 1) the specified vCPU ID is out of range (<0 or >MAX_VIRT_CPUS)
> 2) the specified vCPU ID doesn't match the running vCPU.
> 
> It seems that there is a confusion between the logical vCPU ID and the
> local APIC physical ID.
> [...]
> (XEN) Domain 1 (vcpu#16) VCPUOP_set_singleshot_timer specified vcpuid 1
> [...]
> APIC: CPU 1 has ACPI ID 16

Thanks Matt!  Looks like we need to pass our acpi_id to the Xen hypercall
instead of our cpuid.

Roger, changing the line
	int cpu = PCPU_GET(cpuid);
to
	int cpu = PCPU_GET(acpi_id);
in xentimer_et_start and xentimer_et_stop fixes this panic and gets me
slightly further; the following lines are now added to the console output
prior to the system appearing to hang:
> ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 1 vector 48
> ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 2 vector 48
> ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 3 vector 48
> ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 4 vector 48
> ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 5 vector 48
> ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 6 vector 48
> ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 7 vector 48
> TSC timecounter discards lower 1 bit(s)
> Timecounter "TSC-low" frequency 1300024860 Hz quality -100
> WARNING: WITNESS option enabled, expect reduced performance.

On a cc2.8xlarge EC2 instance, the lines which come after this are
> GEOM: new disk xbd1
> GEOM: new disk xbd2
> GEOM: new disk xbd3
> GEOM: new disk xbd4
> Trying to mount root from ufs:/dev/ad0a [rw]...
> start_init: trying /sbin/init
and then the userland boot process; have you made any bug fixes after
your pvhvm_v7 which would explain why tasting disks was hanging?

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 28/05/13 23:33, Colin Percival wrote:
> On 05/28/13 12:18, Matt Wilson wrote:
>> VCPUOP_set_singleshot_timer returns -EINVAL when:
>>
>> 1) the specified vCPU ID is out of range (<0 or >MAX_VIRT_CPUS)
>> 2) the specified vCPU ID doesn't match the running vCPU.
>>
>> It seems that there is a confusion between the logical vCPU ID and the
>> local APIC physical ID.
>> [...]
>> (XEN) Domain 1 (vcpu#16) VCPUOP_set_singleshot_timer specified vcpuid 1
>> [...]
>> APIC: CPU 1 has ACPI ID 16
> 
> Thanks Matt!  Looks like we need to pass our acpi_id to the Xen hypercall
> instead of our cpuid.
> 
> Roger, changing the line
> 	int cpu = PCPU_GET(cpuid);
> to
> 	int cpu = PCPU_GET(acpi_id);
> in xentimer_et_start and xentimer_et_stop fixes this panic and gets me
> slightly further; the following lines are now added to the console output
> prior to the system appearing to hang:
>> ioapic0: routing intpin 1 (ISA IRQ 1) to lapic 1 vector 48
>> ioapic0: routing intpin 4 (ISA IRQ 4) to lapic 2 vector 48
>> ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 3 vector 48
>> ioapic0: routing intpin 12 (ISA IRQ 12) to lapic 4 vector 48
>> ioapic0: routing intpin 14 (ISA IRQ 14) to lapic 5 vector 48
>> ioapic0: routing intpin 15 (ISA IRQ 15) to lapic 6 vector 48
>> ioapic0: routing intpin 28 (PCI IRQ 28) to lapic 7 vector 48
>> TSC timecounter discards lower 1 bit(s)
>> Timecounter "TSC-low" frequency 1300024860 Hz quality -100
>> WARNING: WITNESS option enabled, expect reduced performance.

Hello,

Thanks Matt and Colin for the testing and help! I've pushed yet another
version, now it's branch pvhvm_v12, which I *think* should solve the
issues with cpuid != acpi_id:

http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12

Since I'm not able to reproduce the cpuid != acpi_id case, could you
give it a try and report the results?

> On a cc2.8xlarge EC2 instance, the lines which come after this are
>> GEOM: new disk xbd1
>> GEOM: new disk xbd2
>> GEOM: new disk xbd3
>> GEOM: new disk xbd4
>> Trying to mount root from ufs:/dev/ad0a [rw]...
>> start_init: trying /sbin/init
> and then the userland boot process; have you made any bug fixes after
> your pvhvm_v7 which would explain why tasting disks was hanging?

I'm not sure I follow, did you found a regression from previous
branches? i.e. it used to work with branch pvhvm_v6 and not pvhvm_v7?

Re: FreeBSD PVHVM call for testing

[Matt Wilson]

On Wed, May 29, 2013 at 07:03:40PM +0200, Roger Pau Monné wrote:
> 
> Hello,
> 
> Thanks Matt and Colin for the testing and help! I've pushed yet another
> version, now it's branch pvhvm_v12, which I *think* should solve the
> issues with cpuid != acpi_id:
> 
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12
> 
> Since I'm not able to reproduce the cpuid != acpi_id case, could you
> give it a try and report the results?

Colin, can you build an AMI with this new kernel?
 
[...]

> On 28/05/13 23:33, Colin Percival wrote:
> > On a cc2.8xlarge EC2 instance, the lines which come after this are
> >> GEOM: new disk xbd1
> >> GEOM: new disk xbd2
> >> GEOM: new disk xbd3
> >> GEOM: new disk xbd4
> >> Trying to mount root from ufs:/dev/ad0a [rw]...
> >> start_init: trying /sbin/init
> > and then the userland boot process; have you made any bug fixes after
> > your pvhvm_v7 which would explain why tasting disks was hanging?
> 
> I'm not sure I follow, did you found a regression from previous
> branches? i.e. it used to work with branch pvhvm_v6 and not pvhvm_v7?

Colin was saying that his local change only moved the boot process a
bit farther for cr1.8xlarge. Perhaps some of the other changes you
made in the latest pvhvm_v12 branch will get the VM all the way up.

--msw

Re: [Xen-users] FreeBSD PVHVM call for testing

[Ian Campbell]

(dropping the BSD lists & xen-users, adding Konrad)

On Wed, 2013-05-29 at 19:03 +0200, Roger Pau Monné wrote:
> Thanks Matt and Colin for the testing and help! I've pushed yet another
> version, now it's branch pvhvm_v12, which I *think* should solve the
> issues with cpuid != acpi_id:
> 
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12
> 
> Since I'm not able to reproduce the cpuid != acpi_id case, could you
> give it a try and report the results?

Konrad,

Might this same problem be related to the issue you saw doing PVHVM VCPU
hotplug which you mentioned the other day?

In general for HVM I suppose there isn't a strict relationship between
the CPU number the kernel chooses for a CPU (which is somewhat
arbitrarily up to the kernel) and the hypervisors VCPU number (which is
exposed via the virtual APIC ID).

Ian.

Re: [Xen-users] FreeBSD PVHVM call for testing

[Konrad Rzeszutek Wilk]

On Wed, May 29, 2013 at 06:25:56PM +0100, Ian Campbell wrote:
> (dropping the BSD lists & xen-users, adding Konrad)
> 
> On Wed, 2013-05-29 at 19:03 +0200, Roger Pau Monné wrote:
> > Thanks Matt and Colin for the testing and help! I've pushed yet another
> > version, now it's branch pvhvm_v12, which I *think* should solve the
> > issues with cpuid != acpi_id:
> > 
> > http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12
> > 
> > Since I'm not able to reproduce the cpuid != acpi_id case, could you
> > give it a try and report the results?
> 
> Konrad,
> 
> Might this same problem be related to the issue you saw doing PVHVM VCPU
> hotplug which you mentioned the other day?

This was http://article.gmane.org/gmane.comp.emulators.xen.devel/159105

> 
> In general for HVM I suppose there isn't a strict relationship between
> the CPU number the kernel chooses for a CPU (which is somewhat
> arbitrarily up to the kernel) and the hypervisors VCPU number (which is
> exposed via the virtual APIC ID).

Right. To troubleshoot this I added some instrumentation to make sure
that the xen_vcpuop_set_mode function was using the right vCPU number.

Initially it got it from smp_processor_id(), which gets it from the
APIC ID (at least that is what it looks like). But perhaps it was not.
I can't seem to reproduce the issue anymore :-(

Re: [Xen-users] FreeBSD PVHVM call for testing

[Ian Campbell]

On Wed, 2013-05-29 at 14:24 -0400, Konrad Rzeszutek Wilk wrote:
> On Wed, May 29, 2013 at 06:25:56PM +0100, Ian Campbell wrote:
> > (dropping the BSD lists & xen-users, adding Konrad)
> > 
> > On Wed, 2013-05-29 at 19:03 +0200, Roger Pau Monné wrote:
> > > Thanks Matt and Colin for the testing and help! I've pushed yet another
> > > version, now it's branch pvhvm_v12, which I *think* should solve the
> > > issues with cpuid != acpi_id:
> > > 
> > > http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12
> > > 
> > > Since I'm not able to reproduce the cpuid != acpi_id case, could you
> > > give it a try and report the results?
> > 
> > Konrad,
> > 
> > Might this same problem be related to the issue you saw doing PVHVM VCPU
> > hotplug which you mentioned the other day?
> 
> This was http://article.gmane.org/gmane.comp.emulators.xen.devel/159105
> 
> > 
> > In general for HVM I suppose there isn't a strict relationship between
> > the CPU number the kernel chooses for a CPU (which is somewhat
> > arbitrarily up to the kernel) and the hypervisors VCPU number (which is
> > exposed via the virtual APIC ID).
> 
> Right. To troubleshoot this I added some instrumentation to make sure
> that the xen_vcpuop_set_mode function was using the right vCPU number.
> 
> Initially it got it from smp_processor_id(), which gets it from the
> APIC ID (at least that is what it looks like). But perhaps it was not.

Is smp_processor_id() always from APIC ID? For x86 I see:
        raw_smp_processor_id()
        stack_smp_processor_id()
        safe_smp_processor_id()
        smp_processor_id()
        logical_smp_processor_id()
        hard_smp_processor_id()
also things like cpu_physical_id(). I'd have to assume they aren't all
equivalent ;-)

> I can't seem to reproduce the issue anymore :-(

Well, that sounds like a good thing to me ;-)

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] FreeBSD PVHVM call for testing

[Matt Wilson]

On Wed, May 29, 2013 at 06:25:56PM +0100, Ian Campbell wrote:
> On Wed, 2013-05-29 at 19:03 +0200, Roger Pau Monné wrote:
> > 
> > Since I'm not able to reproduce the cpuid != acpi_id case, could you
> > give it a try and report the results?
> 
> Konrad,
> 
> Might this same problem be related to the issue you saw doing PVHVM VCPU
> hotplug which you mentioned the other day?
>
> In general for HVM I suppose there isn't a strict relationship between
> the CPU number the kernel chooses for a CPU (which is somewhat
> arbitrarily up to the kernel) and the hypervisors VCPU number (which is
> exposed via the virtual APIC ID).

The CPU enumeration should follow the ACPI ID order in MADT (or
MP-table), right?  The local APIC ID (returned by cpuid
EAX=0000_0001h => EBX bits 31:24) shouldn't be used for
enumeration. Under Xen, the LAPIC ID is hardcoded to vCPU idx * 2:

void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
[...]
     case 0x1:
         /* Fix up VLAPIC details. */
         *ebx &= 0x00FFFFFFu;
         *ebx |= (v->vcpu_id * 2) << 24;
[...]
     case 0xb:
         /* Fix the x2APIC identifier. */
         *edx = v->vcpu_id * 2;
         break;

This isn't the way it works on some of our EC2 instances.  On our high
performance instance types, we provide x2APIC IDs that distinguish
threads, cores and sockets to provide the guest OS with CPU topology
information. The OS and applications can use CPUID EAX=0000_000Bh,
ECX=level (HT=0, Core=1, Socket=2) => EAX bits 4:0 to determine the
topology.

--msw

Re: [Xen-users] FreeBSD PVHVM call for testing

[Ian Campbell]

On Wed, 2013-05-29 at 15:10 -0700, Matt Wilson wrote:
> On Wed, May 29, 2013 at 06:25:56PM +0100, Ian Campbell wrote:
> > On Wed, 2013-05-29 at 19:03 +0200, Roger Pau Monné wrote:
> > > 
> > > Since I'm not able to reproduce the cpuid != acpi_id case, could you
> > > give it a try and report the results?
> > 
> > Konrad,
> > 
> > Might this same problem be related to the issue you saw doing PVHVM VCPU
> > hotplug which you mentioned the other day?
> >
> > In general for HVM I suppose there isn't a strict relationship between
> > the CPU number the kernel chooses for a CPU (which is somewhat
> > arbitrarily up to the kernel) and the hypervisors VCPU number (which is
> > exposed via the virtual APIC ID).
> 
> The CPU enumeration should follow the ACPI ID order in MADT (or
> MP-table), right?

You mean the guest's enumeration? I'd have thought that was entirely up
to the guest. Using the APIC ID might be a reasonable implementation but
there are surely others (e.g. collapsing sparse LAPIC IDs into
contiguous cpu ids, using the top bits for NUMA node, etc etc).

>   The local APIC ID (returned by cpuid
> EAX=0000_0001h => EBX bits 31:24) shouldn't be used for
> enumeration. Under Xen, the LAPIC ID is hardcoded to vCPU idx * 2:
> 
> void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
> [...]
>      case 0x1:
>          /* Fix up VLAPIC details. */
>          *ebx &= 0x00FFFFFFu;
>          *ebx |= (v->vcpu_id * 2) << 24;
> [...]
>      case 0xb:
>          /* Fix the x2APIC identifier. */
>          *edx = v->vcpu_id * 2;
>          break;
> 
> This isn't the way it works on some of our EC2 instances.  On our high
> performance instance types, we provide x2APIC IDs that distinguish
> threads, cores and sockets to provide the guest OS with CPU topology
> information. The OS and applications can use CPUID EAX=0000_000Bh,
> ECX=level (HT=0, Core=1, Socket=2) => EAX bits 4:0 to determine the
> topology.

Hrm, that does beg the question of how an HVM guest is supposed to
determine what Xen's idea of the VCPU number is for a given CPU.

I can't see anything which reverses that * 2 on the vcpu_op path and I
don't suppose you implemented the inverse of your scheme.

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-users] FreeBSD PVHVM call for testing

[Matt Wilson]

On Thu, May 30, 2013 at 10:31:43AM +0100, Ian Campbell wrote:
> On Wed, 2013-05-29 at 15:10 -0700, Matt Wilson wrote:
> > On Wed, May 29, 2013 at 06:25:56PM +0100, Ian Campbell wrote:
> > > On Wed, 2013-05-29 at 19:03 +0200, Roger Pau Monné wrote:
> > > > 
> > > > Since I'm not able to reproduce the cpuid != acpi_id case, could you
> > > > give it a try and report the results?
> > > 
> > > Konrad,
> > > 
> > > Might this same problem be related to the issue you saw doing PVHVM VCPU
> > > hotplug which you mentioned the other day?
> > >
> > > In general for HVM I suppose there isn't a strict relationship between
> > > the CPU number the kernel chooses for a CPU (which is somewhat
> > > arbitrarily up to the kernel) and the hypervisors VCPU number (which is
> > > exposed via the virtual APIC ID).
> > 
> > The CPU enumeration should follow the ACPI ID order in MADT (or
> > MP-table), right?

> You mean the guest's enumeration?

Right.

> I'd have thought that was entirely up to the guest. Using the APIC
> ID might be a reasonable implementation but there are surely others
> (e.g. collapsing sparse LAPIC IDs into contiguous cpu ids, using the
> top bits for NUMA node, etc etc).

On bare metal x86 Linux, the kernel enumerates CPUs based on an order
defined by the BIOS. Typically this means that all the cores are
enumerated first, followed by logical processors (HT/SMT). For Linux,
maxcpus=N/2 should disable HT on systems that enumerate processors in
the recommended order. Some history:
  https://bugzilla.kernel.org/show_bug.cgi?id=2317

> >   The local APIC ID (returned by cpuid
> > EAX=0000_0001h => EBX bits 31:24) shouldn't be used for
> > enumeration. Under Xen, the LAPIC ID is hardcoded to vCPU idx * 2:
> > 
> > void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
> > [...]
> >      case 0x1:
> >          /* Fix up VLAPIC details. */
> >          *ebx &= 0x00FFFFFFu;
> >          *ebx |= (v->vcpu_id * 2) << 24;
> > [...]
> >      case 0xb:
> >          /* Fix the x2APIC identifier. */
> >          *edx = v->vcpu_id * 2;
> >          break;
> > 
> > This isn't the way it works on some of our EC2 instances.  On our high
> > performance instance types, we provide x2APIC IDs that distinguish
> > threads, cores and sockets to provide the guest OS with CPU topology
> > information. The OS and applications can use CPUID EAX=0000_000Bh,
> > ECX=level (HT=0, Core=1, Socket=2) => EAX bits 4:0 to determine the
> > topology.
> 
> Hrm, that does beg the question of how an HVM guest is supposed to
> determine what Xen's idea of the VCPU number is for a given CPU.

The virtual BIOS provides both ACPI tables and a legacy MP-table that
gives the LAPIC id mapping. The guest could infer the Xen vCPU ID from
a processor's position in these tables. Or we could add a VCPUOP that
an enlightened guest could use to get the information more directly.

One question: why does a hypercall take a parameter that only has one
valid value? That value can be determined by looking at the current
running vCPU.

> I can't see anything which reverses that * 2 on the vcpu_op path and I
> don't suppose you implemented the inverse of your scheme.

The *2 is just for assigning the LAPIC ID, and I'm pretty sure that
Linux is assigning processor IDs sequentially at ACPI parse time.

--msw

HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Ian Campbell]

On Thu, 2013-05-30 at 10:16 -0700, Matt Wilson wrote:
> On Thu, May 30, 2013 at 10:31:43AM +0100, Ian Campbell wrote:
> > On Wed, 2013-05-29 at 15:10 -0700, Matt Wilson wrote:
> > > On Wed, May 29, 2013 at 06:25:56PM +0100, Ian Campbell wrote:
> > > > On Wed, 2013-05-29 at 19:03 +0200, Roger Pau Monné wrote:
> > > > > 
> > > > > Since I'm not able to reproduce the cpuid != acpi_id case, could you
> > > > > give it a try and report the results?
> > > > 
> > > > Konrad,
> > > > 
> > > > Might this same problem be related to the issue you saw doing PVHVM VCPU
> > > > hotplug which you mentioned the other day?
> > > >
> > > > In general for HVM I suppose there isn't a strict relationship between
> > > > the CPU number the kernel chooses for a CPU (which is somewhat
> > > > arbitrarily up to the kernel) and the hypervisors VCPU number (which is
> > > > exposed via the virtual APIC ID).
> > > 
> > > The CPU enumeration should follow the ACPI ID order in MADT (or
> > > MP-table), right?
> 
> > You mean the guest's enumeration?
> 
> Right.
> 
> > I'd have thought that was entirely up to the guest. Using the APIC
> > ID might be a reasonable implementation but there are surely others
> > (e.g. collapsing sparse LAPIC IDs into contiguous cpu ids, using the
> > top bits for NUMA node, etc etc).
> 
> On bare metal x86 Linux, the kernel enumerates CPUs based on an order
> defined by the BIOS.
>  Typically this means that all the cores are
> enumerated first, followed by logical processors (HT/SMT). For Linux,
> maxcpus=N/2 should disable HT on systems that enumerate processors in
> the recommended order. Some history:
>   https://bugzilla.kernel.org/show_bug.cgi?id=2317

How the guest chooses to enumerate the CPUs is not terribly relevant so
long as the Xen specific code for that OS knows how to invert that
mapping to get at the underlying ABI which determines Xen's VCPUID for a
CPU.

I think I was wrong to focus on the guest enumeration scheme before,
what actually matters is where in our ABI we expose the VCPUID, which
isn't at all clear to me.

> > >   The local APIC ID (returned by cpuid
> > > EAX=0000_0001h => EBX bits 31:24) shouldn't be used for
> > > enumeration. Under Xen, the LAPIC ID is hardcoded to vCPU idx * 2:
> > > 
> > > void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
> > > [...]
> > >      case 0x1:
> > >          /* Fix up VLAPIC details. */
> > >          *ebx &= 0x00FFFFFFu;
> > >          *ebx |= (v->vcpu_id * 2) << 24;
> > > [...]
> > >      case 0xb:
> > >          /* Fix the x2APIC identifier. */
> > >          *edx = v->vcpu_id * 2;
> > >          break;
> > > 
> > > This isn't the way it works on some of our EC2 instances.  On our high
> > > performance instance types, we provide x2APIC IDs that distinguish
> > > threads, cores and sockets to provide the guest OS with CPU topology
> > > information. The OS and applications can use CPUID EAX=0000_000Bh,
> > > ECX=level (HT=0, Core=1, Socket=2) => EAX bits 4:0 to determine the
> > > topology.
> > 
> > Hrm, that does beg the question of how an HVM guest is supposed to
> > determine what Xen's idea of the VCPU number is for a given CPU.
> 
> The virtual BIOS provides both ACPI tables and a legacy MP-table that
> gives the LAPIC id mapping. The guest could infer the Xen vCPU ID from
> a processor's position in these tables.

Do we consider the ordering given in any of those tables to be an HVM
guest ABI? What about the lapic_id == 2*vcpuid -- is that multiplication
factor part of the ABI (i.e. is the guest expected to pass lapic_id/2 to
vcpuop)?

>  Or we could add a VCPUOP that an enlightened guest could use to get
> the information more directly.

I'm hoping that there is some existing interface which I simply don't
know about, but yes this could be the answer if such a thing doesn't
exist.

> One question: why does a hypercall take a parameter that only has one
> valid value? That value can be determined by looking at the current
> running vCPU.

The generic prototype is:
        vcpu_op(int cmd, int vcpuid, void *extra_args)
Some cmds can act on any vcpuid and others can only act on the current
vcpu. In an ideal world we would have had VCPUID_SELF or something but
its a bit late for that.

> > I can't see anything which reverses that * 2 on the vcpu_op path and I
> > don't suppose you implemented the inverse of your scheme.
> 
> The *2 is just for assigning the LAPIC ID, and I'm pretty sure that
> Linux is assigning processor IDs sequentially at ACPI parse time.

That probably doesn't matter, what matters is the Xen specific parts of
the kernel's ability to reverse that assignment to get at the underlying
APIC ID, assuming that is actually an ABI from which we can infer the
VCPU ID...

Ian


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Matt Wilson]

On Fri, May 31, 2013 at 09:21:50AM +0100, Ian Campbell wrote:
> On Thu, 2013-05-30 at 10:16 -0700, Matt Wilson wrote:
> > 
> > On bare metal x86 Linux, the kernel enumerates CPUs based on an order
> > defined by the BIOS.
> >  Typically this means that all the cores are
> > enumerated first, followed by logical processors (HT/SMT). For Linux,
> > maxcpus=N/2 should disable HT on systems that enumerate processors in
> > the recommended order. Some history:
> >   https://bugzilla.kernel.org/show_bug.cgi?id=2317
> 
> How the guest chooses to enumerate the CPUs is not terribly relevant so
> long as the Xen specific code for that OS knows how to invert that
> mapping to get at the underlying ABI which determines Xen's VCPUID for a
> CPU.

Indeed.

> I think I was wrong to focus on the guest enumeration scheme before,
> what actually matters is where in our ABI we expose the VCPUID, which
> isn't at all clear to me.

Agreed.

> > The virtual BIOS provides both ACPI tables and a legacy MP-table that
> > gives the LAPIC id mapping. The guest could infer the Xen vCPU ID from
> > a processor's position in these tables.
> 
> Do we consider the ordering given in any of those tables to be an HVM
> guest ABI? What about the lapic_id == 2*vcpuid -- is that multiplication
> factor part of the ABI (i.e. is the guest expected to pass lapic_id/2 to
> vcpuop)?

I strongly prefer the order in the BIOS tables, *not* the
lapic_id = 2*vcpuid formula. Once I've done some libxl work, I'll be
proposing a patch that makes the LAPIC / x2APIC IDs configurable,
and that will break this assumption.

> >  Or we could add a VCPUOP that an enlightened guest could use to get
> > the information more directly.
> 
> I'm hoping that there is some existing interface which I simply don't
> know about, but yes this could be the answer if such a thing doesn't
> exist.

I don't know of one that provide the information explicitly. It might
be easiest to provide this as a hypervisor CPUID leaf so it can be
used in early boot.

> > One question: why does a hypercall take a parameter that only has one
> > valid value? That value can be determined by looking at the current
> > running vCPU.
> 
> The generic prototype is:
>         vcpu_op(int cmd, int vcpuid, void *extra_args)
> Some cmds can act on any vcpuid and others can only act on the current
> vcpu. In an ideal world we would have had VCPUID_SELF or something but
> its a bit late for that.

Yea, that makes sense.

> > The *2 is just for assigning the LAPIC ID, and I'm pretty sure that
> > Linux is assigning processor IDs sequentially at ACPI parse time.
> 
> That probably doesn't matter, what matters is the Xen specific parts of
> the kernel's ability to reverse that assignment to get at the underlying
> APIC ID, assuming that is actually an ABI from which we can infer the
> VCPU ID...

Indeed. This seems to be loosely defined so far, and easy to get wrong
as happened with this FreeBSD work.

Konrad, Keir - any thoughts here?

--msw

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Konrad Rzeszutek Wilk]

On Fri, May 31, 2013 at 11:53:22AM -0700, Matt Wilson wrote:
> On Fri, May 31, 2013 at 09:21:50AM +0100, Ian Campbell wrote:
> > On Thu, 2013-05-30 at 10:16 -0700, Matt Wilson wrote:
> > > 
> > > On bare metal x86 Linux, the kernel enumerates CPUs based on an order
> > > defined by the BIOS.
> > >  Typically this means that all the cores are
> > > enumerated first, followed by logical processors (HT/SMT). For Linux,
> > > maxcpus=N/2 should disable HT on systems that enumerate processors in
> > > the recommended order. Some history:
> > >   https://bugzilla.kernel.org/show_bug.cgi?id=2317
> > 
> > How the guest chooses to enumerate the CPUs is not terribly relevant so
> > long as the Xen specific code for that OS knows how to invert that
> > mapping to get at the underlying ABI which determines Xen's VCPUID for a
> > CPU.
> 
> Indeed.
> 
> > I think I was wrong to focus on the guest enumeration scheme before,
> > what actually matters is where in our ABI we expose the VCPUID, which
> > isn't at all clear to me.
> 
> Agreed.
> 
> > > The virtual BIOS provides both ACPI tables and a legacy MP-table that
> > > gives the LAPIC id mapping. The guest could infer the Xen vCPU ID from
> > > a processor's position in these tables.
> > 
> > Do we consider the ordering given in any of those tables to be an HVM
> > guest ABI? What about the lapic_id == 2*vcpuid -- is that multiplication
> > factor part of the ABI (i.e. is the guest expected to pass lapic_id/2 to
> > vcpuop)?
> 
> I strongly prefer the order in the BIOS tables, *not* the
> lapic_id = 2*vcpuid formula. Once I've done some libxl work, I'll be
> proposing a patch that makes the LAPIC / x2APIC IDs configurable,
> and that will break this assumption.
> 
> > >  Or we could add a VCPUOP that an enlightened guest could use to get
> > > the information more directly.
> > 
> > I'm hoping that there is some existing interface which I simply don't
> > know about, but yes this could be the answer if such a thing doesn't
> > exist.
> 
> I don't know of one that provide the information explicitly. It might
> be easiest to provide this as a hypervisor CPUID leaf so it can be
> used in early boot.
> 
> > > One question: why does a hypercall take a parameter that only has one
> > > valid value? That value can be determined by looking at the current
> > > running vCPU.
> > 
> > The generic prototype is:
> >         vcpu_op(int cmd, int vcpuid, void *extra_args)
> > Some cmds can act on any vcpuid and others can only act on the current
> > vcpu. In an ideal world we would have had VCPUID_SELF or something but
> > its a bit late for that.
> 
> Yea, that makes sense.
> 
> > > The *2 is just for assigning the LAPIC ID, and I'm pretty sure that
> > > Linux is assigning processor IDs sequentially at ACPI parse time.
> > 
> > That probably doesn't matter, what matters is the Xen specific parts of
> > the kernel's ability to reverse that assignment to get at the underlying
> > APIC ID, assuming that is actually an ABI from which we can infer the
> > VCPU ID...
> 
> Indeed. This seems to be loosely defined so far, and easy to get wrong
> as happened with this FreeBSD work.
> 
> Konrad, Keir - any thoughts here?

I am a bit confused by 'I strongly prefer the order in the BIOS tables'.
The way I understand it - Linux setup up the vCPUs based on the LAPIC
which are created by the hvmloader. There are no hypercalls or any
lapic_id =2*vcpuid formule in the Linux kernel. I presume what you meant
by the lapic_id = 2 * vcpuid is more of this:

144     for ( i = 0; i < nr_processor_objects; i++ )                                
145     {                                                                           
146         memset(lapic, 0, sizeof(*lapic));                                       
147         lapic->type    = ACPI_PROCESSOR_LOCAL_APIC;                             
148         lapic->length  = sizeof(*lapic);                                        
149         /* Processor ID must match processor-object IDs in the DSDT. */         
150         lapic->acpi_processor_id = i;                                           
151         lapic->apic_id = LAPIC_ID(i);                        

Which sets this up.

So .. assuming this was thought out, why are we starting on vCPUs
that don't match to this? That seems like a bug? (Note, this is 
with maxvcpus=32, vcpus=1 and the starting of a VCPU1 actually
ended up starting at VCPU4?!).

I think all of this can be sorted out if the hvmloader sets the
LAPIC CPU == VCPU ID properly. So perhaps a better question is - why
is it not setup properly nowadays? If the formal is baked in for
the PVHVM guests, somewhere the formula is not being evaluated properly?

The new hypercall to figure this out could be used, but that wouldn't
explain why we are failing to start on the correct VCPU?

> 
> --msw

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Matt Wilson]

On Mon, Jun 03, 2013 at 10:44:43AM -0400, Konrad Rzeszutek Wilk wrote:
> On Fri, May 31, 2013 at 11:53:22AM -0700, Matt Wilson wrote:
> > On Fri, May 31, 2013 at 09:21:50AM +0100, Ian Campbell wrote:
> > > On Thu, 2013-05-30 at 10:16 -0700, Matt Wilson wrote:
> > > > 
> > > > On bare metal x86 Linux, the kernel enumerates CPUs based on an order
> > > > defined by the BIOS.
> > > >  Typically this means that all the cores are
> > > > enumerated first, followed by logical processors (HT/SMT). For Linux,
> > > > maxcpus=N/2 should disable HT on systems that enumerate processors in
> > > > the recommended order. Some history:
> > > >   https://bugzilla.kernel.org/show_bug.cgi?id=2317
> > > 
> > > How the guest chooses to enumerate the CPUs is not terribly relevant so
> > > long as the Xen specific code for that OS knows how to invert that
> > > mapping to get at the underlying ABI which determines Xen's VCPUID for a
> > > CPU.
> > 
> > Indeed.
> > 
> > > I think I was wrong to focus on the guest enumeration scheme before,
> > > what actually matters is where in our ABI we expose the VCPUID, which
> > > isn't at all clear to me.
> > 
> > Agreed.
> > 
> > > > The virtual BIOS provides both ACPI tables and a legacy MP-table that
> > > > gives the LAPIC id mapping. The guest could infer the Xen vCPU ID from
> > > > a processor's position in these tables.
> > > 
> > > Do we consider the ordering given in any of those tables to be an HVM
> > > guest ABI? What about the lapic_id == 2*vcpuid -- is that multiplication
> > > factor part of the ABI (i.e. is the guest expected to pass lapic_id/2 to
> > > vcpuop)?
> > 
> > I strongly prefer the order in the BIOS tables, *not* the
> > lapic_id = 2*vcpuid formula. Once I've done some libxl work, I'll be
> > proposing a patch that makes the LAPIC / x2APIC IDs configurable,
> > and that will break this assumption.
> > 
> > > >  Or we could add a VCPUOP that an enlightened guest could use to get
> > > > the information more directly.
> > > 
> > > I'm hoping that there is some existing interface which I simply don't
> > > know about, but yes this could be the answer if such a thing doesn't
> > > exist.
> > 
> > I don't know of one that provide the information explicitly. It might
> > be easiest to provide this as a hypervisor CPUID leaf so it can be
> > used in early boot.
> > 
> > > > One question: why does a hypercall take a parameter that only has one
> > > > valid value? That value can be determined by looking at the current
> > > > running vCPU.
> > > 
> > > The generic prototype is:
> > >         vcpu_op(int cmd, int vcpuid, void *extra_args)
> > > Some cmds can act on any vcpuid and others can only act on the current
> > > vcpu. In an ideal world we would have had VCPUID_SELF or something but
> > > its a bit late for that.
> > 
> > Yea, that makes sense.
> > 
> > > > The *2 is just for assigning the LAPIC ID, and I'm pretty sure that
> > > > Linux is assigning processor IDs sequentially at ACPI parse time.
> > > 
> > > That probably doesn't matter, what matters is the Xen specific parts of
> > > the kernel's ability to reverse that assignment to get at the underlying
> > > APIC ID, assuming that is actually an ABI from which we can infer the
> > > VCPU ID...
> > 
> > Indeed. This seems to be loosely defined so far, and easy to get wrong
> > as happened with this FreeBSD work.
> > 
> > Konrad, Keir - any thoughts here?
> 
> I am a bit confused by 'I strongly prefer the order in the BIOS tables'.
> The way I understand it - Linux setup up the vCPUs based on the LAPIC
> which are created by the hvmloader. There are no hypercalls or any
> lapic_id =2*vcpuid formule in the Linux kernel. I presume what you meant
> by the lapic_id = 2 * vcpuid is more of this:
> 
> 144     for ( i = 0; i < nr_processor_objects; i++ )                                
> 145     {                                                                           
> 146         memset(lapic, 0, sizeof(*lapic));                                       
> 147         lapic->type    = ACPI_PROCESSOR_LOCAL_APIC;                             
> 148         lapic->length  = sizeof(*lapic);                                        
> 149         /* Processor ID must match processor-object IDs in the DSDT. */         
> 150         lapic->acpi_processor_id = i;                                           
> 151         lapic->apic_id = LAPIC_ID(i);                        
> 
> Which sets this up.

Right, all of the LAPIC information is provided to the guest OS via
the MADT. I believe what I'm observing is that Linux and Windows use
the order of entries to enumerate processors in the system.

What we typically see on bare metal Intel systems is something like
this (example system has 16 cores with HT):

All of the "cores"...
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] enabled)
...

Followed by all of the "threads"...
[    0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x03] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x05] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x07] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x09] enabled)

Since Xen hard codes the LAPIC ID (and x2APIC ID) to 0, 2, 4, 6, 8,
etc. (vCPUID * 2), everything looks like a core.

> So .. assuming this was thought out, why are we starting on vCPUs
> that don't match to this? That seems like a bug? (Note, this is 
> with maxvcpus=32, vcpus=1 and the starting of a VCPU1 actually
> ended up starting at VCPU4?!).

I'm lost. What?

> I think all of this can be sorted out if the hvmloader sets the
> LAPIC CPU == VCPU ID properly.

No, that's not the right answer. Or, at least, not completely. Right
now Xen provides the same ID for both the LAPIC and x2APIC. In order
for cpu topology discovery to work, the x2APIC needs to follow a
particular structure. See the Intel whitepaper on processor topology
enumeration:
  http://software.intel.com/sites/default/files/m/d/4/1/d/8/Kuo_CpuTopology_rc1.rh1.final.pdf

> So perhaps a better question is - why is it not setup properly
> nowadays? If the formal is baked in for the PVHVM guests, somewhere
> the formula is not being evaluated properly?

The "LAPIC ID = 2 * vCPUID" formula is not baked into any OS that I
know of, and it shouldn't be. It should all be discovered via
firmware/BIOS tables. The enumeration order in the tables should,
under best practices, match the logical processor ID assignment in the
OS.

> The new hypercall to figure this out could be used, but that wouldn't
> explain why we are failing to start on the correct VCPU?

I didn't follow the jump here. Can you provide an example?

--msw

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Konrad Rzeszutek Wilk]

On Mon, Jun 03, 2013 at 08:57:26AM -0700, Matt Wilson wrote:
> On Mon, Jun 03, 2013 at 10:44:43AM -0400, Konrad Rzeszutek Wilk wrote:
> > On Fri, May 31, 2013 at 11:53:22AM -0700, Matt Wilson wrote:
> > > On Fri, May 31, 2013 at 09:21:50AM +0100, Ian Campbell wrote:
> > > > On Thu, 2013-05-30 at 10:16 -0700, Matt Wilson wrote:
> > > > > 
> > > > > On bare metal x86 Linux, the kernel enumerates CPUs based on an order
> > > > > defined by the BIOS.
> > > > >  Typically this means that all the cores are
> > > > > enumerated first, followed by logical processors (HT/SMT). For Linux,
> > > > > maxcpus=N/2 should disable HT on systems that enumerate processors in
> > > > > the recommended order. Some history:
> > > > >   https://bugzilla.kernel.org/show_bug.cgi?id=2317
> > > > 
> > > > How the guest chooses to enumerate the CPUs is not terribly relevant so
> > > > long as the Xen specific code for that OS knows how to invert that
> > > > mapping to get at the underlying ABI which determines Xen's VCPUID for a
> > > > CPU.
> > > 
> > > Indeed.
> > > 
> > > > I think I was wrong to focus on the guest enumeration scheme before,
> > > > what actually matters is where in our ABI we expose the VCPUID, which
> > > > isn't at all clear to me.
> > > 
> > > Agreed.
> > > 
> > > > > The virtual BIOS provides both ACPI tables and a legacy MP-table that
> > > > > gives the LAPIC id mapping. The guest could infer the Xen vCPU ID from
> > > > > a processor's position in these tables.
> > > > 
> > > > Do we consider the ordering given in any of those tables to be an HVM
> > > > guest ABI? What about the lapic_id == 2*vcpuid -- is that multiplication
> > > > factor part of the ABI (i.e. is the guest expected to pass lapic_id/2 to
> > > > vcpuop)?
> > > 
> > > I strongly prefer the order in the BIOS tables, *not* the
> > > lapic_id = 2*vcpuid formula. Once I've done some libxl work, I'll be
> > > proposing a patch that makes the LAPIC / x2APIC IDs configurable,
> > > and that will break this assumption.
> > > 
> > > > >  Or we could add a VCPUOP that an enlightened guest could use to get
> > > > > the information more directly.
> > > > 
> > > > I'm hoping that there is some existing interface which I simply don't
> > > > know about, but yes this could be the answer if such a thing doesn't
> > > > exist.
> > > 
> > > I don't know of one that provide the information explicitly. It might
> > > be easiest to provide this as a hypervisor CPUID leaf so it can be
> > > used in early boot.
> > > 
> > > > > One question: why does a hypercall take a parameter that only has one
> > > > > valid value? That value can be determined by looking at the current
> > > > > running vCPU.
> > > > 
> > > > The generic prototype is:
> > > >         vcpu_op(int cmd, int vcpuid, void *extra_args)
> > > > Some cmds can act on any vcpuid and others can only act on the current
> > > > vcpu. In an ideal world we would have had VCPUID_SELF or something but
> > > > its a bit late for that.
> > > 
> > > Yea, that makes sense.
> > > 
> > > > > The *2 is just for assigning the LAPIC ID, and I'm pretty sure that
> > > > > Linux is assigning processor IDs sequentially at ACPI parse time.
> > > > 
> > > > That probably doesn't matter, what matters is the Xen specific parts of
> > > > the kernel's ability to reverse that assignment to get at the underlying
> > > > APIC ID, assuming that is actually an ABI from which we can infer the
> > > > VCPU ID...
> > > 
> > > Indeed. This seems to be loosely defined so far, and easy to get wrong
> > > as happened with this FreeBSD work.
> > > 
> > > Konrad, Keir - any thoughts here?
> > 
> > I am a bit confused by 'I strongly prefer the order in the BIOS tables'.
> > The way I understand it - Linux setup up the vCPUs based on the LAPIC
> > which are created by the hvmloader. There are no hypercalls or any
> > lapic_id =2*vcpuid formule in the Linux kernel. I presume what you meant
> > by the lapic_id = 2 * vcpuid is more of this:
> > 
> > 144     for ( i = 0; i < nr_processor_objects; i++ )                                
> > 145     {                                                                           
> > 146         memset(lapic, 0, sizeof(*lapic));                                       
> > 147         lapic->type    = ACPI_PROCESSOR_LOCAL_APIC;                             
> > 148         lapic->length  = sizeof(*lapic);                                        
> > 149         /* Processor ID must match processor-object IDs in the DSDT. */         
> > 150         lapic->acpi_processor_id = i;                                           
> > 151         lapic->apic_id = LAPIC_ID(i);                        
> > 
> > Which sets this up.
> 
> Right, all of the LAPIC information is provided to the guest OS via
> the MADT. I believe what I'm observing is that Linux and Windows use
> the order of entries to enumerate processors in the system.
> 
> What we typically see on bare metal Intel systems is something like
> this (example system has 16 cores with HT):
> 
> All of the "cores"...
> [    0.000000] ACPI: Local APIC address 0xfee00000
> [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] enabled)
> ...
> 
> Followed by all of the "threads"...
> [    0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x01] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x03] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x05] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x07] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x09] enabled)
> 
> Since Xen hard codes the LAPIC ID (and x2APIC ID) to 0, 2, 4, 6, 8,
> etc. (vCPUID * 2), everything looks like a core.

OK.
> 
> > So .. assuming this was thought out, why are we starting on vCPUs
> > that don't match to this? That seems like a bug? (Note, this is 
> > with maxvcpus=32, vcpus=1 and the starting of a VCPU1 actually
> > ended up starting at VCPU4?!).
> 
> I'm lost. What?

http://lists.xen.org/archives/html/xen-devel/2013-05/msg00941.html
> 
> > I think all of this can be sorted out if the hvmloader sets the
> > LAPIC CPU == VCPU ID properly.
> 
> No, that's not the right answer. Or, at least, not completely. Right
> now Xen provides the same ID for both the LAPIC and x2APIC. In order
> for cpu topology discovery to work, the x2APIC needs to follow a
> particular structure. See the Intel whitepaper on processor topology
> enumeration:
>   http://software.intel.com/sites/default/files/m/d/4/1/d/8/Kuo_CpuTopology_rc1.rh1.final.pdf

Nice explanation. However, I was under impression that right now we
don't virtualize the x2APIC registers?
> 
> > So perhaps a better question is - why is it not setup properly
> > nowadays? If the formal is baked in for the PVHVM guests, somewhere
> > the formula is not being evaluated properly?
> 
> The "LAPIC ID = 2 * vCPUID" formula is not baked into any OS that I
> know of, and it shouldn't be. It should all be discovered via
> firmware/BIOS tables. The enumeration order in the tables should,
> under best practices, match the logical processor ID assignment in the
> OS.

OK, good. That is my understanding too.

> 
> > The new hypercall to figure this out could be used, but that wouldn't
> > explain why we are failing to start on the correct VCPU?
> 
> I didn't follow the jump here. Can you provide an example?

http://lists.xen.org/archives/html/xen-devel/2013-05/msg00941.html

> 
> --msw

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Matt Wilson]

On Mon, Jun 03, 2013 at 01:33:51PM -0400, Konrad Rzeszutek Wilk wrote:
> On Mon, Jun 03, 2013 at 08:57:26AM -0700, Matt Wilson wrote:

[...]

> > 
> > Right, all of the LAPIC information is provided to the guest OS via
> > the MADT. I believe what I'm observing is that Linux and Windows use
> > the order of entries to enumerate processors in the system.
> > 
> > What we typically see on bare metal Intel systems is something like
> > this (example system has 16 cores with HT):
> > 
> > All of the "cores"...
> > [    0.000000] ACPI: Local APIC address 0xfee00000
> > [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] enabled)
> > ...
> > 
> > Followed by all of the "threads"...
> > [    0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x01] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x03] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x05] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x07] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x09] enabled)
> > 
> > Since Xen hard codes the LAPIC ID (and x2APIC ID) to 0, 2, 4, 6, 8,
> > etc. (vCPUID * 2), everything looks like a core.
> 
> OK.
>
> > > So .. assuming this was thought out, why are we starting on vCPUs
> > > that don't match to this? That seems like a bug? (Note, this is 
> > > with maxvcpus=32, vcpus=1 and the starting of a VCPU1 actually
> > > ended up starting at VCPU4?!).
> > 
> > I'm lost. What?
> 
> http://lists.xen.org/archives/html/xen-devel/2013-05/msg00941.html

Aah, gotcha.

> > > I think all of this can be sorted out if the hvmloader sets the
> > > LAPIC CPU == VCPU ID properly.
> > 
> > No, that's not the right answer. Or, at least, not completely. Right
> > now Xen provides the same ID for both the LAPIC and x2APIC. In order
> > for cpu topology discovery to work, the x2APIC needs to follow a
> > particular structure. See the Intel whitepaper on processor topology
> > enumeration:
> >   http://software.intel.com/sites/default/files/m/d/4/1/d/8/Kuo_CpuTopology_rc1.rh1.final.pdf
> 
> Nice explanation. However, I was under impression that right now we
> don't virtualize the x2APIC registers?

Hmm?

http://xenbits.xen.org/gitweb/?p=xen.git;a=commitdiff;h=21c4d089ba2a236dcf33a06222070bd6c60c6e3d

> > > So perhaps a better question is - why is it not setup properly
> > > nowadays? If the formal is baked in for the PVHVM guests, somewhere
> > > the formula is not being evaluated properly?
> > 
> > The "LAPIC ID = 2 * vCPUID" formula is not baked into any OS that I
> > know of, and it shouldn't be. It should all be discovered via
> > firmware/BIOS tables. The enumeration order in the tables should,
> > under best practices, match the logical processor ID assignment in the
> > OS.
> 
> OK, good. That is my understanding too.
> 
> > 
> > > The new hypercall to figure this out could be used, but that wouldn't
> > > explain why we are failing to start on the correct VCPU?
> > 
> > I didn't follow the jump here. Can you provide an example?
> 
> http://lists.xen.org/archives/html/xen-devel/2013-05/msg00941.html

OK, got it.

[   84.619508] smpboot: Booting Node 0 Processor 1 APIC 0x8

So it seems like, in this case:

int __cpuinit native_cpu_up(unsigned int cpu)
{
        int apicid = apic->cpu_present_to_apicid(cpu);

apic->cpu_present_to_apicid(1) returned 8 instead of 2.

All of that should have been set up correctly ahead of time by
generic_processor_info() for all possible CPUs. Do you have the full
boot log?

--msw

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Konrad Rzeszutek Wilk]

> > > > The new hypercall to figure this out could be used, but that wouldn't
> > > > explain why we are failing to start on the correct VCPU?
> > > 
> > > I didn't follow the jump here. Can you provide an example?
> > 
> > http://lists.xen.org/archives/html/xen-devel/2013-05/msg00941.html
> 
> OK, got it.
> 
> [   84.619508] smpboot: Booting Node 0 Processor 1 APIC 0x8
> 
> So it seems like, in this case:
> 
> int __cpuinit native_cpu_up(unsigned int cpu)
> {
>         int apicid = apic->cpu_present_to_apicid(cpu);
> 
> apic->cpu_present_to_apicid(1) returned 8 instead of 2.
> 
> All of that should have been set up correctly ahead of time by
> generic_processor_info() for all possible CPUs. Do you have the full
> boot log?
> 

Loading initramf.gz.............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
 ..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
 .......................................................................................................................................................................................................................................................................................................................................................................................................................ready.
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 3.9.0upstream-00022-g49c1bf1-dirty (konrad@build-external.dumpdata.com) (gcc version 4.4.4 20100503 (Red Hat 4.4.4-2) (GCC) ) #6 SMP Tue May 7 15:02:36 EDT 2013
[    0.000000] Command line: initrd=initramf.gz  console=ttyS0,115200 test=net nofb BOOT_IMAGE=vmlinuz 
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009dfff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009e000-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000e0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007f7fffff] usable
[    0.000000] BIOS-e820: [mem 0x00000000fc000000-0x00000000ffffffff] reserved
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.4 present.
[    0.000000] Hypervisor detected: Xen HVM
[    0.000000] Xen version 4.3.
[    0.000000] Netfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated NICs.
[    0.000000] Blkfront and the Xen platform PCI driver have been compiled for this kernel: unplug emulated disks.
[    0.000000] You might have to change the root device
[    0.000000] from /dev/hd[a-d] to /dev/xvd[a-d]
[    0.000000] in your root= kernel command line option
[    0.000000] No AGP bridge found
[    0.000000] e820: last_pfn = 0x7f800 max_arch_pfn = 0x400000000
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] found SMP MP-table at [mem 0x000fbba0-0x000fbbaf] mapped at [ffff8800000fbba0]
[    0.000000] Scanning 1 areas for low memory corruption
[    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[    0.000000] init_memory_mapping: [mem 0x77800000-0x779fffff]
[    0.000000] init_memory_mapping: [mem 0x74000000-0x777fffff]
[    0.000000] init_memory_mapping: [mem 0x00100000-0x73ffffff]
[    0.000000] init_memory_mapping: [mem 0x77a00000-0x7f7fffff]
[    0.000000] RAMDISK: [mem 0x77bec000-0x7f7defff]
[    0.000000] ACPI: RSDP 00000000000ea020 00024 (v02    Xen)
[    0.000000] ACPI: XSDT 00000000fc0150c0 00054 (v01    Xen      HVM 00000000 HVML 00000000)
[    0.000000] ACPI: FACP 00000000fc0149f0 000F4 (v04    Xen      HVM 00000000 HVML 00000000)
[    0.000000] ACPI: DSDT 00000000fc0035e0 1138D (v02    Xen      HVM 00000000 INTL 20090123)
[    0.000000] ACPI: FACS 00000000fc0035a0 00040
[    0.000000] ACPI: APIC 00000000fc014af0 00460 (v02    Xen      HVM 00000000 HVML 00000000)
[    0.000000] ACPI: HPET 00000000fc014fd0 00038 (v01    Xen      HVM 00000000 HVML 00000000)
[    0.000000] ACPI: WAET 00000000fc015010 00028 (v01    Xen      HVM 00000000 HVML 00000000)
[    0.000000] ACPI: SSDT 00000000fc015040 00031 (v02    Xen      HVM 00000000 INTL 20090123)
[    0.000000] ACPI: SSDT 00000000fc015080 00031 (v02    Xen      HVM 00000000 INTL 20090123)
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at [mem 0x0000000000000000-0x000000007f7fffff]
[    0.000000] Initmem setup node 0 [mem 0x00000000-0x7f7fffff]
[    0.000000]   NODE_DATA [mem 0x7f7fb000-0x7f7fffff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
[    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00001000-0x0009dfff]
[    0.000000]   node   0: [mem 0x00100000-0x7f7fffff]
[    0.000000] ACPI: PM-Timer IO Port: 0xb008
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x0c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x10] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x12] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x14] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x16] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x18] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x1a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x1c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x1e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x20] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x22] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x24] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x26] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x28] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x15] lapic_id[0x2a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x16] lapic_id[0x2c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x17] lapic_id[0x2e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x18] lapic_id[0x30] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x19] lapic_id[0x32] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x34] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x36] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x38] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x3a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x3c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x3e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x20] lapic_id[0x40] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x21] lapic_id[0x42] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x22] lapic_id[0x44] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x23] lapic_id[0x46] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x24] lapic_id[0x48] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x25] lapic_id[0x4a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x26] lapic_id[0x4c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x27] lapic_id[0x4e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x28] lapic_id[0x50] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x29] lapic_id[0x52] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2a] lapic_id[0x54] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2b] lapic_id[0x56] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2c] lapic_id[0x58] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2d] lapic_id[0x5a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2e] lapic_id[0x5c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2f] lapic_id[0x5e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x30] lapic_id[0x60] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x31] lapic_id[0x62] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x32] lapic_id[0x64] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x33] lapic_id[0x66] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x34] lapic_id[0x68] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x35] lapic_id[0x6a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x36] lapic_id[0x6c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x37] lapic_id[0x6e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x38] lapic_id[0x70] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x39] lapic_id[0x72] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3a] lapic_id[0x74] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3b] lapic_id[0x76] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3c] lapic_id[0x78] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3d] lapic_id[0x7a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3e] lapic_id[0x7c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3f] lapic_id[0x7e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x40] lapic_id[0x80] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x41] lapic_id[0x82] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x42] lapic_id[0x84] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x43] lapic_id[0x86] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x44] lapic_id[0x88] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x45] lapic_id[0x8a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x46] lapic_id[0x8c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x47] lapic_id[0x8e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x48] lapic_id[0x90] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x49] lapic_id[0x92] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4a] lapic_id[0x94] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4b] lapic_id[0x96] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4c] lapic_id[0x98] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4d] lapic_id[0x9a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4e] lapic_id[0x9c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4f] lapic_id[0x9e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x50] lapic_id[0xa0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x51] lapic_id[0xa2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x52] lapic_id[0xa4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x53] lapic_id[0xa6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x54] lapic_id[0xa8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x55] lapic_id[0xaa] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x56] lapic_id[0xac] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x57] lapic_id[0xae] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x58] lapic_id[0xb0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x59] lapic_id[0xb2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5a] lapic_id[0xb4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5b] lapic_id[0xb6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5c] lapic_id[0xb8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5d] lapic_id[0xba] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5e] lapic_id[0xbc] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5f] lapic_id[0xbe] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x60] lapic_id[0xc0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x61] lapic_id[0xc2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x62] lapic_id[0xc4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x63] lapic_id[0xc6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x64] lapic_id[0xc8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x65] lapic_id[0xca] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x66] lapic_id[0xcc] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x67] lapic_id[0xce] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x68] lapic_id[0xd0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x69] lapic_id[0xd2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6a] lapic_id[0xd4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6b] lapic_id[0xd6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6c] lapic_id[0xd8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6d] lapic_id[0xda] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6e] lapic_id[0xdc] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6f] lapic_id[0xde] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x70] lapic_id[0xe0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x71] lapic_id[0xe2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x72] lapic_id[0xe4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x73] lapic_id[0xe6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x74] lapic_id[0xe8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x75] lapic_id[0xea] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x76] lapic_id[0xec] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x77] lapic_id[0xee] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x78] lapic_id[0xf0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x79] lapic_id[0xf2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7a] lapic_id[0xf4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7b] lapic_id[0xf6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7c] lapic_id[0xf8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7d] lapic_id[0xfa] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7e] lapic_id[0xfc] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7f] lapic_id[0xfe] disabled)
[    0.000000] ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-47
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 low level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 low level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 low level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] smpboot: Allowing 128 CPUs, 127 hotplug CPUs
[    0.000000] PM: Registered nosave memory: 000000000009e000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000
[    0.000000] PM: Registered nosave memory: 00000000000e0000 - 0000000000100000
[    0.000000] e820: [mem 0x7f800000-0xfbffffff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on Xen HVM
[    0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:128 nr_node_ids:1
[    0.000000] PERCPU: Embedded 29 pages/cpu @ffff880074200000 s87808 r8192 d22784 u131072
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 514980
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line: initrd=initramf.gz  console=ttyS0,115200 test=net nofb BOOT_IMAGE=vmlinuz 
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] __ex_table already sorted, skipping sort
[    0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Memory: 1893840k/2088960k available (6809k kernel code, 396k absent, 194724k reserved, 6266k data, 1060k init)
[    0.000000] kmemleak: Kernel memory leak detector disabled
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=128.
[    0.000000] NR_IRQS:33024 nr_irqs:2112 16
[    0.000000] Xen HVM callback vector for event delivery is enabled
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
[    0.000000] ... MAX_LOCKDEP_SUBCLASSES:  8
[    0.000000] ... MAX_LOCK_DEPTH:          48
[    0.000000] ... MAX_LOCKDEP_KEYS:        8191
[    0.000000] ... CLASSHASH_SIZE:          4096
[    0.000000] ... MAX_LOCKDEP_ENTRIES:     16384
[    0.000000] ... MAX_LOCKDEP_CHAINS:      32768
[    0.000000] ... CHAINHASH_SIZE:          16384
[    0.000000]  memory used by lock dependency info: 5855 kB
[    0.000000]  per task-struct memory footprint: 1920 bytes
[    0.000000] kmemleak: Early log buffer exceeded (799), please increase DEBUG_KMEMLEAK_EARLY_LOG_SIZE
[    0.000000] tsc: Detected 3093.074 MHz processor
[    0.006000] Calibrating delay loop (skipped), value calculated using timer frequency.. 6186.14 BogoMIPS (lpj=3093074)
[    0.018004] pid_max: default: 131072 minimum: 1024
[    0.023082] Security Framework initialized
[    0.028004] SELinux:  Initializing.
[    0.032178] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.040332] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.049167] Mount-cache hash table entries: 256
[    0.055547] Initializing cgroup subsys cpuacct
[    0.060010] Initializing cgroup subsys freezer
[    0.066115] CPU: Physical Processor ID: 0
[    0.070004] CPU: Processor Core ID: 0
[    0.075005] mce: CPU supports 2 MCE banks
[    0.079027] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
[    0.079027] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32
[    0.079027] tlb_flushall_shift: 5
[    0.108691] ACPI: Core revision 20130117
[    0.127967] ACPI: All ACPI Tables successfully acquired
[    0.134575] Switched APIC routing to physical flat.
[    0.143211] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0
[    0.160552] smpboot: CPU0: Intel(R) Xeon(R) CPU E31225 @ 3.10GHz (fam: 06, model: 2a, stepping: 07)
[    0.171020] installing Xen timer for CPU 0
[    0.176285] Performance Events: unsupported p6 CPU model 42 no PMU driver, software events only.
[    0.181742] Brought up 1 CPUs
[    0.182008] smpboot: Total of 1 processors activated (6186.14 BogoMIPS)
[    0.183549] NMI watchdog: disabled (cpu0): hardware events not enabled
[    0.187218] kworker/u:0 (15) used greatest stack depth: 5728 bytes left
[    0.188091] RTC time: 16:02:23, date: 05/08/13
[    0.189197] NET: Registered protocol family 16
[    0.190567] kworker/u:0 (26) used greatest stack depth: 5656 bytes left
[    0.191270] ACPI: bus type PCI registered
[    0.192223] dca service started, version 1.12.1
[    0.193510] PCI: Using configuration type 1 for base access
[    0.203121] bio: create slab <bio-0> at 0
[    0.204246] ACPI: Added _OSI(Module Device)
[    0.205070] ACPI: Added _OSI(Processor Device)
[    0.206007] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.207021] ACPI: Added _OSI(Processor Aggregator Device)
[    0.230407] ACPI: Interpreter enabled
[    0.231010] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20130117/hwxface-568)
[    0.234008] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20130117/hwxface-568)
[    0.237030] ACPI: (supports S0 S3 S4 S5)
[    0.238006] ACPI: Using IOAPIC for interrupt routing
[    0.239097] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.304682] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.305233] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[    0.306150] PCI host bridge to bus 0000:00
[    0.307013] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.308009] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
[    0.309009] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
[    0.310008] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
[    0.311007] pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfbffffff]
[    0.321122] * Found PM-Timer Bug on the chipset. Due to workarounds for a bug,
[    0.321122] * this clock source is slow. Consider trying other clock sources
[    0.324801] pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
[    0.334009] acpi PNP0A03:00: ACPI _OSC support notification failed, disabling PCIe ASPM
[    0.335008] acpi PNP0A03:00: Unable to request _OSC control (_OSC support mask: 0x08)
[    0.336488] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11)
[    0.339000] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[    0.341023] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[    0.344029] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11)
[    0.375796] ACPI: Enabled 2 GPEs in block 00 to 0F
[    0.408515] ACPI: No dock devices found.
[    0.409127] xen/balloon: Initialising balloon driver.
[    0.412082] xen-balloon: Initialising balloon driver.
[    0.413350] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
[    0.414015] vgaarb: loaded
[    0.415007] vgaarb: bridge control possible 0000:00:02.0
[    0.416216] ACPI: bus type USB registered
[    0.417132] usbcore: registered new interface driver usbfs
[    0.418087] usbcore: registered new interface driver hub
[    0.419109] usbcore: registered new device driver usb
[    0.420197] pps_core: LinuxPPS API ver. 1 registered
[    0.421013] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.422067] PTP clock support registered
[    0.423147] PCI: Using ACPI for IRQ routing
[    0.425142] NetLabel: Initializing
[    0.426016] NetLabel:  domain hash size = 128
[    0.427006] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.428041] NetLabel:  unlabeled traffic allowed by default
[    0.429261] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[    0.430032] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    0.432343] hpet0: 3 comparators, 64-bit 62.500000 MHz counter
[    0.434050] Switching to clocksource xen
[    0.445570] pnp: PnP ACPI init
[    0.449585] ACPI: bus type PNP registered
[    0.454852] system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved
[    0.463906] system 00:02: [io  0x08a0-0x08a3] has been reserved
[    0.471170] system 00:02: [io  0x0cc0-0x0ccf] has been reserved
[    0.478425] system 00:02: [io  0x04d0-0x04d1] has been reserved
[    0.486674] system 00:0b: [io  0x10c0-0x1141] has been reserved
[    0.494022] system 00:0b: [io  0xb044-0xb047] has been reserved
[    0.529363] pnp: PnP ACPI: found 12 devices
[    0.534578] ACPI: bus type PNP unregistered
[    0.548620] NET: Registered protocol family 2
[    0.554102] TCP established hash table entries: 16384 (order: 6, 262144 bytes)
[    0.562833] TCP bind hash table entries: 16384 (order: 8, 1048576 bytes)
[    0.571383] TCP: Hash tables configured (established 16384 bind 16384)
[    0.579085] TCP: reno registered
[    0.583084] UDP hash table entries: 1024 (order: 5, 163840 bytes)
[    0.590605] UDP-Lite hash table entries: 1024 (order: 5, 163840 bytes)
[    0.599144] NET: Registered protocol family 1
[    0.604570] RPC: Registered named UNIX socket transport module.
[    0.611845] RPC: Registered udp transport module.
[    0.617442] RPC: Registered tcp transport module.
[    0.623361] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.631110] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[    0.638355] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[    0.645352] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[    0.653163] Unpacking initramfs...
[    2.665438] Freeing initrd memory: 126924k freed
[    2.685115] Machine check injector initialized
[    2.691259] microcode: CPU0 sig=0x206a7, pf=0x2, revision=0x25
[    2.698107] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    2.708013] Scanning for low memory corruption every 60 seconds
[    2.715204] audit: initializing netlink socket (disabled)
[    2.721340] type=2000 audit(1368028946.427:1): initialized
[    2.746153] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    2.754076] VFS: Disk quotas dquot_6.5.2
[    2.758740] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    2.766586] NFS: Registering the id_resolver key type
[    2.772412] Key type id_resolver registered
[    2.777297] Key type id_legacy registered
[    2.782015] NTFS driver 2.1.30 [Flags: R/W].
[    2.787057] msgmni has been set to 3946
[    2.792475] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[    2.801055] io scheduler noop registered
[    2.805620] io scheduler deadline registered
[    2.810571] io scheduler cfq registered (default)
[    2.816243] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    2.823155] cirrusfb 0000:00:02.0: Cirrus Logic chipset on PCI bus, RAM (4096 kB) at 0xf0000000
[    2.833820] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    2.842362] ACPI: Power Button [PWRF]
[    2.846802] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
[    2.855305] ACPI: Sleep Button [SLPF]
[    2.896600] GHES: HEST is not enabled!
[    2.901377] ioatdma: Intel(R) QuickData Technology Driver 4.00
[    2.908901] Grant tables using version 1 layout.
[    2.914474] Grant table initialized
[    2.944409] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    2.983474] 00:09: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    2.991905] Non-volatile memory driver v1.3
[    2.996920] Linux agpgart interface v0.103
[    3.002120] [drm] Initialized drm 1.1.0 20060810
[    3.009382] loop: module loaded
[    3.013471] libphy: Fixed MDIO Bus: probed
[    3.018145] tun: Universal TUN/TAP device driver, 1.6
[    3.023930] tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
[    3.031109] ixgbevf: Intel(R) 10 Gigabit PCI Express Virtual Function Network Driver - version 2.7.12-k
[    3.041764] ixgbevf: Copyright (c) 2009 - 2012 Intel Corporation.
[    3.049129] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    3.056864] ehci-pci: EHCI PCI platform driver
[    3.062223] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    3.069486] uhci_hcd: USB Universal Host Controller Interface driver
[    3.077003] usbcore: registered new interface driver usblp
[    3.083477] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
[    3.095464] serio: i8042 KBD port at 0x60,0x64 irq 1
[    3.101303] serio: i8042 AUX port at 0x60,0x64 irq 12
[    3.107627] mousedev: PS/2 mouse device common for all mice
[    3.115733] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
[    3.133702] rtc_cmos 00:04: rtc core: registered rtc_cmos as rtc0
[    3.140779] rtc_cmos 00:04: alarms up to one day, 114 bytes nvram, hpet irqs
[    3.149263] cpuidle: using governor ladder
[    3.154239] cpuidle: using governor menu
[    3.159122] EFI Variables Facility v0.08 2004-May-17
[    3.165190] zram: Created 1 device(s) ...
[    3.170185] Netfilter messages via NETLINK v0.30.
[    3.176241] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    3.183848] ctnetlink v0.93: registering with nfnetlink.
[    3.190689] ip_tables: (C) 2000-2006 Netfilter Core Team
[    3.197472] TCP: cubic registered
[    3.201780] Initializing XFRM netlink socket
[    3.207111] NET: Registered protocol family 10
[    3.213416] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    3.220348] sit: IPv6 over IPv4 tunneling driver
[    3.226247] NET: Registered protocol family 17
[    3.231481] Key type dns_resolver registered
[    3.236849] registered taskstats version 1
[    3.242382] XENBUS: Device with no driver: device/vbd/5632
[    3.248633] XENBUS: Device with no driver: device/vkbd/0
[    3.254808] XENBUS: Device with no driver: device/vif/0
[    3.261081]   Magic number: 13:252:34
[    3.267863] Freeing unused kernel memory: 1060k freed
[    3.273942] Write protecting the kernel read-only data: 12288k
[    3.283168] Freeing unused kernel memory: 1372k freed
[    3.291587] Freeing unused kernel memory: 1492k freed
init started: BusyBox v1.14.3 (2013-05-05 20:35:32 EDT)
[    3.306168] consoletype (1092) used greatest stack depth: 5192 bytes left
Mounting directories  [  OK  ]
[    3.419189] mv (1106) used greatest stack depth: 5024 bytes left
mount: mount point /proc/bus/usb does not exist
[    3.445186] modprobe (1121) used greatest stack depth: 4496 bytes left
mount: mount point /sys/kernel/config does not exist
[    3.467269] input: Xen Virtual Keyboard as /devices/virtual/input/input3
[    3.475315] input: Xen Virtual Pointer as /devices/virtual/input/input4
FATAL: Error inserting xen_fbfront (/lib/modules/3.9.0upstream-00022-g49c1bf1-dirty/kernel/drivers/video/xen-fbfront.ko): No such device
[    3.601797] Initialising Xen virtual ethernet driver.
[    3.710082] vbd vbd-5632: 19 xenbus_dev_probe on device/vbd/5632
[    3.717646] tsc: Refined TSC clocksource calibration: 3092.973 MHz
[    3.721460] vbd vbd-5632: failed to write error node for device/vbd/5632 (19 xenbus_dev_probe on device/vbd/5632)
[    3.750127] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input5
[    3.761383] udevd (1159): /proc/1159/oom_adj is deprecated, please use /proc/1159/oom_score_adj instead.
[    3.833519] SCSI subsystem initialized
[    3.840470] ACPI: bus type ATA registered
[    3.847543] libata version 3.00 loaded.
[    3.853983] ata_piix 0000:00:01.1: version 2.13
[    3.859870] ata_piix 0000:00:01.1: setting latency timer to 64
[    3.878641] scsi0 : ata_piix
[    3.885457] scsi1 : ata_piix
[    3.889541] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc200 irq 14
[    3.898192] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc208 irq 15
udevd-work[1391]: error opening ATTR{/sys/devices/system/cpu/cpu0/online} for writing: No such file or directory

[    4.050772] ip (1723) used greatest stack depth: 3440 bytes left
[    4.061922] ata2.01: NODEV after polling detection
[    4.069697] ata2.00: ATAPI: QEMU DVD-ROM, 0.10.2, max UDMA/100
[    4.078630] ata2.00: configured for MWDMA2
[    4.086794] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     0.10 PQ: 0 ANSI: 5
[    4.156734] sr0: scsi3-mmc drive: 4x/4x xa/form2 tray
[    4.163052] cdrom: Uniform CD-ROM driver Revision: 3.20
[    4.197236] sr 1:0:0:0: Attached scsi CD-ROM sr0
[    4.220390] sr 1:0:0:0: Attached scsi generic sg0 type 5
Waiting for devices [  OK  ]
Waiting for init.pre_custom [  OK  ]
Waiting for fb [  OK  ]
Waiting for network [  OK  ]
Bringing up loopback interface:  [  OK  ]
Bringing up interface eth0:  
Determining IP information for eth0... done.
[  OK  ]
Waiting for init.custom [  OK  ]

Starting SSHd ...

    SSH started [2414]


Waiting for SSHd [  OK  ]
WARNING: ssh currently running [2414] ignoring start request
FATAL: Module dump_dma not found.
ERROR: Module dump_dma does not exist in /proc/modules
[    7.609564] Loading iSCSI transport class v2.0-870.
[    7.618305] iscsi: registered transport (tcp)
hostname: Name or service not known
iscsistart: transport class version 2.0-870. iscsid version 2.0-872
Could not get list of targets from firmware.
May  8 16:02:31 (none) syslogd 1.5.0: restart.
Running in HVM context on Xen v4.3.
You might have to do kill -1 1 if you see 'can't open /dev/hvc0'
[1:0:0:0]    cd/dvd  QEMU     QEMU DVD-ROM     0.10  /dev/sr0 
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
            CPU0       
   0:        176   IO-APIC-edge      timer
   1:          9  xen-pirq-ioapic-edge  i8042
   4:        235  xen-pirq-ioapic-edge  serial
   8:          4  xen-pirq-ioapic-edge  rtc0
   9:          0   IO-APIC-fasteoi   acpi
  12:        125  xen-pirq-ioapic-edge  i8042
  14:          0   IO-APIC-edge      ata_piix
  15:         96   IO-APIC-edge      ata_piix
  64:       3675  xen-percpu-virq      timer0
  65:          0  xen-percpu-ipi       resched0
  66:          0  xen-percpu-ipi       callfunc0
  67:          0  xen-percpu-virq      debug0
  68:          0  xen-percpu-ipi       callfuncsingle0
  69:        300   xen-dyn-event     xenbus
  71:          0   xen-dyn-event     vkbd
  72:         19   xen-dyn-event     eth0
 NMI:          0   Non-maskable interrupts
 LOC:          0   Local timer interrupts
 SPU:          0   Spurious interrupts
 PMI:          0   Performance monitoring interrupts
 IWI:          0   IRQ work interrupts
 RTR:          0   APIC ICR read retries
 RES:          0   Rescheduling interrupts
 CAL:          0   Function call interrupts
 TLB:          0   TLB shootdowns
 TRM:          0   Thermal event interrupts
 THR:          0   Threshold APIC interrupts
 MCE:          0   Machine check exceptions
 MCP:          1   Machine check polls
 ERR:          0
 MIS:          0
00000000-00000fff : reserved
00001000-0009dfff : System RAM
0009e000-0009ffff : reserved
000a0000-000bffff : PCI Bus 0000:00
000c0000-000c8bff : Video ROM
000c9000-000c99ff : Adapter ROM
000e0000-000fffff : reserved
  000f0000-000fffff : System ROM
00100000-7f7fffff : System RAM
  01000000-016a6728 : Kernel code
  016a6729-01cc52bf : Kernel data
  01dd7000-026f5fff : Kernel bss
7f800000-7fffffff : RAM buffer
f0000000-fbffffff : PCI Bus 0000:00
  f0000000-f1ffffff : 0000:00:02.0
    f0000000-f1ffffff : cirrusfb
  f2000000-f2ffffff : 0000:00:03.0
    f2000000-f2ffffff : xen-platform-pci
  f3000000-f3000fff : 0000:00:02.0
    f3000000-f3000fff : cirrusfb
fc000000-ffffffff : reserved
  fec00000-fec003ff : IOAPIC 0
  fed00000-fed003ff : HPET 0
  fee00000-fee00fff : Local APIC
MemTotal:        2024688 kB
MemFree:         1510268 kB
Buffers:               0 kB
Cached:           458324 kB
SwapCached:            0 kB
Active:            14096 kB
Inactive:         443996 kB
Active(anon):       7944 kB
Inactive(anon):    86980 kB
Active(file):       6152 kB
Inactive(file):   357016 kB
Unevictable:        4952 kB
Mlocked:            4952 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:          4736 kB
Mapped:             5024 kB
Shmem:             91572 kB
Slab:              38928 kB
SReclaimable:      20052 kB
SUnreclaim:        18876 kB
KernelStack:        1448 kB
PageTables:          660 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     1012344 kB
Committed_AS:     100504 kB
VmallocTotal:   34359738367 kB
VmallocUsed:        9184 kB
VmallocChunk:   34359728763 kB
AnonHugePages:         0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       49152 kB
DirectMap2M:     2039808 kB
Waiting for init.late [  OK  ]
PING build.dumpdata.com (192.168.102.1) 56(84) bytes of data.

--- build.dumpdata.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 1ms
rtt min/avg/max/mdev = 0.303/0.303/0.303/0.000 ms
[    8.135473] mount.nfs (2505) used greatest stack depth: 2768 bytes left
NFS done
libxl: error: libxl.c:87:libxl_ctx_alloc: Is xenstore daemon running?
failed to stat /var/run/xenstored.pid: No such file or directory
cannot init xl context
[    8.164452] device-mapper: ioctl: 4.24.0-ioctl (2013-01-15) initialised: dm-devel@redhat.com
[    8.175591] device-mapper: multipath: version 1.5.1 loaded
  No matching physical volumes found
  No volume groups found
 8 May 16:02:34 ntpdate[2524]: step time server 17.171.4.14 offset -0.507791 sec
Wed May  8 16:02:35 UTC 2013
Starting net testcase..
May  8 16:02:35 (none) init: starting pid 2541, tty '/dev/tty0': '/bin/sh'


BusyBox v1.14.3 (2013-05-05 20:35:32 EDT) built-in shell (ash)
Enter 'help' for a list of built-in commands.

May  8 16:02:35 (none) init: starting pid 2542, tty '/dev/tty1': '/bin/sh'
May  8 16:02:35 (none) init: starting pid 2543, tty '/dev/ttyS0': '/bin/sh'
May  8 16:02:35 (none) init: starting pid 2544, tty '/dev/hvc0': '/bin/sh'
Running Iperf Server as a daemon
Starting netserver at port 12865
The Iperf daemon process ID : 2545
Starting netserver at hostname 0.0.0.0 port 12865 and family AF_UNSPEC
# May  8 16:02:36 (none) init: process '/bin/sh' (pid 2544) exited. Scheduling for restart.
May  8 16:02:36 (none) init: starting pid 2547, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:37 (none) init: process '/bin/sh' (pid 2547) exited. Scheduling for restart.
May  8 16:02:37 (none) init: starting pid 2548, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:38 (none) init: process '/bin/sh' (pid 2548) exited. Scheduling for restart.
May  8 16:02:38 (none) init: starting pid 2549, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:39 (none) init: process '/bin/sh' (pid 2549) exited. Scheduling for restart.
May  8 16:02:39 (none) init: starting pid 2550, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:40 (none) init: process '/bin/sh' (pid 2550) exited. Scheduling for restart.
May  8 16:02:40 (none) init: starting pid 2551, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:41 (none) init: process '/bin/sh' (pid 2551) exited. Scheduling for restart.
May  8 16:02:41 (none) init: starting pid 2552, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:42 (none) init: process '/bin/sh' (pid 2552) exited. Scheduling for restart.
May  8 16:02:42 (none) init: starting pid 2553, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:43 (none) init: process '/bin/sh' (pid 2553) exited. Scheduling for restart.
May  8 16:02:43 (none) init: starting pid 2554, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:44 (none) init: process '/bin/sh' (pid 2554) exited. Scheduling for restart.
May  8 16:02:44 (none) init: starting pid 2555, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:45 (none) init: process '/bin/sh' (pid 2555) exited. Scheduling for restart.
May  8 16:02:45 (none) init: starting pid 2556, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:46 (none) init: process '/bin/sh' (pid 2556) exited. Scheduling for restart.
May  8 16:02:46 (none) init: starting pid 2557, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:47 (none) init: process '/bin/sh' (pid 2557) exited. Scheduling for restart.
May  8 16:02:47 (none) init: starting pid 2558, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:48 (none) init: process '/bin/sh' (pid 2558) exited. Scheduling for restart.
May  8 16:02:48 (none) init: starting pid 2559, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:49 (none) init: process '/bin/sh' (pid 2559) exited. Scheduling for restart.
May  8 16:02:49 (none) init: starting pid 2560, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:50 (none) init: process '/bin/sh' (pid 2560) exited. Scheduling for restart.
May  8 16:02:50 (none) init: starting pid 2561, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:51 (none) init: process '/bin/sh' (pid 2561) exited. Scheduling for restart.
May  8 16:02:51 (none) init: starting pid 2562, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:52 (none) init: process '/bin/sh' (pid 2562) exited. Scheduling for restart.
May  8 16:02:52 (none) init: starting pid 2563, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:53 (none) init: process '/bin/sh' (pid 2563) exited. Scheduling for restart.
May  8 16:02:53 (none) init: starting pid 2564, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:54 (none) init: process '/bin/sh' (pid 2564) exited. Scheduling for restart.
May  8 16:02:54 (none) init: starting pid 2565, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:55 (none) init: process '/bin/sh' (pid 2565) exited. Scheduling for restart.
May  8 16:02:55 (none) init: starting pid 2566, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:56 (none) init: process '/bin/sh' (pid 2566) exited. Scheduling for restart.
May  8 16:02:56 (none) init: starting pid 2567, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:57 (none) init: process '/bin/sh' (pid 2567) exited. Scheduling for restart.
May  8 16:02:57 (none) init: starting pid 2568, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:58 (none) init: process '/bin/sh' (pid 2568) exited. Scheduling for restart.
May  8 16:02:58 (none) init: starting pid 2569, tty '/dev/hvc0': '/bin/sh'
May  8 16:02:59 (none) init: process '/bin/sh' (pid 2569) exited. Scheduling for restart.
May  8 16:02:59 (none) init: starting pid 2570, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:00 (none) init: process '/bin/sh' (pid 2570) exited. Scheduling for restart.
May  8 16:03:00 (none) init: starting pid 2571, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:01 (none) init: process '/bin/sh' (pid 2571) exited. Scheduling for restart.
May  8 16:03:01 (none) init: starting pid 2572, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:02 (none) init: process '/bin/sh' (pid 2572) exited. Scheduling for restart.
May  8 16:03:02 (none) init: starting pid 2573, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:03 (none) init: process '/bin/sh' (pid 2573) exited. Scheduling for restart.
May  8 16:03:03 (none) init: starting pid 2574, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:04 (none) init: process '/bin/sh' (pid 2574) exited. Scheduling for restart.
May  8 16:03:04 (none) init: starting pid 2575, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:05 (none) init: process '/bin/sh' (pid 2575) exited. Scheduling for restart.
May  8 16:03:05 (none) init: starting pid 2576, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:06 (none) init: process '/bin/sh' (pid 2576) exited. Scheduling for restart.
May  8 16:03:06 (none) init: starting pid 2577, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:07 (none) init: process '/bin/sh' (pid 2577) exited. Scheduling for restart.
May  8 16:03:07 (none) init: starting pid 2578, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:08 (none) init: process '/bin/sh' (pid 2578) exited. Scheduling for restart.
May  8 16:03:08 (none) init: starting pid 2579, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:09 (none) init: process '/bin/sh' (pid 2579) exited. Scheduling for restart.
May  8 16:03:09 (none) init: starting pid 2580, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:10 (none) init: process '/bin/sh' (pid 2580) exited. Scheduling for restart.
May  8 16:03:10 (none) init: starting pid 2581, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:11 (none) init: process '/bin/sh' (pid 2581) exited. Scheduling for restart.
May  8 16:03:11 (none) init: starting pid 2582, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:12 (none) init: process '/bin/sh' (pid 2582) exited. Scheduling for restart.
May  8 16:03:12 (none) init: starting pid 2583, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:13 (none) init: process '/bin/sh' (pid 2583) exited. Scheduling for restart.
May  8 16:03:13 (none) init: starting pid 2584, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:14 (none) init: process '/bin/sh' (pid 2584) exited. Scheduling for restart.
May  8 16:03:14 (none) init: starting pid 2585, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:15 (none) init: process '/bin/sh' (pid 2585) exited. Scheduling for restart.
May  8 16:03:15 (none) init: starting pid 2586, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:16 (none) init: process '/bin/sh' (pid 2586) exited. Scheduling for restart.
May  8 16:03:16 (none) init: starting pid 2587, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:17 (none) init: process '/bin/sh' (pid 2587) exited. Scheduling for restart.
May  8 16:03:17 (none) init: starting pid 2588, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:18 (none) init: process '/bin/sh' (pid 2588) exited. Scheduling for restart.
May  8 16:03:18 (none) init: starting pid 2589, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:19 (none) init: process '/bin/sh' (pid 2589) exited. Scheduling for restart.
May  8 16:03:19 (none) init: starting pid 2590, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:20 (none) init: process '/bin/sh' (pid 2590) exited. Scheduling for restart.
May  8 16:03:20 (none) init: starting pid 2591, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:21 (none) init: process '/bin/sh' (pid 2591) exited. Scheduling for restart.
May  8 16:03:21 (none) init: starting pid 2592, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:22 (none) init: process '/bin/sh' (pid 2592) exited. Scheduling for restart.
May  8 16:03:22 (none) init: starting pid 2593, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:23 (none) init: process '/bin/sh' (pid 2593) exited. Scheduling for restart.
May  8 16:03:23 (none) init: starting pid 2594, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:24 (none) init: process '/bin/sh' (pid 2594) exited. Scheduling for restart.
May  8 16:03:24 (none) init: starting pid 2595, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:25 (none) init: process '/bin/sh' (pid 2595) exited. Scheduling for restart.
May  8 16:03:25 (none) init: starting pid 2596, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:26 (none) init: process '/bin/sh' (pid 2596) exited. Scheduling for restart.
May  8 16:03:26 (none) init: starting pid 2597, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:27 (none) init: process '/bin/sh' (pid 2597) exited. Scheduling for restart.
May  8 16:03:27 (none) init: starting pid 2598, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:28 (none) init: process '/bin/sh' (pid 2598) exited. Scheduling for restart.
May  8 16:03:28 (none) init: starting pid 2599, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:29 (none) init: process '/bin/sh' (pid 2599) exited. Scheduling for restart.
May  8 16:03:29 (none) init: starting pid 2600, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:30 (none) init: process '/bin/sh' (pid 2600) exited. Scheduling for restart.
May  8 16:03:30 (none) init: starting pid 2601, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:31 (none) init: process '/bin/sh' (pid 2601) exited. Scheduling for restart.
May  8 16:03:31 (none) init: starting pid 2602, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:32 (none) init: process '/bin/sh' (pid 2602) exited. Scheduling for restart.
May  8 16:03:32 (none) init: starting pid 2603, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:33 (none) init: process '/bin/sh' (pid 2603) exited. Scheduling for restart.
May  8 16:03:33 (none) init: starting pid 2604, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:34 (none) init: process '/bin/sh' (pid 2604) exited. Scheduling for restart.
May  8 16:03:34 (none) init: starting pid 2605, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:35 (none) init: process '/bin/sh' (pid 2605) exited. Scheduling for restart.
May  8 16:03:35 (none) init: starting pid 2606, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:36 (none) init: process '/bin/sh' (pid 2606) exited. Scheduling for restart.
May  8 16:03:36 (none) init: starting pid 2607, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:37 (none) init: process '/bin/sh' (pid 2607) exited. Scheduling for restart.
May  8 16:03:37 (none) init: starting pid 2608, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:38 (none) init: process '/bin/sh' (pid 2608) exited. Scheduling for restart.
May  8 16:03:38 (none) init: starting pid 2609, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:39 (none) init: process '/bin/sh' (pid 2609) exited. Scheduling for restart.
May  8 16:03:39 (none) init: starting pid 2610, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:40 (none) init: process '/bin/sh' (pid 2610) exited. Scheduling for restart.
May  8 16:03:40 (none) init: starting pid 2611, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:41 (none) init: process '/bin/sh' (pid 2611) exited. Scheduling for restart.
May  8 16:03:41 (none) init: starting pid 2612, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:42 (none) init: process '/bin/sh' (pid 2612) exited. Scheduling for restart.
May  8 16:03:42 (none) init: starting pid 2613, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:43 (none) init: process '/bin/sh' (pid 2613) exited. Scheduling for restart.
May  8 16:03:43 (none) init: starting pid 2614, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:44 (none) init: process '/bin/sh' (pid 2614) exited. Scheduling for restart.
May  8 16:03:44 (none) init: starting pid 2615, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:45 (none) init: process '/bin/sh' (pid 2615) exited. Scheduling for restart.
May  8 16:03:45 (none) init: starting pid 2616, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:46 (none) init: process '/bin/sh' (pid 2616) exited. Scheduling for restart.
May  8 16:03:46 (none) init: starting pid 2617, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:47 (none) init: process '/bin/sh' (pid 2617) exited. Scheduling for restart.
May  8 16:03:47 (none) init: starting pid 2618, tty '/dev/hvc0': '/bin/sh'
May  8 16:03:48 (none) udevd-work[2153]: error opening ATTR{/sys/devices/system/cpu/cpu1/online} for writing: No such file or directory 
[   84.585905] CPU 1 got hotplugged
[   84.590192] installing Xen timer for CPU 1
[   84.596371] SMP alternatives: lockdep: fixing up alternatives
[   84.603560] SMP alternatives: switching to SMP code
[   84.619508] smpboot: Booting Node 0 Processor 1 APIC 0x8
[   84.639766] ------------[ cut here ]------------
[   84.639766] WARNING: at /home/konrad/ssd/konrad/linux/arch/x86/xen/time.c:336 xen_vcpuop_set_mode+0xc2/0xd0()
[   84.639766] Hardware name: HVM domU
[   84.639766] VCPUOP_stop_singleshot_timer on CPU1 ret: -22
[   84.639766] Modules linked in: dm_multipath dm_mod iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi libcrc32c crc32c sg sr_mod cdrom ata_generic crc32c_intel ata_piix libata scsi_mod xen_blkfront xen_netfront fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xenfs xen_privcmd
[   84.639766] Pid: 0, comm: swapper/1 Not tainted 3.9.0upstream-00022-g49c1bf1-dirty #6
[   84.639766] Call Trace:
[   84.639766]  [<ffffffff8109261a>] warn_slowpath_common+0x7a/0xc0
[   84.639766]  [<ffffffff81092701>] warn_slowpath_fmt+0x41/0x50
[   84.639766]  [<ffffffff810e8ae4>] ? tick_notify+0x114/0x420
[   84.639766]  [<ffffffff81044fe2>] xen_vcpuop_set_mode+0xc2/0xd0
[   84.639766]  [<ffffffff810e8385>] clockevents_set_mode+0x25/0x70
[   84.639766]  [<ffffffff810e83e6>] clockevents_shutdown+0x16/0x30
[   84.639766]  [<ffffffff810e84b7>] clockevents_exchange_device+0xb7/0x110
[   84.639766]  [<ffffffff810e8ae4>] ? tick_notify+0x114/0x420
[   84.639766]  [<ffffffff810e8b99>] tick_notify+0x1c9/0x420
[   84.639766]  [<ffffffff810e8221>] ? clockevents_register_device+0x31/0x170
[   84.639766]  [<ffffffff8169dddd>] notifier_call_chain+0x4d/0x70
[   84.639766]  [<ffffffff810be851>] raw_notifier_call_chain+0x11/0x20
[   84.639766]  [<ffffffff810e82d0>] clockevents_register_device+0xe0/0x170
[   84.639766]  [<ffffffff8104507c>] xen_setup_cpu_clockevents+0x2c/0x50
[   84.639766]  [<ffffffff810450b6>] xen_hvm_setup_cpu_clockevents+0x16/0x20
[   84.639766]  [<ffffffff8168d4af>] start_secondary+0x1ea/0x1f9
[   84.639766] ---[ end trace f218984223a7067d ]---
[   84.639766] ------------[ cut here ]------------
[   84.639766] WARNING: at /home/konrad/ssd/konrad/linux/arch/x86/xen/time.c:338 xen_vcpuop_set_mode+0x9f/0xd0()
[   84.639766] Hardware name: HVM domU
[   84.639766] VCPUOP_stop_periodic_timer on CPU1 ret: -22
[   84.639766] Modules linked in: dm_multipath dm_mod iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi libcrc32c crc32c sg sr_mod cdrom ata_generic crc32c_intel ata_piix libata scsi_mod xen_blkfront xen_netfront fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xenfs xen_privcmd
[   84.639766] Pid: 0, comm: swapper/1 Tainted: G        W    3.9.0upstream-00022-g49c1bf1-dirty #6
[   84.639766] Call Trace:
[   84.639766]  [<ffffffff8109261a>] warn_slowpath_common+0x7a/0xc0
[   84.639766]  [<ffffffff81092701>] warn_slowpath_fmt+0x41/0x50
[   84.639766]  [<ffffffff810e8ae4>] ? tick_notify+0x114/0x420
[   84.639766]  [<ffffffff81044fbf>] xen_vcpuop_set_mode+0x9f/0xd0
[   84.639766]  [<ffffffff810e8385>] clockevents_set_mode+0x25/0x70
[   84.639766]  [<ffffffff810e83e6>] clockevents_shutdown+0x16/0x30
[   84.639766]  [<ffffffff810e84b7>] clockevents_exchange_device+0xb7/0x110
[   84.639766]  [<ffffffff810e8ae4>] ? tick_notify+0x114/0x420
[   84.639766]  [<ffffffff810e8b99>] tick_notify+0x1c9/0x420
[   84.639766]  [<ffffffff810e8221>] ? clockevents_register_device+0x31/0x170
[   84.639766]  [<ffffffff8169dddd>] notifier_call_chain+0x4d/0x70
[   84.639766]  [<ffffffff810be851>] raw_notifier_call_chain+0x11/0x20
[   84.639766]  [<ffffffff810e82d0>] clockevents_register_device+0xe0/0x170
[   84.639766]  [<ffffffff8104507c>] xen_setup_cpu_clockevents+0x2c/0x50
[   84.639766]  [<ffffffff810450b6>] xen_hvm_setup_cpu_clockevents+0x16/0x20
[   84.639766]  [<ffffffff8168d4af>] start_secondary+0x1ea/0x1f9
[   84.639766] ---[ end trace f218984223a7067e ]---
[   84.639766] ------------[ cut here ]------------
[   84.639766] kernel BUG at /home/konrad/ssd/konrad/linux/arch/x86/xen/time.c:340!
[   84.639766] invalid opcode: 0000 [#1] SMP 
[   84.639766] Modules linked in: dm_multipath dm_mod iscsi_boot_sysfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi libcrc32c crc32c sg sr_mod cdrom ata_generic crc32c_intel ata_piix libata scsi_mod xen_blkfront xen_netfront fb_sys_fops sysimgblt sysfillrect syscopyarea xen_kbdfront xenfs xen_privcmd
[   84.639766] CPU 1 
[   84.639766] Pid: 0, comm: swapper/1 Tainted: G        W    3.9.0upstream-00022-g49c1bf1-dirty #6 Xen HVM domU
[   84.639766] RIP: 0010:[<ffffffff81044fbf>]  [<ffffffff81044fbf>] xen_vcpuop_set_mode+0x9f/0xd0
[   84.639766] RSP: 0000:ffff880073c6bd98  EFLAGS: 00010092
[   84.639766] RAX: 0000000000000024 RBX: 0000000000000001 RCX: 0000000000000000
[   84.639766] RDX: ffff880073c68300 RSI: 0000000000000000 RDI: 0000000000000009
[   84.639766] RBP: ffff880073c6bdb8 R08: 0000000000000001 R09: 0000000000000000
[   84.639766] R10: 0000000000000258 R11: 0000000000000000 R12: 00000000ffffffea
[   84.639766] R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000082
[   84.639766] FS:  0000000000000000(0000) GS:ffff880074220000(0000) knlGS:0000000000000000
[   84.639766] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[   84.639766] CR2: 0000000000000000 CR3: 0000000001c0c000 CR4: 00000000000406e0
[   84.639766] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   84.639766] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[   84.639766] Process swapper/1 (pid: 0, threadinfo ffff880073c6a000, task ffff880073c68300)
[   84.639766] Stack:
[   84.639766]  0000000000000000 ffff88007422be40 0000000000000001 0000000000000082
[   84.639766]  ffff880073c6bdd8 ffffffff810e8385 ffff88007422be40 ffff88007422be40
[   84.639766]  ffff880073c6bdf8 ffffffff810e83e6 ffff880073c6bdf8 0000000000000000
[   84.639766] Call Trace:
[   84.639766]  [<ffffffff810e8385>] clockevents_set_mode+0x25/0x70
[   84.639766]  [<ffffffff810e83e6>] clockevents_shutdown+0x16/0x30
[   84.639766]  [<ffffffff810e84b7>] clockevents_exchange_device+0xb7/0x110
[   84.639766]  [<ffffffff810e8ae4>] ? tick_notify+0x114/0x420
[   84.639766]  [<ffffffff810e8b99>] tick_notify+0x1c9/0x420
[   84.639766]  [<ffffffff810e8221>] ? clockevents_register_device+0x31/0x170
[   84.639766]  [<ffffffff8169dddd>] notifier_call_chain+0x4d/0x70
[   84.639766]  [<ffffffff810be851>] raw_notifier_call_chain+0x11/0x20
[   84.639766]  [<ffffffff810e82d0>] clockevents_register_device+0xe0/0x170
[   84.639766]  [<ffffffff8104507c>] xen_setup_cpu_clockevents+0x2c/0x50
[   84.639766]  [<ffffffff810450b6>] xen_hvm_setup_cpu_clockevents+0x16/0x20
[   84.639766]  [<ffffffff8168d4af>] start_secondary+0x1ea/0x1f9
[   84.639766] Code: 00 48 c7 c7 f8 bc 9b 81 e8 bf d6 04 00 eb c9 89 d9 48 c7 c2 98 bd 9b 81 be 52 01 00 00 48 c7 c7 f8 bc 9b 81 31 c0 e8 01 d7 04 00 <0f> 0b eb fe 41 89 c0 89 d9 48 c7 c2 68 bd 9b 81 be 50 01 00 00 
[   84.639766] RIP  [<ffffffff81044fbf>] xen_vcpuop_set_mode+0x9f/0xd0
[   84.639766]  RSP <ffff880073c6bd98>
[   84.639766] ---[ end trace f218984223a7067f ]---
[   84.639766] Kernel panic - not syncing: Attempted to kill the idle task!
Parsing config from /mnt/lab/latest/vm.cfg
Daemon running with PID 7341

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Matt Wilson]

On Tue, Jun 04, 2013 at 10:22:24AM -0400, Konrad Rzeszutek Wilk wrote:
> > > > > The new hypercall to figure this out could be used, but that wouldn't
> > > > > explain why we are failing to start on the correct VCPU?
> > > > 
> > > > I didn't follow the jump here. Can you provide an example?
> > > 
> > > http://lists.xen.org/archives/html/xen-devel/2013-05/msg00941.html
> > 
> > OK, got it.
> > 
> > [   84.619508] smpboot: Booting Node 0 Processor 1 APIC 0x8
> > 
> > So it seems like, in this case:
> > 
> > int __cpuinit native_cpu_up(unsigned int cpu)
> > {
> >         int apicid = apic->cpu_present_to_apicid(cpu);
> > 
> > apic->cpu_present_to_apicid(1) returned 8 instead of 2.
> > 
> > All of that should have been set up correctly ahead of time by
> > generic_processor_info() for all possible CPUs. Do you have the full
> > boot log?
> > 

[...]

> [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] disabled)
[...]
> [    0.000000] ACPI: LAPIC (acpi_id[0x7b] lapic_id[0xf6] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x7c] lapic_id[0xf8] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x7d] lapic_id[0xfa] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x7e] lapic_id[0xfc] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x7f] lapic_id[0xfe] disabled)

OK, so generic_processor_info() is not called for disabled processors,
so it must happen at processor UP time.

Can you boot with "acpi.debug_layer=0x20000000 acpi.debug_level=0x4"?

--msw

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Konrad Rzeszutek Wilk]

On Wed, Jun 05, 2013 at 09:15:41AM -0700, Matt Wilson wrote:
> On Tue, Jun 04, 2013 at 10:22:24AM -0400, Konrad Rzeszutek Wilk wrote:
> > > > > > The new hypercall to figure this out could be used, but that wouldn't
> > > > > > explain why we are failing to start on the correct VCPU?
> > > > > 
> > > > > I didn't follow the jump here. Can you provide an example?
> > > > 
> > > > http://lists.xen.org/archives/html/xen-devel/2013-05/msg00941.html
> > > 
> > > OK, got it.
> > > 
> > > [   84.619508] smpboot: Booting Node 0 Processor 1 APIC 0x8
> > > 
> > > So it seems like, in this case:
> > > 
> > > int __cpuinit native_cpu_up(unsigned int cpu)
> > > {
> > >         int apicid = apic->cpu_present_to_apicid(cpu);
> > > 
> > > apic->cpu_present_to_apicid(1) returned 8 instead of 2.
> > > 
> > > All of that should have been set up correctly ahead of time by
> > > generic_processor_info() for all possible CPUs. Do you have the full
> > > boot log?
> > > 
> 
> [...]
> 
> > [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] disabled)
> [...]
> > [    0.000000] ACPI: LAPIC (acpi_id[0x7b] lapic_id[0xf6] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x7c] lapic_id[0xf8] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x7d] lapic_id[0xfa] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x7e] lapic_id[0xfc] disabled)
> > [    0.000000] ACPI: LAPIC (acpi_id[0x7f] lapic_id[0xfe] disabled)
> 
> OK, so generic_processor_info() is not called for disabled processors,
> so it must happen at processor UP time.
> 
> Can you boot with "acpi.debug_layer=0x20000000 acpi.debug_level=0x4"?

If I can reproduce it again - absolutly. Right now the race seems to have
gone away and the APIC id is 0x02 (the right value) instead of 0x08.

Re: HVM CPU enumeration, mapping to VCPU ID (Was: Re: [Xen-users] FreeBSD PVHVM call for testing)

[Konrad Rzeszutek Wilk]

On Tue, Jun 04, 2013 at 10:22:24AM -0400, Konrad Rzeszutek Wilk wrote:
> > > > > The new hypercall to figure this out could be used, but that wouldn't
> > > > > explain why we are failing to start on the correct VCPU?
> > > > 
> > > > I didn't follow the jump here. Can you provide an example?
> > > 
> > > http://lists.xen.org/archives/html/xen-devel/2013-05/msg00941.html
> > 
> > OK, got it.
> > 
> > [   84.619508] smpboot: Booting Node 0 Processor 1 APIC 0x8
> > 
> > So it seems like, in this case:
> > 
> > int __cpuinit native_cpu_up(unsigned int cpu)
> > {
> >         int apicid = apic->cpu_present_to_apicid(cpu);
> > 
> > apic->cpu_present_to_apicid(1) returned 8 instead of 2.
> > 
> > All of that should have been set up correctly ahead of time by
> > generic_processor_info() for all possible CPUs. Do you have the full
> > boot log?
> > 
> 
..snip...
> [    0.000000] ACPI: PM-Timer IO Port: 0xb008
> [    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] disabled)
> [    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] disabled)
                                                     ^^^^ - take a note of that
.. snip..
> [   84.585905] CPU 1 got hotplugged
> [   84.590192] installing Xen timer for CPU 1
> [   84.596371] SMP alternatives: lockdep: fixing up alternatives
> [   84.603560] SMP alternatives: switching to SMP code
> [   84.619508] smpboot: Booting Node 0 Processor 1 APIC 0x8
                                                          ^^^ and that

> [   84.639766] ------------[ cut here ]------------
> [   84.639766] WARNING: at /home/konrad/ssd/konrad/linux/arch/x86/xen/time.c:336 xen_vcpuop_set_mode+0xc2/0xd0()
> [   84.639766] Hardware name: HVM domU

I discussed this with Matt over IRC, but the analysis was that the APIC
ID is wrong. Instead of using APIC 0x02 for CPU1, it ended up using APIC 0x08
(which is for CPU4). And that triggered the xen_vcpuop_set_mode to fail
(as the hypervisor would say - you are running on CPU4, not CPU1, return
-ENODEV) and we hit the BUG_ON() in the Linux.

Chuck Anderson (CC-ed here) discovered that he was hitting this as well
and realized that if he was using the QEMU with these three patches:

169b8fa piix4acpi, xen, hotplug: Fix race with ACPI AML code and hotplug.
309149c piix4acpi, xen: Clarify that the qemu_set_irq calls just do an IRQ pulse.
82b10d1 piix4acpi, xen, vcpu hotplug: Split the notification from the changes.

the problem would go away. We did not go any deeper to figure out the culprit.

My money is on the AML code modifying the MADT and while it is executing
another ACPI GSI is triggered (the old code would trigger it for every CPU
being hot-plugged) and as the Linux ACPI AML interpreter reads the MADT and writes
out the newly updated MADT another of the CPUs ends up reading the MADT as well -
but ends up with the data being bogus as the AML interpreter did not finish
and got garbage.

Either way, the resolution is if you see this - the fix is to update the
qemu to have those patches.

Thanks Chuck for digging in this and finding the clues.


P.S.
I don't know how SeaBIOS does the CPU hotplug, but it might be worth looking
at that too at some point.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 29/05/13 19:22, Matt Wilson wrote:
> On Wed, May 29, 2013 at 07:03:40PM +0200, Roger Pau Monné wrote:
>>
>> Hello,
>>
>> Thanks Matt and Colin for the testing and help! I've pushed yet another
>> version, now it's branch pvhvm_v12, which I *think* should solve the
>> issues with cpuid != acpi_id:
>>
>> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12
>>
>> Since I'm not able to reproduce the cpuid != acpi_id case, could you
>> give it a try and report the results?
> 
> Colin, can you build an AMI with this new kernel?
>  
> [...]
> 
>> On 28/05/13 23:33, Colin Percival wrote:
>>> On a cc2.8xlarge EC2 instance, the lines which come after this are
>>>> GEOM: new disk xbd1
>>>> GEOM: new disk xbd2
>>>> GEOM: new disk xbd3
>>>> GEOM: new disk xbd4
>>>> Trying to mount root from ufs:/dev/ad0a [rw]...
>>>> start_init: trying /sbin/init
>>> and then the userland boot process; have you made any bug fixes after
>>> your pvhvm_v7 which would explain why tasting disks was hanging?
>>
>> I'm not sure I follow, did you found a regression from previous
>> branches? i.e. it used to work with branch pvhvm_v6 and not pvhvm_v7?
> 
> Colin was saying that his local change only moved the boot process a
> bit farther for cr1.8xlarge. Perhaps some of the other changes you
> made in the latest pvhvm_v12 branch will get the VM all the way up.

Oh, sure, more changes where needed in order to get it to work, like
using acpi_id to map the vcpu_info and perform the cpu bindings.

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/29/13 10:45, Roger Pau Monné wrote:
> Oh, sure, more changes where needed in order to get it to work, like
> using acpi_id to map the vcpu_info and perform the cpu bindings.

Ah, that explains it.  I looked at timer.c for other places where that
change was needed but it didn't occur to me that the same problem would
exist in other parts of the tree.

> On 29/05/13 19:22, Matt Wilson wrote:
>> On Wed, May 29, 2013 at 07:03:40PM +0200, Roger Pau Monné wrote:
>>> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12
>>
>> Colin, can you build an AMI with this new kernel?

Done, ami-95177dfc in us-east-1.

This is now booting successfully on cr1.8xlarge; are there any other instance
types I should test?  I don't know which Xen versions you have deployed across
the entire fleet.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Re: FreeBSD PVHVM call for testing

[Matt Wilson]

On Wed, May 29, 2013 at 01:58:59PM -0700, Colin Percival wrote:
> On 29/05/13 19:22, Matt Wilson wrote:
> > Colin, can you build an AMI with this new kernel?
> 
> Done, ami-95177dfc in us-east-1.
> 
> This is now booting successfully on cr1.8xlarge; are there any other instance
> types I should test?  I don't know which Xen versions you have deployed across
> the entire fleet.

Nice! The other interesting instance types for problems like this are
cc1.4xlarge, cg1.4xlarge and cc2.8xlarge.

--msw

Re: FreeBSD PVHVM call for testing

[Colin Percival]

On 05/29/13 15:19, Matt Wilson wrote:
> On Wed, May 29, 2013 at 01:58:59PM -0700, Colin Percival wrote:
>> On 29/05/13 19:22, Matt Wilson wrote:
>>> Colin, can you build an AMI with this new kernel?
>>
>> Done, ami-95177dfc in us-east-1.
>>
>> This is now booting successfully on cr1.8xlarge; are there any other instance
>> types I should test?  I don't know which Xen versions you have deployed across
>> the entire fleet.
> 
> Nice! The other interesting instance types for problems like this are
> cc1.4xlarge, cg1.4xlarge and cc2.8xlarge.

I've tried on all three of those and everything seems good -- but I'm only
seeing Xen 3.4 on those instance types, so we're not really testing anything
interesting there.

I'll resist asking whether there will be Xen 4.x anywhere else in EC2 in the
near future since I've had enough of a taste of Amazon NDAs to know that I
almost certainly wouldn't get an answer... ;-)

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Re: FreeBSD PVHVM call for testing

[Konrad Rzeszutek Wilk]

On Mon, May 13, 2013 at 08:32:56PM +0200, Roger Pau Monné wrote:
> Hello,
> 
> Recently Justin T Gibbs, Will Andrews and myself have been working on
> improving the Xen support in FreeBSD. The main goal of this was to bring
> full PVHVM support to FreeBSD, right now FreeBSD is only using PV
> interfaces for disk and network interfaces when running as a HVM guest.
> The main benefits of this changes are that Xen virtual interrupts (event
> channels) are now delivered to the guest using a vector callback
> injection, that is a per-cpu mechanism that allows each vCPU to have
> different interrupts assigned, so for example network and disk
> interrupts are delivered to different vCPUs in order to improve
> performance. With this changes FreeBSD also uses PV timers when running
> as an HVM guest, which should provide better time keeping and reduce the
> virtualization overhead, since emulated timers are no longer used. PV
> IPIs can also be used inside a HVM guest, but this will be implemented
> later.
> 
> Right now the code is in a state where it can be tested by users, so we
> would like to encourage FreeBSD and Xen users to test it and provide
> feedback.
> 
> The code is available in the following git repository, under the branch
> pvhvm_v5:
> 
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary
> 
> Also, I've created a wiki page that explains how to set up a FreeBSD
> PVHVM for testing:
> 
> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM

I tried on my Linux box to do this:


HEAD is now at 9b25356... xen-netfront: fix detach of network interfaces
konrad@phenom:~/git/freebsd$ make kernel-toolchain && make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM
Makefile:123: *** missing separator.  Stop.


As I thought it would compile the same way you can compile NetBSD - that is
even on non-BSD distros. Is that not the case? Should I only do this under
a FreeBSD guest?

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 21/05/13 19:40, Konrad Rzeszutek Wilk wrote:
> On Mon, May 13, 2013 at 08:32:56PM +0200, Roger Pau Monné wrote:
>> Hello,
>>
>> Recently Justin T Gibbs, Will Andrews and myself have been working on
>> improving the Xen support in FreeBSD. The main goal of this was to bring
>> full PVHVM support to FreeBSD, right now FreeBSD is only using PV
>> interfaces for disk and network interfaces when running as a HVM guest.
>> The main benefits of this changes are that Xen virtual interrupts (event
>> channels) are now delivered to the guest using a vector callback
>> injection, that is a per-cpu mechanism that allows each vCPU to have
>> different interrupts assigned, so for example network and disk
>> interrupts are delivered to different vCPUs in order to improve
>> performance. With this changes FreeBSD also uses PV timers when running
>> as an HVM guest, which should provide better time keeping and reduce the
>> virtualization overhead, since emulated timers are no longer used. PV
>> IPIs can also be used inside a HVM guest, but this will be implemented
>> later.
>>
>> Right now the code is in a state where it can be tested by users, so we
>> would like to encourage FreeBSD and Xen users to test it and provide
>> feedback.
>>
>> The code is available in the following git repository, under the branch
>> pvhvm_v5:
>>
>> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary
>>
>> Also, I've created a wiki page that explains how to set up a FreeBSD
>> PVHVM for testing:
>>
>> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM
> 
> I tried on my Linux box to do this:
> 
> 
> HEAD is now at 9b25356... xen-netfront: fix detach of network interfaces
> konrad@phenom:~/git/freebsd$ make kernel-toolchain && make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM
> Makefile:123: *** missing separator.  Stop.
> 
> 
> As I thought it would compile the same way you can compile NetBSD - that is
> even on non-BSD distros. Is that not the case? Should I only do this under
> a FreeBSD guest?

I have never tried to run the FreeBSD build system under something
different than FreeBSD, also keep in mind that installkernel will place
a bunch of FreeBSD files on your Linux box if you manage to run it. The
error itself can probably be fixed by installing and using BSD make
(bmake), but anyway you need a FreeBSD HVM DomU in order to test the
kernel, so why not do the compilation on it?

Re: FreeBSD PVHVM call for testing

[Konrad Rzeszutek Wilk]

On Wed, May 22, 2013 at 01:41:02PM +0200, Roger Pau Monné wrote:
> On 21/05/13 19:40, Konrad Rzeszutek Wilk wrote:
> > On Mon, May 13, 2013 at 08:32:56PM +0200, Roger Pau Monné wrote:
> >> Hello,
> >>
> >> Recently Justin T Gibbs, Will Andrews and myself have been working on
> >> improving the Xen support in FreeBSD. The main goal of this was to bring
> >> full PVHVM support to FreeBSD, right now FreeBSD is only using PV
> >> interfaces for disk and network interfaces when running as a HVM guest.
> >> The main benefits of this changes are that Xen virtual interrupts (event
> >> channels) are now delivered to the guest using a vector callback
> >> injection, that is a per-cpu mechanism that allows each vCPU to have
> >> different interrupts assigned, so for example network and disk
> >> interrupts are delivered to different vCPUs in order to improve
> >> performance. With this changes FreeBSD also uses PV timers when running
> >> as an HVM guest, which should provide better time keeping and reduce the
> >> virtualization overhead, since emulated timers are no longer used. PV
> >> IPIs can also be used inside a HVM guest, but this will be implemented
> >> later.
> >>
> >> Right now the code is in a state where it can be tested by users, so we
> >> would like to encourage FreeBSD and Xen users to test it and provide
> >> feedback.
> >>
> >> The code is available in the following git repository, under the branch
> >> pvhvm_v5:
> >>
> >> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary
> >>
> >> Also, I've created a wiki page that explains how to set up a FreeBSD
> >> PVHVM for testing:
> >>
> >> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM
> > 
> > I tried on my Linux box to do this:
> > 
> > 
> > HEAD is now at 9b25356... xen-netfront: fix detach of network interfaces
> > konrad@phenom:~/git/freebsd$ make kernel-toolchain && make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM
> > Makefile:123: *** missing separator.  Stop.
> > 
> > 
> > As I thought it would compile the same way you can compile NetBSD - that is
> > even on non-BSD distros. Is that not the case? Should I only do this under
> > a FreeBSD guest?
> 
> I have never tried to run the FreeBSD build system under something
> different than FreeBSD, also keep in mind that installkernel will place
> a bunch of FreeBSD files on your Linux box if you manage to run it. The
> error itself can probably be fixed by installing and using BSD make
> (bmake), but anyway you need a FreeBSD HVM DomU in order to test the
> kernel, so why not do the compilation on it?

Oh, sure. Just thought that the NetBSD method (which also allows one
to build the kernel and the iso) was the same on FreeBSd. Will of course
install first the HVM domU and do it again.
> 

Re: FreeBSD PVHVM call for testing

[Yuriy Kohut]

[-- Attachment #1.1: Type: text/plain, Size: 6876 bytes --]

Hi,

I've just successfully run FreeBSD 9.1 based guest with 'pvhvm_v8' based kernel under Xen 3.4.4.

Hypervisor details:
# xm info
host                   : *******
release                : 2.6.18-348.2.1.el5xen
version                : #1 SMP Tue Mar 5 17:05:33 EST 2013
machine                : x86_64
nr_cpus                : 4
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 1
cpu_mhz                : 2128
hw_caps                : bfebfbff:28100800:00000000:00000340:009ce3bd:00000000:00000001:00000000
virt_caps              : hvm
total_memory           : 6135
free_memory            : 262
node_to_cpu            : node0:0-3
node_to_memory         : node0:262
xen_major              : 3
xen_minor              : 4
xen_extra              : .4
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
cc_compiler            : gcc version 4.1.2 20080704 (Red Hat 4.1.2-52)
cc_compile_by          : root
cc_compile_domain      : ******
cc_compile_date        : Wed Sep  5 18:01:10 EEST 2012
xend_config_format     : 4


DomU details:
# xm list --long h283bpm53f9rnx
(domain
    (domid 61)
    (on_crash restart)
    (uuid a2cbcba9-1d66-87ce-6d2f-412e70eab051)
    (bootloader_args )
    (vcpus 2)
    (name h283bpm53f9rnx)
    (on_poweroff destroy)
    (on_reboot restart)
    (cpus (() ()))
    (bootloader )
    (maxmem 1024)
    (memory 1024)
    (shadow_memory 10)
    (features )
    (on_xend_start ignore)
    (on_xend_stop ignore)
    (start_time 1369235607.92)
    (cpu_time 31.914003553)
    (online_vcpus 2)
    (image
        (hvm
            (kernel )
            (videoram 4)
            (hpet 0)
            (stdvga 0)
            (loader /usr/lib/xen/boot/hvmloader)
            (vncunused 1)
            (xen_platform_pci 1)
            (boot cd)
            (rtc_timeoffset 7202)
            (pci ())
            (pae 1)
            (vpt_align 1)
            (hap 1)
            (viridian 0)
            (acpi 1)
            (localtime 0)
            (timer_mode 1)
            (vnc 1)
            (nographic 0)
            (guest_os_type default)
            (pci_msitranslate 1)
            (apic 1)
            (monitor 0)
            (usbdevice tablet)
            (device_model /usr/lib64/xen/bin/qemu-dm)
            (pci_power_mgmt 0)
            (usb 0)
            (xauthority /root/.Xauthority)
            (isa 0)
            (notes (SUSPEND_CANCEL 1))
        )
    )
    (status 2)
    (state -b----)
    (store_mfn 1044476)
    (device
        (vif
            (bridge xnh5getjoj54ke)
            (uuid 6133c146-48ea-b7a5-0263-fda98e1a30fe)
            (script /etc/xen/scripts/vif-bridge)
            (ip 83.170.81.183)
            (mac 00:16:3e:a4:02:5a)
            (vifname t2vd5w22msrv5d)
            (backend 0)
        )
    )
    (device
        (vbd
            (protocol x86_64-abi)
            (uuid dd857cd1-2a4c-ea21-a5a5-e95d811f607a)
            (bootable 1)
            (dev hda:disk)
            (uname phy:/dev/9yblt1m70pdtdp/ddfhogyred6bby)
            (mode w)
            (backend 0)
            (bootable 1)
            (VDI )
        )
    )
    (device
        (vbd
            (protocol x86_64-abi)
            (uuid 19cae15c-354d-77cb-57ec-dc313f1d05ba)
            (bootable 0)
            (dev hdb:disk)
            (uname phy:/dev/9yblt1m70pdtdp/dhnnwhs6jh9kdd)
            (mode w)
            (backend 0)
            (bootable 0)
            (VDI )
        )
    )
    (device
        (vbd
            (protocol x86_64-abi)
            (uuid 2b97ec8c-0cc5-7197-f510-63c272449680)
            (bootable 0)
            (dev hdc:disk)
            (uname phy:/dev/9yblt1m70pdtdp/d1jilc7s7jxsaq)
            (mode w)
            (backend 0)
            (bootable 0)
            (VDI )
        )
    )
    (device
        (vbd
            (protocol x86_64-abi)
            (uuid 1b472270-1ef3-2f49-81f1-031cc00c0eb7)
            (bootable 0)
            (dev hdd:cdrom)
            (uname file:/tools/freebsd/boot-freebsd-generic.iso)
            (mode r)
            (backend 0)
            (bootable 0)
            (VDI )
        )
    )
    (device
        (vfb
            (vncunused 1)
            (vnc 1)
            (uuid b3defeea-4acc-1408-9b22-71547a64e705)
            (location 0.0.0.0:5900)
        )
    )
    (device
        (console
            (protocol vt100)
            (location 4)
            (uuid 58a089ce-a4d0-037e-23e8-9df37b2bd5da)
        )
    )
)


DomU from "inside":
# uname -a
FreeBSD yurak1.vm 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r+03cdadc: Wed May 22 17:47:40 EEST 2013     root@yurak1.vm:/usr/obj/data/freebsd/sys/XENHVM  amd64


I'll also set up one (hope will have some time) under Xen 4.2.2 tomorrow.
---
Yura

On May 13, 2013, at 21:32 PM, Roger Pau Monné <roger.pau@citrix.com> wrote:

> Hello,
> 
> Recently Justin T Gibbs, Will Andrews and myself have been working on
> improving the Xen support in FreeBSD. The main goal of this was to bring
> full PVHVM support to FreeBSD, right now FreeBSD is only using PV
> interfaces for disk and network interfaces when running as a HVM guest.
> The main benefits of this changes are that Xen virtual interrupts (event
> channels) are now delivered to the guest using a vector callback
> injection, that is a per-cpu mechanism that allows each vCPU to have
> different interrupts assigned, so for example network and disk
> interrupts are delivered to different vCPUs in order to improve
> performance. With this changes FreeBSD also uses PV timers when running
> as an HVM guest, which should provide better time keeping and reduce the
> virtualization overhead, since emulated timers are no longer used. PV
> IPIs can also be used inside a HVM guest, but this will be implemented
> later.
> 
> Right now the code is in a state where it can be tested by users, so we
> would like to encourage FreeBSD and Xen users to test it and provide
> feedback.
> 
> The code is available in the following git repository, under the branch
> pvhvm_v5:
> 
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary
> 
> Also, I've created a wiki page that explains how to set up a FreeBSD
> PVHVM for testing:
> 
> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM
> _______________________________________________
> freebsd-xen@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-xen
> To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org"


[-- Attachment #1.2: Type: text/html, Size: 13205 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
> Right now the code is in a state where it can be tested by users, so we
> would like to encourage FreeBSD and Xen users to test it and provide
> feedback.

I've just been able to install it on a VPS using the latest pvhvm_v9 branch.
This is good news, because the system I had before actually had trouble with the HVM kernel from 9.1 [0].

I'm going to leave this running for a while and do some more tests on it.

Jeroen.


[0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
> Also, I've created a wiki page that explains how to set up a FreeBSD
> PVHVM for testing:
> 
> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM


You mention on that page that it is easier to install on 10.0-CURRENT snapshots.
What are the issues with installing this on 9.1? Is it possible?

Jeroen.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 23/05/13 15:20, Jeroen van der Ham wrote:
> 
> On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
>> Also, I've created a wiki page that explains how to set up a FreeBSD
>> PVHVM for testing:
>>
>> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM
> 
> 
> You mention on that page that it is easier to install on 10.0-CURRENT snapshots.
> What are the issues with installing this on 9.1? Is it possible?

I don't think it is recommended to use a HEAD (10) kernel with a 9.1
userland. You can always install a 9.1 and then do a full update with
the source on my repository.

Re: FreeBSD PVHVM call for testing

[Craig Rodrigues]

[-- Attachment #1.1: Type: text/plain, Size: 1715 bytes --]

On Thu, May 23, 2013 at 6:30 AM, Roger Pau Monné <roger.pau@citrix.com>wrote:

> On 23/05/13 15:20, Jeroen van der Ham wrote:
> >
> > On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
> >> Also, I've created a wiki page that explains how to set up a FreeBSD
> >> PVHVM for testing:
> >>
> >> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM
> >
> >
> > You mention on that page that it is easier to install on 10.0-CURRENT
> snapshots.
> > What are the issues with installing this on 9.1? Is it possible?
>
> I don't think it is recommended to use a HEAD (10) kernel with a 9.1
> userland. You can always install a 9.1 and then do a full update with
> the source on my repository.
>


Actually in FreeBSD, it is possible to run an older userland on a newer
kernel,
and a lot of effort is spent in preserving this type of backwards
compatibility.
So a 9.1 userland with a 10 kernel will work.
However, running a newer userland on an older kernel is not guaranteed to
work.
So running a 10 userland with a 9.1 kernel will most likely not work.

However, since you guys are doing very cutting edge stuff with 10-CURRENT,
it is better that you do not waste time with 9.1.

I recommend you start with a 10.0 CURRENT snapshot ISO and go from there.
I am going through a similar setup exercise with a Google Summer of Code
student
where he needs to have a latest CURRENT system running in a VM.

I wrote this blog post:

http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/

for the steps how to do it.  You can follow those steps to get bootstrapped
with a working environment if it helps you out.

Good luck.

-- 
Craig

[-- Attachment #1.2: Type: text/html, Size: 2554 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 2194 bytes --]

On Fri, May 24, 2013 at 6:27 PM, Craig Rodrigues <rodrigc@crodrigues.org>wrote:

> On Thu, May 23, 2013 at 6:30 AM, Roger Pau Monné <roger.pau@citrix.com
> >wrote:
>
> > On 23/05/13 15:20, Jeroen van der Ham wrote:
> > >
> > > On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com>
> wrote:
> > >> Also, I've created a wiki page that explains how to set up a FreeBSD
> > >> PVHVM for testing:
> > >>
> > >> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM
> > >
> > >
> > > You mention on that page that it is easier to install on 10.0-CURRENT
> > snapshots.
> > > What are the issues with installing this on 9.1? Is it possible?
> >
> > I don't think it is recommended to use a HEAD (10) kernel with a 9.1
> > userland. You can always install a 9.1 and then do a full update with
> > the source on my repository.
> >
>
>
> Actually in FreeBSD, it is possible to run an older userland on a newer
> kernel,
> and a lot of effort is spent in preserving this type of backwards
> compatibility.
> So a 9.1 userland with a 10 kernel will work.
> However, running a newer userland on an older kernel is not guaranteed to
> work.
> So running a 10 userland with a 9.1 kernel will most likely not work.
>
> However, since you guys are doing very cutting edge stuff with 10-CURRENT,
> it is better that you do not waste time with 9.1.
>
> I recommend you start with a 10.0 CURRENT snapshot ISO and go from there.
> I am going through a similar setup exercise with a Google Summer of Code
> student
> where he needs to have a latest CURRENT system running in a VM.
>
> I wrote this blog post:
>
>
> http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/
>
> for the steps how to do it.  You can follow those steps to get bootstrapped
> with a working environment if it helps you out.
>
> Good luck.
>
>
Any chance this will be backported to 9.X or 9-STABLE at least ??


> --
> Craig
> _______________________________________________
> freebsd-xen@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-xen
> To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org"
>

[-- Attachment #1.2: Type: text/html, Size: 3387 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Craig Rodrigues]

[-- Attachment #1.1: Type: text/plain, Size: 772 bytes --]

On Tue, May 28, 2013 at 8:21 AM, Outback Dingo <outbackdingo@gmail.com>wrote:

>
>
>
> On Fri, May 24, 2013 at 6:27 PM, Craig Rodrigues <rodrigc@crodrigues.org>wrote:
>
>> I wrote this blog post:
>>
>>
>> http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/
>>
>> for the steps how to do it.  You can follow those steps to get
>> bootstrapped
>> with a working environment if it helps you out.
>>
>> Good luck.
>>
>>
> Any chance this will be backported to 9.X or 9-STABLE at least ??
>
>


I wrote a blog post specifically for installing 10-CURRENT from a snapshot
ISO and getting bootstrapped
from there, to help the Google Summer of Code Student that I am mentoring.
What specifically do you want to be backported to 9-STABLE?

-- 
Craig

[-- Attachment #1.2: Type: text/html, Size: 1679 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 1498 bytes --]

On Tue, May 28, 2013 at 6:05 PM, Craig Rodrigues <rodrigc@crodrigues.org>wrote:

>
>
>
> On Tue, May 28, 2013 at 8:21 AM, Outback Dingo <outbackdingo@gmail.com>wrote:
>
>>
>>
>>
>> On Fri, May 24, 2013 at 6:27 PM, Craig Rodrigues <rodrigc@crodrigues.org>wrote:
>>
>>> I wrote this blog post:
>>>
>>>
>>> http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/
>>>
>>> for the steps how to do it.  You can follow those steps to get
>>> bootstrapped
>>> with a working environment if it helps you out.
>>>
>>> Good luck.
>>>
>>>
>> Any chance this will be backported to 9.X or 9-STABLE at least ??
>>
>>
>
>
> I wrote a blog post specifically for installing 10-CURRENT from a snapshot
> ISO and getting bootstrapped
> from there, to help the Google Summer of Code Student that I am mentoring.
> What specifically do you want to be backported to 9-STABLE?
>

Yes Ive seen the post, however we have a hard requirement for 9-STABLE for
some development work we are
currently involved in. And I would prefer not to move our current
infrastructure for development to CURRENT.
A major portion of our development environment is based on XEN for testing
and development.
Therefor we realize PVHVM is a work in progress, and we have built some
 VMs from it, they run exceptionally
well, However our build environments refuse to build 9-STABLE on CURRENT
PVHVM, or 10-CURRENT.
Lastly Im sure theres a ton of users who would like to see this in 9.x or
9-STABLE



>
>
> --
> Craig
>

[-- Attachment #1.2: Type: text/html, Size: 3270 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Craig Rodrigues]

[-- Attachment #1.1: Type: text/plain, Size: 1381 bytes --]

On Tue, May 28, 2013 at 3:43 PM, Outback Dingo <outbackdingo@gmail.com>wrote:

>
> I wrote a blog post specifically for installing 10-CURRENT from a snapshot
>> ISO and getting bootstrapped
>> from there, to help the Google Summer of Code Student that I am mentoring.
>> What specifically do you want to be backported to 9-STABLE?
>>
>
> Yes Ive seen the post, however we have a hard requirement for 9-STABLE for
> some development work we are
> currently involved in. And I would prefer not to move our current
> infrastructure for development to CURRENT.
> A major portion of our development environment is based on XEN for testing
> and development.
> Therefor we realize PVHM is a work in progress, and we have built some
>  VMs from it, they run exceptionally
> well, However our build environments refuse to build 9-STABLE on CURRENT
> PVHVM, or 10-CURRENT.
> Lastly Im sure theres a ton of users who would like to see this in 9.x or
> 9-STABLE
>
>

OK, I misunderstood you since you responded in the thread and quoted some
of my post.
My blog post has nothing relevant to Xen and PVHM, and is specific to
setting up
a Google Summer of Code student on a 10-CURRENT environment.  The FreeBSD
Xen
and PVHM developers will need to provide the details about their plans to
merge their changes to the 9-STABLE branch.....
that's something which I am not familiar with.

-- 
Craig

[-- Attachment #1.2: Type: text/html, Size: 2188 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 1630 bytes --]

On Tue, May 28, 2013 at 7:09 PM, Craig Rodrigues <rodrigc@crodrigues.org>wrote:

>
>
>
> On Tue, May 28, 2013 at 3:43 PM, Outback Dingo <outbackdingo@gmail.com>wrote:
>
>>
>>  I wrote a blog post specifically for installing 10-CURRENT from a
>>> snapshot ISO and getting bootstrapped
>>> from there, to help the Google Summer of Code Student that I am
>>> mentoring.
>>> What specifically do you want to be backported to 9-STABLE?
>>>
>>
>> Yes Ive seen the post, however we have a hard requirement for 9-STABLE
>> for some development work we are
>> currently involved in. And I would prefer not to move our current
>> infrastructure for development to CURRENT.
>> A major portion of our development environment is based on XEN for
>> testing and development.
>> Therefor we realize PVHM is a work in progress, and we have built some
>>  VMs from it, they run exceptionally
>> well, However our build environments refuse to build 9-STABLE on CURRENT
>> PVHVM, or 10-CURRENT.
>> Lastly Im sure theres a ton of users who would like to see this in 9.x or
>> 9-STABLE
>>
>>
>
> OK, I misunderstood you since you responded in the thread and quoted some
> of my post.
> My blog post has nothing relevant to Xen and PVHM, and is specific to
> setting up
> a Google Summer of Code student on a 10-CURRENT environment.  The FreeBSD
> Xen
> and PVHM developers will need to provide the details about their plans to
> merge their changes to the 9-STABLE branch.....
> that's something which I am not familiar with.
>

If we can get a diff of the work done against the master Id be happy to
help backporting and testing


>
>
> --
> Craig
>

[-- Attachment #1.2: Type: text/html, Size: 3176 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Justin T. Gibbs]

On May 28, 2013, at 5:20 PM, Outback Dingo <outbackdingo@gmail.com> wrote:

> On Tue, May 28, 2013 at 7:09 PM, Craig Rodrigues <rodrigc@crodrigues.org>wrote:
> 
>> On Tue, May 28, 2013 at 3:43 PM, Outback Dingo <outbackdingo@gmail.com>wrote:
>> 
>>> 
>>> I wrote a blog post specifically for installing 10-CURRENT from a
>>>> snapshot ISO and getting bootstrapped
>>>> from there, to help the Google Summer of Code Student that I am
>>>> mentoring.
>>>> What specifically do you want to be backported to 9-STABLE?
>>>> 
>>> 
>>> Yes Ive seen the post, however we have a hard requirement for 9-STABLE
>>> for some development work we are
>>> currently involved in. And I would prefer not to move our current
>>> infrastructure for development to CURRENT.
>>> A major portion of our development environment is based on XEN for
>>> testing and development.
>>> Therefor we realize PVHM is a work in progress, and we have built some
>>> VMs from it, they run exceptionally
>>> well, However our build environments refuse to build 9-STABLE on CURRENT
>>> PVHVM, or 10-CURRENT.
>>> Lastly Im sure theres a ton of users who would like to see this in 9.x or
>>> 9-STABLE
>>> 
>>> 
>> 
>> OK, I misunderstood you since you responded in the thread and quoted some
>> of my post.
>> My blog post has nothing relevant to Xen and PVHM, and is specific to
>> setting up
>> a Google Summer of Code student on a 10-CURRENT environment.  The FreeBSD
>> Xen
>> and PVHM developers will need to provide the details about their plans to
>> merge their changes to the 9-STABLE branch.....
>> that's something which I am not familiar with.
>> 
> 
> If we can get a diff of the work done against the master Id be happy to
> help backporting and testing

I plan to backport the work once we have it all in -current.

--
Justin

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 1022 bytes --]

On Thu, May 23, 2013 at 8:57 AM, Jeroen van der Ham <jeroen@dckd.nl> wrote:

> Hi,
>
> On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
> > Right now the code is in a state where it can be tested by users, so we
> > would like to encourage FreeBSD and Xen users to test it and provide
> > feedback.
>
> I've just been able to install it on a VPS using the latest pvhvm_v9
> branch.
> This is good news, because the system I had before actually had trouble
> with the HVM kernel from 9.1 [0].
>
> I'm going to leave this running for a while and do some more tests on it.
>
> Jeroen.
>

Curious if this would work under XEN XCP (Xen Cloud Platform)



>
> [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
>
> _______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe@freebsd.org"
>

[-- Attachment #1.2: Type: text/html, Size: 1963 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 23/05/13 14:57, Jeroen van der Ham wrote:
> Hi,
> 
> On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
>> Right now the code is in a state where it can be tested by users, so we
>> would like to encourage FreeBSD and Xen users to test it and provide
>> feedback.
> 
> I've just been able to install it on a VPS using the latest pvhvm_v9 branch.

The branch pvhvm_v9 contains an initial implementation of PV IPIs for
amd64. I've now finished it and I'm going to port it to i386 also, and
push a new branch to the repository.

> This is good news, because the system I had before actually had trouble with the HVM kernel from 9.1 [0].
> 
> I'm going to leave this running for a while and do some more tests on it.
> 
> Jeroen.
> 
> 
> [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
> 

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 1382 bytes --]

On Thu, May 23, 2013 at 9:33 AM, Roger Pau Monné <roger.pau@citrix.com>wrote:

> On 23/05/13 14:57, Jeroen van der Ham wrote:
> > Hi,
> >
> > On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
> >> Right now the code is in a state where it can be tested by users, so we
> >> would like to encourage FreeBSD and Xen users to test it and provide
> >> feedback.
> >
> > I've just been able to install it on a VPS using the latest pvhvm_v9
> branch.
>
> The branch pvhvm_v9 contains an initial implementation of PV IPIs for
> amd64. I've now finished it and I'm going to port it to i386 also, and
> push a new branch to the repository.
>


curious as the from what rev you guys forked your XENPVM work from HEAD, so
i can assure
Ive not lost some fixes, new commits from upstream


>
> > This is good news, because the system I had before actually had trouble
> with the HVM kernel from 9.1 [0].
> >
> > I'm going to leave this running for a while and do some more tests on it.
> >
> > Jeroen.
> >
> >
> > [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
> >
>
> _______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe@freebsd.org"
>

[-- Attachment #1.2: Type: text/html, Size: 2414 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 1989 bytes --]

On Thu, May 23, 2013 at 9:33 AM, Roger Pau Monné <roger.pau@citrix.com>wrote:

> On 23/05/13 14:57, Jeroen van der Ham wrote:
> > Hi,
> >
> > On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
> >> Right now the code is in a state where it can be tested by users, so we
> >> would like to encourage FreeBSD and Xen users to test it and provide
> >> feedback.
> >
> > I've just been able to install it on a VPS using the latest pvhvm_v9
> branch.
>
> The branch pvhvm_v9 contains an initial implementation of PV IPIs for
> amd64. I've now finished it and I'm going to port it to i386 also, and
> push a new branch to the repository.
>
> > This is good news, because the system I had before actually had trouble
> with the HVM kernel from 9.1 [0].
> >
> > I'm going to leave this running for a while and do some more tests on it.
> >
> > Jeroen.
> >
> >
> > [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
> >
>
> I built the rev_9 branch on a XCP host and rebooted, however I am seeing

on boot after ugen0.2: <QEMU 0.10.2> at usbus0

run_interrupt_driven_hooks: still waiting after 60 seconds for
xenbus_nop_confighook_cb
run_interrupt_driven_hooks: still waiting after 120 seconds for
xenbus_nop_confighook_cb
run_interrupt_driven_hooks: still waiting after 180 seconds for
xenbus_nop_confighook_cb
run_interrupt_driven_hooks: still waiting after 240 seconds for
xenbus_nop_confighook_cb
run_interrupt_driven_hooks: still waiting after 300 seconds for
xenbus_nop_confighook_cb
panic: run_interrupt_driven_confighooks: waited too long
cpuid = 0
KDB: enter: panic
[ thread pid 0 tid 100000 ]
Stropped at kdb_enter +0x3b: movq $0,0xad6522(%rip)
db>



_______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe@freebsd.org"
>

[-- Attachment #1.2: Type: text/html, Size: 3399 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Sergey Kandaurov]

On 23 May 2013 20:30, Outback Dingo <outbackdingo@gmail.com> wrote:
> On Thu, May 23, 2013 at 9:33 AM, Roger Pau Monné <roger.pau@citrix.com>wrote:
>
>> On 23/05/13 14:57, Jeroen van der Ham wrote:
>> > Hi,
>> >
>> > On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
>> >> Right now the code is in a state where it can be tested by users, so we
>> >> would like to encourage FreeBSD and Xen users to test it and provide
>> >> feedback.
>> >
>> > I've just been able to install it on a VPS using the latest pvhvm_v9
>> branch.
>>
>> The branch pvhvm_v9 contains an initial implementation of PV IPIs for
>> amd64. I've now finished it and I'm going to port it to i386 also, and
>> push a new branch to the repository.
>>
>> > This is good news, because the system I had before actually had trouble
>> with the HVM kernel from 9.1 [0].
>> >
>> > I'm going to leave this running for a while and do some more tests on it.
>> >
>> > Jeroen.
>> >
>> >
>> > [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
>> >
>>
>> I built the rev_9 branch on a XCP host and rebooted, however I am seeing
>
> on boot after ugen0.2: <QEMU 0.10.2> at usbus0
>
> run_interrupt_driven_hooks: still waiting after 60 seconds for
> xenbus_nop_confighook_cb
> run_interrupt_driven_hooks: still waiting after 120 seconds for
> xenbus_nop_confighook_cb
> run_interrupt_driven_hooks: still waiting after 180 seconds for
> xenbus_nop_confighook_cb
> run_interrupt_driven_hooks: still waiting after 240 seconds for
> xenbus_nop_confighook_cb
> run_interrupt_driven_hooks: still waiting after 300 seconds for
> xenbus_nop_confighook_cb
> panic: run_interrupt_driven_confighooks: waited too long
> cpuid = 0
> KDB: enter: panic
> [ thread pid 0 tid 100000 ]
> Stropped at kdb_enter +0x3b: movq $0,0xad6522(%rip)
> db>

Can you recheck this on stock HEAD?

>From your description this looks like a rather old bug seen with 8.2
or above and XCP (referenced in PR kern/164630). You can trigger it
when e.g. booting with empty cdrom.

-- 
wbr,
pluknet

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 23/05/13 18:30, Outback Dingo wrote:
> 
> 
> 
> On Thu, May 23, 2013 at 9:33 AM, Roger Pau Monné <roger.pau@citrix.com
> <mailto:roger.pau@citrix.com>> wrote:
> 
>     On 23/05/13 14:57, Jeroen van der Ham wrote:
>     > Hi,
>     >
>     > On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com
>     <mailto:roger.pau@citrix.com>> wrote:
>     >> Right now the code is in a state where it can be tested by users,
>     so we
>     >> would like to encourage FreeBSD and Xen users to test it and provide
>     >> feedback.
>     >
>     > I've just been able to install it on a VPS using the latest
>     pvhvm_v9 branch.
> 
>     The branch pvhvm_v9 contains an initial implementation of PV IPIs for
>     amd64. I've now finished it and I'm going to port it to i386 also, and
>     push a new branch to the repository.
> 
>     > This is good news, because the system I had before actually had
>     trouble with the HVM kernel from 9.1 [0].
>     >
>     > I'm going to leave this running for a while and do some more tests
>     on it.
>     >
>     > Jeroen.
>     >
>     >
>     > [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
>     >
> 
> I built the rev_9 branch on a XCP host and rebooted, however I am seeing
> 
> on boot after ugen0.2: <QEMU 0.10.2> at usbus0
> 
> run_interrupt_driven_hooks: still waiting after 60 seconds for
> xenbus_nop_confighook_cb
> run_interrupt_driven_hooks: still waiting after 120 seconds for
> xenbus_nop_confighook_cb
> run_interrupt_driven_hooks: still waiting after 180 seconds for
> xenbus_nop_confighook_cb
> run_interrupt_driven_hooks: still waiting after 240 seconds for
> xenbus_nop_confighook_cb
> run_interrupt_driven_hooks: still waiting after 300 seconds for
> xenbus_nop_confighook_cb
> panic: run_interrupt_driven_confighooks: waited too long
> cpuid = 0
> KDB: enter: panic
> [ thread pid 0 tid 100000 ]
> Stropped at kdb_enter +0x3b: movq $0,0xad6522(%rip)
> db>

>From what I've read on the list, it seems like you cannot boot the PVHVM
kernel if you have a cdrom attached to the guest, could you try
disabling the cdrom and booting again?

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 2514 bytes --]

On Thu, May 23, 2013 at 12:48 PM, Roger Pau Monné <roger.pau@citrix.com>wrote:

> On 23/05/13 18:30, Outback Dingo wrote:
> >
> >
> >
> > On Thu, May 23, 2013 at 9:33 AM, Roger Pau Monné <roger.pau@citrix.com
> > <mailto:roger.pau@citrix.com>> wrote:
> >
> >     On 23/05/13 14:57, Jeroen van der Ham wrote:
> >     > Hi,
> >     >
> >     > On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com
> >     <mailto:roger.pau@citrix.com>> wrote:
> >     >> Right now the code is in a state where it can be tested by users,
> >     so we
> >     >> would like to encourage FreeBSD and Xen users to test it and
> provide
> >     >> feedback.
> >     >
> >     > I've just been able to install it on a VPS using the latest
> >     pvhvm_v9 branch.
> >
> >     The branch pvhvm_v9 contains an initial implementation of PV IPIs for
> >     amd64. I've now finished it and I'm going to port it to i386 also,
> and
> >     push a new branch to the repository.
> >
> >     > This is good news, because the system I had before actually had
> >     trouble with the HVM kernel from 9.1 [0].
> >     >
> >     > I'm going to leave this running for a while and do some more tests
> >     on it.
> >     >
> >     > Jeroen.
> >     >
> >     >
> >     > [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
> >     >
> >
> > I built the rev_9 branch on a XCP host and rebooted, however I am seeing
> >
> > on boot after ugen0.2: <QEMU 0.10.2> at usbus0
> >
> > run_interrupt_driven_hooks: still waiting after 60 seconds for
> > xenbus_nop_confighook_cb
> > run_interrupt_driven_hooks: still waiting after 120 seconds for
> > xenbus_nop_confighook_cb
> > run_interrupt_driven_hooks: still waiting after 180 seconds for
> > xenbus_nop_confighook_cb
> > run_interrupt_driven_hooks: still waiting after 240 seconds for
> > xenbus_nop_confighook_cb
> > run_interrupt_driven_hooks: still waiting after 300 seconds for
> > xenbus_nop_confighook_cb
> > panic: run_interrupt_driven_confighooks: waited too long
> > cpuid = 0
> > KDB: enter: panic
> > [ thread pid 0 tid 100000 ]
> > Stropped at kdb_enter +0x3b: movq $0,0xad6522(%rip)
> > db>
>
> From what I've read on the list, it seems like you cannot boot the PVHVM
> kernel if you have a cdrom attached to the guest, could you try
> disabling the cdrom and booting again?
>
>
great how does one go about disabling the cdrom, i get some disk parameters
needs to be removed from the vm template before boot

[-- Attachment #1.2: Type: text/html, Size: 3566 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

Just remove this line (or pointing to a similar file from the template: (It's part of the disks definition:

> 	'file:/root/freebsd-10.iso,hdc:cdrom,r',

Jeroen.

On 23 May 2013, at 19:02, Outback Dingo <outbackdingo@gmail.com> wrote:

> On Thu, May 23, 2013 at 12:48 PM, Roger Pau Monné <roger.pau@citrix.com>wrote:
> 
>> On 23/05/13 18:30, Outback Dingo wrote:
>>> 
>>> 
>>> 
>>> On Thu, May 23, 2013 at 9:33 AM, Roger Pau Monné <roger.pau@citrix.com
>>> <mailto:roger.pau@citrix.com>> wrote:
>>> 
>>>    On 23/05/13 14:57, Jeroen van der Ham wrote:
>>>> Hi,
>>>> 
>>>> On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com
>>>    <mailto:roger.pau@citrix.com>> wrote:
>>>>> Right now the code is in a state where it can be tested by users,
>>>    so we
>>>>> would like to encourage FreeBSD and Xen users to test it and
>> provide
>>>>> feedback.
>>>> 
>>>> I've just been able to install it on a VPS using the latest
>>>    pvhvm_v9 branch.
>>> 
>>>    The branch pvhvm_v9 contains an initial implementation of PV IPIs for
>>>    amd64. I've now finished it and I'm going to port it to i386 also,
>> and
>>>    push a new branch to the repository.
>>> 
>>>> This is good news, because the system I had before actually had
>>>    trouble with the HVM kernel from 9.1 [0].
>>>> 
>>>> I'm going to leave this running for a while and do some more tests
>>>    on it.
>>>> 
>>>> Jeroen.
>>>> 
>>>> 
>>>> [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
>>>> 
>>> 
>>> I built the rev_9 branch on a XCP host and rebooted, however I am seeing
>>> 
>>> on boot after ugen0.2: <QEMU 0.10.2> at usbus0
>>> 
>>> run_interrupt_driven_hooks: still waiting after 60 seconds for
>>> xenbus_nop_confighook_cb
>>> run_interrupt_driven_hooks: still waiting after 120 seconds for
>>> xenbus_nop_confighook_cb
>>> run_interrupt_driven_hooks: still waiting after 180 seconds for
>>> xenbus_nop_confighook_cb
>>> run_interrupt_driven_hooks: still waiting after 240 seconds for
>>> xenbus_nop_confighook_cb
>>> run_interrupt_driven_hooks: still waiting after 300 seconds for
>>> xenbus_nop_confighook_cb
>>> panic: run_interrupt_driven_confighooks: waited too long
>>> cpuid = 0
>>> KDB: enter: panic
>>> [ thread pid 0 tid 100000 ]
>>> Stropped at kdb_enter +0x3b: movq $0,0xad6522(%rip)
>>> db>
>> 
>> From what I've read on the list, it seems like you cannot boot the PVHVM
>> kernel if you have a cdrom attached to the guest, could you try
>> disabling the cdrom and booting again?
>> 
>> 
> great how does one go about disabling the cdrom, i get some disk parameters
> needs to be removed from the vm template before boot

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 3074 bytes --]

On Thu, May 23, 2013 at 1:22 PM, Jeroen van der Ham <jeroen@dckd.nl> wrote:

> Hi,
>
> Just remove this line (or pointing to a similar file from the template:
> (It's part of the disks definition:
>
> >       'file:/root/freebsd-10.iso,hdc:cdrom,r',
>
>
Thanks, but this is XCP not a generic XEN server where there are vm config
files under /etc/xen/ in XCP they dont exists


> Jeroen.
>
> On 23 May 2013, at 19:02, Outback Dingo <outbackdingo@gmail.com> wrote:
>
> > On Thu, May 23, 2013 at 12:48 PM, Roger Pau Monné <roger.pau@citrix.com
> >wrote:
> >
> >> On 23/05/13 18:30, Outback Dingo wrote:
> >>>
> >>>
> >>>
> >>> On Thu, May 23, 2013 at 9:33 AM, Roger Pau Monné <roger.pau@citrix.com
> >>> <mailto:roger.pau@citrix.com>> wrote:
> >>>
> >>>    On 23/05/13 14:57, Jeroen van der Ham wrote:
> >>>> Hi,
> >>>>
> >>>> On 13 May 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com
> >>>    <mailto:roger.pau@citrix.com>> wrote:
> >>>>> Right now the code is in a state where it can be tested by users,
> >>>    so we
> >>>>> would like to encourage FreeBSD and Xen users to test it and
> >> provide
> >>>>> feedback.
> >>>>
> >>>> I've just been able to install it on a VPS using the latest
> >>>    pvhvm_v9 branch.
> >>>
> >>>    The branch pvhvm_v9 contains an initial implementation of PV IPIs
> for
> >>>    amd64. I've now finished it and I'm going to port it to i386 also,
> >> and
> >>>    push a new branch to the repository.
> >>>
> >>>> This is good news, because the system I had before actually had
> >>>    trouble with the HVM kernel from 9.1 [0].
> >>>>
> >>>> I'm going to leave this running for a while and do some more tests
> >>>    on it.
> >>>>
> >>>> Jeroen.
> >>>>
> >>>>
> >>>> [0]: http://www.freebsd.org/cgi/query-pr.cgi?pr=175822
> >>>>
> >>>
> >>> I built the rev_9 branch on a XCP host and rebooted, however I am
> seeing
> >>>
> >>> on boot after ugen0.2: <QEMU 0.10.2> at usbus0
> >>>
> >>> run_interrupt_driven_hooks: still waiting after 60 seconds for
> >>> xenbus_nop_confighook_cb
> >>> run_interrupt_driven_hooks: still waiting after 120 seconds for
> >>> xenbus_nop_confighook_cb
> >>> run_interrupt_driven_hooks: still waiting after 180 seconds for
> >>> xenbus_nop_confighook_cb
> >>> run_interrupt_driven_hooks: still waiting after 240 seconds for
> >>> xenbus_nop_confighook_cb
> >>> run_interrupt_driven_hooks: still waiting after 300 seconds for
> >>> xenbus_nop_confighook_cb
> >>> panic: run_interrupt_driven_confighooks: waited too long
> >>> cpuid = 0
> >>> KDB: enter: panic
> >>> [ thread pid 0 tid 100000 ]
> >>> Stropped at kdb_enter +0x3b: movq $0,0xad6522(%rip)
> >>> db>
> >>
> >> From what I've read on the list, it seems like you cannot boot the PVHVM
> >> kernel if you have a cdrom attached to the guest, could you try
> >> disabling the cdrom and booting again?
> >>
> >>
> > great how does one go about disabling the cdrom, i get some disk
> parameters
> > needs to be removed from the vm template before boot
>
>

[-- Attachment #1.2: Type: text/html, Size: 4753 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

Hello,

I've pushed a new branch, pvhvm_v10 that contains a PV IPI
implementation for both amd64 and i386. I've also updated the wiki to
point to the pvhvm_v10 branch:

http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v10

I've updated my tree to latest HEAD, so now branch pvhvm_v10 is on top
of this commit:

commit b44da0fb82647f2cfb06f65a6695c7e36c98828c
Author: gber <gber@FreeBSD.org>
Date:   Thu May 23 12:24:46 2013 +0000

    Rework and organize pmap_enter_locked() function.

    pmap_enter_locked() implementation was very ambiguous and confusing.
    Rearrange it so that each part of the mapping creation is separated.
    Avoid walking through the redundant conditions.
    Extract vector_page specific PTE setup from normal PTE setting.

    Submitted by:   Zbigniew Bodek <zbb@semihalf.com>
    Sponsored by:   The FreeBSD Foundation, Semihalf

Thanks for the testing, Roger.

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 2397 bytes --]

On Mon, May 13, 2013 at 2:32 PM, Roger Pau Monné <roger.pau@citrix.com>wrote:

> Hello,
>
> Recently Justin T Gibbs, Will Andrews and myself have been working on
> improving the Xen support in FreeBSD. The main goal of this was to bring
> full PVHVM support to FreeBSD, right now FreeBSD is only using PV
> interfaces for disk and network interfaces when running as a HVM guest.
> The main benefits of this changes are that Xen virtual interrupts (event
> channels) are now delivered to the guest using a vector callback
> injection, that is a per-cpu mechanism that allows each vCPU to have
> different interrupts assigned, so for example network and disk
> interrupts are delivered to different vCPUs in order to improve
> performance. With this changes FreeBSD also uses PV timers when running
> as an HVM guest, which should provide better time keeping and reduce the
> virtualization overhead, since emulated timers are no longer used. PV
> IPIs can also be used inside a HVM guest, but this will be implemented
> later.
>
> Right now the code is in a state where it can be tested by users, so we
> would like to encourage FreeBSD and Xen users to test it and provide
> feedback.
>
> The code is available in the following git repository, under the branch
> pvhvm_v5:
>
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary
>
> Also, I've created a wiki page that explains how to set up a FreeBSD
> PVHVM for testing:
>
> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM



Hey Guys,

Huge KUDOS to some great work, Ive gotten 4 VMs booting from zfsonroot with
the rev_9 finally under XCP 1.6
after getting through removal of the cdrom device from VMs, they are booted
and running supremely nice now.
IO for zfs filesystems is much better inside the VMs now, even with 4mb
compared to what it was. This is a huge
move forward and personally I just wanted to say thanks for this and all
the work you put into it. The VMs appear quite stable for me, and are
pleasantly responsive with low mem on zpools. Great job and again.....
Thanks. Keep up the great effort.


>
> _______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe@freebsd.org"
>

[-- Attachment #1.2: Type: text/html, Size: 3417 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Yuriy Kohut]

[-- Attachment #1.1: Type: text/plain, Size: 12281 bytes --]

Hi,

I've just successfully run FreeBSD 9.1 based guest with 'pvhvm_v8' based kernel under Xen 4.2.2.

So I couldn't confirm any issue with the kernel both on Xen 3.4.4 and 4.2.2.
Nice Job.

Hypervisor details:
# xm info
host                   : ********
release                : 3.8.7-1.el6xen
version                : #1 SMP Tue Apr 16 13:14:14 EEST 2013
machine                : x86_64
nr_cpus                : 4
nr_nodes               : 1
cores_per_socket       : 4
threads_per_core       : 1
cpu_mhz                : 1995
hw_caps                : bfebfbff:28100800:00000000:00003b40:009ce3bd:00000000:00000001:00000000
virt_caps              : hvm
total_memory           : 16374
free_memory            : 7194
free_cpus              : 0
xen_major              : 4
xen_minor              : 2
xen_extra              : .2
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
xen_commandline        : dom0_mem=409600
cc_compiler            : gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-3)
cc_compile_by          : mockbuild
cc_compile_domain      : ********
cc_compile_date        : Tue May  7 19:26:49 EST 2013
xend_config_format     : 4


DomU details:
# xm list --long f2tbrxodmgk9ng
(domain
    (domid 146)
    (cpu_weight 400)
    (cpu_cap 0)
    (pool_name Pool-0)
    (bootloader '')
    (vcpus 4)
    (cpus (() () () ()))
    (on_poweroff destroy)
    (description '')
    (on_crash restart)
    (uuid 355cfb26-4793-f85a-e330-7b6bfcae49b5)
    (bootloader_args '')
    (name f2tbrxodmgk9ng)
    (on_reboot restart)
    (maxmem 1024)
    (memory 1024)
    (shadow_memory 12)
    (features '')
    (on_xend_start ignore)
    (on_xend_stop ignore)
    (start_time 1369384081.34)
    (cpu_time 59.956655605)
    (online_vcpus 4)
    (image
        (hvm
            (kernel '')
            (superpages 0)
            (videoram 4)
            (hpet 0)
            (stdvga 0)
            (loader /usr/lib/xen/boot/hvmloader)
            (xen_platform_pci 1)
            (nestedhvm 0)
            (rtc_timeoffset 10801)
            (pci ())
            (hap 1)
            (localtime 0)
            (timer_mode 1)
            (pci_msitranslate 1)
            (oos 1)
            (apic 1)
            (usbdevice tablet)
            (vpt_align 1)
            (vncunused 1)
            (boot cd)
            (pae 1)
            (viridian 0)
            (acpi 1)
            (vnc 1)
            (nographic 0)
            (nomigrate 0)
            (usb 0)
            (tsc_mode 0)
            (guest_os_type default)
            (device_model /usr/lib64/xen/bin/qemu-dm)
            (pci_power_mgmt 0)
            (xauthority /root/.Xauthority)
            (isa 0)
            (notes (SUSPEND_CANCEL 1))
        )
    )
    (status 2)
    (state -b----)
    (store_mfn 1044476)
    (device
        (vif
            (bridge nnl9l2z5l3q3d8)
            (uuid daeabd68-05a0-f25b-ba65-394627505b50)
            (script /etc/xen/scripts/vif-bridge)
            (ip 109.123.91.166)
            (mac 00:16:3e:e8:88:49)
            (vifname qgdvmt5h6d2l9s)
            (backend 0)
        )
    )
    (device
        (console
            (protocol vt100)
            (location 7)
            (uuid c670a71d-4c3b-1fcb-974a-587f17740a6c)
        )
    )
    (device
        (vbd
            (protocol x86_64-abi)
            (uuid 9067929c-9b48-99c6-5526-e771d43f427c)
            (bootable 1)
            (dev hda:disk)
            (uname phy:/dev/fv4zl7t2h5wbeq/o76ciuubu0r986)
            (mode w)
            (backend 0)
            (VDI '')
        )
    )
    (device
        (vbd
            (protocol x86_64-abi)
            (uuid 2ae3630e-0e8e-04e4-8caf-ac4d1e9fd402)
            (bootable 0)
            (dev hdb:disk)
            (uname phy:/dev/fv4zl7t2h5wbeq/xi0nw7u4zo0bu9)
            (mode w)
            (backend 0)
            (VDI '')
        )
    )
    (device
        (vbd
            (protocol x86_64-abi)
            (uuid 33d3e25c-0c1c-ced6-c8b6-5a706ab0d403)
            (bootable 0)
            (dev hdc:cdrom)
            (uname file:/tools/freebsd/boot-freebsd-generic.iso)
            (mode r)
            (backend 0)
            (VDI '')
        )
    )
    (device
        (vfb
            (vncunused 1)
            (vnc 1)
            (uuid 438a2ffd-bec7-1e54-bb0b-4fdd400517cf)
            (location 0.0.0.0:5908)
        )
    )
)



DomU from "inside":
# uname -a
FreeBSD yurak2.vm 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r+03cdadc: Thu May 23 18:55:33 AST 2013     root@yurak2.vm:/usr/obj/freebsd/sys/XENHVM  amd64
---
Yura

On May 22, 2013, at 18:27 PM, Yuriy Kohut <ykohut@onapp.com> wrote:

> Hi,
> 
> I've just successfully run FreeBSD 9.1 based guest with 'pvhvm_v8' based kernel under Xen 3.4.4.
> 
> Hypervisor details:
> # xm info
> host                   : *******
> release                : 2.6.18-348.2.1.el5xen
> version                : #1 SMP Tue Mar 5 17:05:33 EST 2013
> machine                : x86_64
> nr_cpus                : 4
> nr_nodes               : 1
> cores_per_socket       : 4
> threads_per_core       : 1
> cpu_mhz                : 2128
> hw_caps                : bfebfbff:28100800:00000000:00000340:009ce3bd:00000000:00000001:00000000
> virt_caps              : hvm
> total_memory           : 6135
> free_memory            : 262
> node_to_cpu            : node0:0-3
> node_to_memory         : node0:262
> xen_major              : 3
> xen_minor              : 4
> xen_extra              : .4
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64 
> xen_scheduler          : credit
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : unavailable
> cc_compiler            : gcc version 4.1.2 20080704 (Red Hat 4.1.2-52)
> cc_compile_by          : root
> cc_compile_domain      : ******
> cc_compile_date        : Wed Sep  5 18:01:10 EEST 2012
> xend_config_format     : 4
> 
> 
> DomU details:
> # xm list --long h283bpm53f9rnx
> (domain
>     (domid 61)
>     (on_crash restart)
>     (uuid a2cbcba9-1d66-87ce-6d2f-412e70eab051)
>     (bootloader_args )
>     (vcpus 2)
>     (name h283bpm53f9rnx)
>     (on_poweroff destroy)
>     (on_reboot restart)
>     (cpus (() ()))
>     (bootloader )
>     (maxmem 1024)
>     (memory 1024)
>     (shadow_memory 10)
>     (features )
>     (on_xend_start ignore)
>     (on_xend_stop ignore)
>     (start_time 1369235607.92)
>     (cpu_time 31.914003553)
>     (online_vcpus 2)
>     (image
>         (hvm
>             (kernel )
>             (videoram 4)
>             (hpet 0)
>             (stdvga 0)
>             (loader /usr/lib/xen/boot/hvmloader)
>             (vncunused 1)
>             (xen_platform_pci 1)
>             (boot cd)
>             (rtc_timeoffset 7202)
>             (pci ())
>             (pae 1)
>             (vpt_align 1)
>             (hap 1)
>             (viridian 0)
>             (acpi 1)
>             (localtime 0)
>             (timer_mode 1)
>             (vnc 1)
>             (nographic 0)
>             (guest_os_type default)
>             (pci_msitranslate 1)
>             (apic 1)
>             (monitor 0)
>             (usbdevice tablet)
>             (device_model /usr/lib64/xen/bin/qemu-dm)
>             (pci_power_mgmt 0)
>             (usb 0)
>             (xauthority /root/.Xauthority)
>             (isa 0)
>             (notes (SUSPEND_CANCEL 1))
>         )
>     )
>     (status 2)
>     (state -b----)
>     (store_mfn 1044476)
>     (device
>         (vif
>             (bridge xnh5getjoj54ke)
>             (uuid 6133c146-48ea-b7a5-0263-fda98e1a30fe)
>             (script /etc/xen/scripts/vif-bridge)
>             (ip 83.170.81.183)
>             (mac 00:16:3e:a4:02:5a)
>             (vifname t2vd5w22msrv5d)
>             (backend 0)
>         )
>     )
>     (device
>         (vbd
>             (protocol x86_64-abi)
>             (uuid dd857cd1-2a4c-ea21-a5a5-e95d811f607a)
>             (bootable 1)
>             (dev hda:disk)
>             (uname phy:/dev/9yblt1m70pdtdp/ddfhogyred6bby)
>             (mode w)
>             (backend 0)
>             (bootable 1)
>             (VDI )
>         )
>     )
>     (device
>         (vbd
>             (protocol x86_64-abi)
>             (uuid 19cae15c-354d-77cb-57ec-dc313f1d05ba)
>             (bootable 0)
>             (dev hdb:disk)
>             (uname phy:/dev/9yblt1m70pdtdp/dhnnwhs6jh9kdd)
>             (mode w)
>             (backend 0)
>             (bootable 0)
>             (VDI )
>         )
>     )
>     (device
>         (vbd
>             (protocol x86_64-abi)
>             (uuid 2b97ec8c-0cc5-7197-f510-63c272449680)
>             (bootable 0)
>             (dev hdc:disk)
>             (uname phy:/dev/9yblt1m70pdtdp/d1jilc7s7jxsaq)
>             (mode w)
>             (backend 0)
>             (bootable 0)
>             (VDI )
>         )
>     )
>     (device
>         (vbd
>             (protocol x86_64-abi)
>             (uuid 1b472270-1ef3-2f49-81f1-031cc00c0eb7)
>             (bootable 0)
>             (dev hdd:cdrom)
>             (uname file:/tools/freebsd/boot-freebsd-generic.iso)
>             (mode r)
>             (backend 0)
>             (bootable 0)
>             (VDI )
>         )
>     )
>     (device
>         (vfb
>             (vncunused 1)
>             (vnc 1)
>             (uuid b3defeea-4acc-1408-9b22-71547a64e705)
>             (location 0.0.0.0:5900)
>         )
>     )
>     (device
>         (console
>             (protocol vt100)
>             (location 4)
>             (uuid 58a089ce-a4d0-037e-23e8-9df37b2bd5da)
>         )
>     )
> )
> 
> 
> DomU from "inside":
> # uname -a
> FreeBSD yurak1.vm 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r+03cdadc: Wed May 22 17:47:40 EEST 2013     root@yurak1.vm:/usr/obj/data/freebsd/sys/XENHVM  amd64
> 
> 
> I'll also set up one (hope will have some time) under Xen 4.2.2 tomorrow.
> ---
> Yura
> 
> On May 13, 2013, at 21:32 PM, Roger Pau Monné <roger.pau@citrix.com> wrote:
> 
>> Hello,
>> 
>> Recently Justin T Gibbs, Will Andrews and myself have been working on
>> improving the Xen support in FreeBSD. The main goal of this was to bring
>> full PVHVM support to FreeBSD, right now FreeBSD is only using PV
>> interfaces for disk and network interfaces when running as a HVM guest.
>> The main benefits of this changes are that Xen virtual interrupts (event
>> channels) are now delivered to the guest using a vector callback
>> injection, that is a per-cpu mechanism that allows each vCPU to have
>> different interrupts assigned, so for example network and disk
>> interrupts are delivered to different vCPUs in order to improve
>> performance. With this changes FreeBSD also uses PV timers when running
>> as an HVM guest, which should provide better time keeping and reduce the
>> virtualization overhead, since emulated timers are no longer used. PV
>> IPIs can also be used inside a HVM guest, but this will be implemented
>> later.
>> 
>> Right now the code is in a state where it can be tested by users, so we
>> would like to encourage FreeBSD and Xen users to test it and provide
>> feedback.
>> 
>> The code is available in the following git repository, under the branch
>> pvhvm_v5:
>> 
>> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary
>> 
>> Also, I've created a wiki page that explains how to set up a FreeBSD
>> PVHVM for testing:
>> 
>> http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM
>> _______________________________________________
>> freebsd-xen@freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-xen
>> To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org"
> 


[-- Attachment #1.2: Type: text/html, Size: 24196 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Konrad Rzeszutek Wilk]

On Thu, May 23, 2013 at 07:41:50PM +0200, Roger Pau Monné wrote:
> Hello,
> 
> I've pushed a new branch, pvhvm_v10 that contains a PV IPI
> implementation for both amd64 and i386. I've also updated the wiki to
> point to the pvhvm_v10 branch:

I feel a bit stupid to ask this, but how I install 'gmake'? Doing 'pkg_add -r gmake'
tells me there is no package (perhaps I am using a too modern version of FreeBSD
(FreeBSD-10.0-CURRENT-amd64-20130512-r250582-release.iso)?

The Wiki mentions how to install git but that fails b/c it can't find gmake.

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 1020 bytes --]

On Fri, May 24, 2013 at 10:14 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:

> On Thu, May 23, 2013 at 07:41:50PM +0200, Roger Pau Monné wrote:
> > Hello,
> >
> > I've pushed a new branch, pvhvm_v10 that contains a PV IPI
> > implementation for both amd64 and i386. I've also updated the wiki to
> > point to the pvhvm_v10 branch:
>
> I feel a bit stupid to ask this, but how I install 'gmake'? Doing 'pkg_add
> -r gmake'
> tells me there is no package (perhaps I am using a too modern version of
> FreeBSD
> (FreeBSD-10.0-CURRENT-amd64-20130512-r250582-release.iso)?
>
> The Wiki mentions how to install git but that fails b/c it can't find
> gmake.
>

on 10-CURRENT you need to use pkg or build from ports/devel/gmake


> _______________________________________________
> freebsd-virtualization@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "
> freebsd-virtualization-unsubscribe@freebsd.org"
>

[-- Attachment #1.2: Type: text/html, Size: 1841 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 24/05/13 16:14, Konrad Rzeszutek Wilk wrote:
> On Thu, May 23, 2013 at 07:41:50PM +0200, Roger Pau Monné wrote:
>> Hello,
>>
>> I've pushed a new branch, pvhvm_v10 that contains a PV IPI
>> implementation for both amd64 and i386. I've also updated the wiki to
>> point to the pvhvm_v10 branch:
> 
> I feel a bit stupid to ask this, but how I install 'gmake'? Doing 'pkg_add -r gmake'
> tells me there is no package (perhaps I am using a too modern version of FreeBSD
> (FreeBSD-10.0-CURRENT-amd64-20130512-r250582-release.iso)?
> 
> The Wiki mentions how to install git but that fails b/c it can't find gmake.

Did you install the ports tree during the installation? If so I've
always successfully installed git using:

# whereis git
# cd <output of above command>
# make install

Maybe the ISO you picked as a broken ports snapshot?

Re: FreeBSD PVHVM call for testing

[Craig Rodrigues]

[-- Attachment #1.1: Type: text/plain, Size: 1470 bytes --]

On Fri, May 24, 2013 at 7:14 AM, Konrad Rzeszutek Wilk <
konrad.wilk@oracle.com> wrote:

> On Thu, May 23, 2013 at 07:41:50PM +0200, Roger Pau Monné wrote:
> > Hello,
> >
> > I've pushed a new branch, pvhvm_v10 that contains a PV IPI
> > implementation for both amd64 and i386. I've also updated the wiki to
> > point to the pvhvm_v10 branch:
>
> I feel a bit stupid to ask this, but how I install 'gmake'? Doing 'pkg_add
> -r gmake'
> tells me there is no package (perhaps I am using a too modern version of
> FreeBSD
> (FreeBSD-10.0-CURRENT-amd64-20130512-r250582-release.iso)?
>
> The Wiki mentions how to install git but that fails b/c it can't find
> gmake.
>

For 10.0-CURRENT, not all the packages are available yet from the main
FreeBSD.org ftp site.
I am going through a similar setup issue with someone who has signed up for
Google Summer of Code,
and I need him to use close to the latest 10.0-CURRENT and have a usable
system.

I wrote this blog post for the student:
http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/

You may wish to follow those steps to configure your system to download
packages from
one of the 3rd party pkg mirrors, and then use the "pkg" command to install
gmake and whatever other packages you need.
Outback Dingo has mentioned using "pkg" command for installing packages in
10-CURRENT, and that is
what I am instructing my GSoC to do.

Good luck.
-- 
Craig

[-- Attachment #1.2: Type: text/html, Size: 2100 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Konrad Rzeszutek Wilk]

On Fri, May 24, 2013 at 03:21:54PM -0700, Craig Rodrigues wrote:
> On Fri, May 24, 2013 at 7:14 AM, Konrad Rzeszutek Wilk <
> konrad.wilk@oracle.com> wrote:
> 
> > On Thu, May 23, 2013 at 07:41:50PM +0200, Roger Pau Monné wrote:
> > > Hello,
> > >
> > > I've pushed a new branch, pvhvm_v10 that contains a PV IPI
> > > implementation for both amd64 and i386. I've also updated the wiki to
> > > point to the pvhvm_v10 branch:
> >
> > I feel a bit stupid to ask this, but how I install 'gmake'? Doing 'pkg_add
> > -r gmake'
> > tells me there is no package (perhaps I am using a too modern version of
> > FreeBSD
> > (FreeBSD-10.0-CURRENT-amd64-20130512-r250582-release.iso)?
> >
> > The Wiki mentions how to install git but that fails b/c it can't find
> > gmake.
> >
> 
> For 10.0-CURRENT, not all the packages are available yet from the main
> FreeBSD.org ftp site.
> I am going through a similar setup issue with someone who has signed up for
> Google Summer of Code,
> and I need him to use close to the latest 10.0-CURRENT and have a usable
> system.
> 
> I wrote this blog post for the student:
> http://blogs.freebsdish.org/rodrigc/2013/05/24/setting-up-a-vm-for-doing-gsoc-work/

Excellent! Thanks for the link.

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 23 May 2013, at 19:41, Roger Pau Monné <roger.pau@citrix.com> wrote:

> Hello,
> 
> I've pushed a new branch, pvhvm_v10 that contains a PV IPI
> implementation for both amd64 and i386. I've also updated the wiki to
> point to the pvhvm_v10 branch:

I've been running a VM with this kernel for about a week now. It ran fine, until about 3:30 in the morning. The only thing I can see is the following cryptic messages in /var/log/messages, followed by a reboot of the system.

May 29 23:42:30 image01 sshd[31227]: error: Received disconnect from 150.165.15.175: 11: Bye Bye [preauth]
May 30 03:30:57 image01 kernel: .
May 30 03:30:57 image01 ntpd[4436]: ntpd exiting on signal 15
May 30 03:30:57 image01 kernel: .
May 30 03:30:58 image01 kernel: .
May 30 03:31:00 image01 syslogd: exiting on signal 15
May 30 03:32:52 image01 syslogd: kernel boot file is /boot/kernel/kernel
May 30 03:32:52 image01 kernel: Copyright (c) 1992-2013 The FreeBSD Project.
May 30 03:32:52 image01 kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
May 30 03:32:52 image01 kernel: The Regents of the University of California. All rights reserved.
May 30 03:32:52 image01 kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.

I'm happy to help to gather more information, just tell me what you need.

Jeroen.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 30/05/13 10:50, Jeroen van der Ham wrote:
> Hi,
> 
> On 23 May 2013, at 19:41, Roger Pau Monné <roger.pau@citrix.com> wrote:
> 
>> Hello,
>>
>> I've pushed a new branch, pvhvm_v10 that contains a PV IPI
>> implementation for both amd64 and i386. I've also updated the wiki to
>> point to the pvhvm_v10 branch:
> 
> I've been running a VM with this kernel for about a week now. It ran fine, until about 3:30 in the morning. The only thing I can see is the following cryptic messages in /var/log/messages, followed by a reboot of the system.
> 
> May 29 23:42:30 image01 sshd[31227]: error: Received disconnect from 150.165.15.175: 11: Bye Bye [preauth]
> May 30 03:30:57 image01 kernel: .
> May 30 03:30:57 image01 ntpd[4436]: ntpd exiting on signal 15
> May 30 03:30:57 image01 kernel: .
> May 30 03:30:58 image01 kernel: .
> May 30 03:31:00 image01 syslogd: exiting on signal 15
> May 30 03:32:52 image01 syslogd: kernel boot file is /boot/kernel/kernel
> May 30 03:32:52 image01 kernel: Copyright (c) 1992-2013 The FreeBSD Project.
> May 30 03:32:52 image01 kernel: Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> May 30 03:32:52 image01 kernel: The Regents of the University of California. All rights reserved.
> May 30 03:32:52 image01 kernel: FreeBSD is a registered trademark of The FreeBSD Foundation.
> 
> I'm happy to help to gather more information, just tell me what you need.

Hello Jeroen,

So it looks like the system rebooted (but it was not a crash or a
sporadic reboot? the kernel seems to be aware of the reboot request). It
would be interesting if you could provide the output of the serial
console when this happens, that might be helpful. Did you enable
xenconsoled logging?

Also, could you provide more info about your system, Xen version, what
workload was the DomU running, Dom0 kernel version?

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 30 May 2013, at 11:04, Roger Pau Monné <roger.pau@citrix.com> wrote:
> So it looks like the system rebooted (but it was not a crash or a
> sporadic reboot? the kernel seems to be aware of the reboot request). It
> would be interesting if you could provide the output of the serial
> console when this happens, that might be helpful. Did you enable
> xenconsoled logging?

Unfortunately I did not.

> Also, could you provide more info about your system, Xen version, what
> workload was the DomU running, Dom0 kernel version?

There was no one logged in at the time of the reboot according to the last log.
I did do some sysbench tests during the day, but that was way before it rebooted. The only thing that could be running during that time was daily periodic.

$ sudo xm info
host                   : soleus01.soleus.nu
release                : 2.6.32-5-xen-amd64
version                : #1 SMP Mon Oct 3 07:53:54 UTC 2011
machine                : x86_64
nr_cpus                : 8
nr_nodes               : 2
cores_per_socket       : 4
threads_per_core       : 1
cpu_mhz                : 2200
hw_caps                : 178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000037ff:00000000
virt_caps              : hvm
total_memory           : 65534
free_memory            : 6866
node_to_cpu            : node0:0-3
                         node1:4-7
node_to_memory         : node0:3128
                         node1:3737
node_to_dma32_mem      : node0:3128
                         node1:0
max_node_id            : 1
xen_major              : 4
xen_minor              : 0
xen_extra              : .1
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
xen_commandline        : placeholder dom0_mem=1852M
cc_compiler            : gcc version 4.4.5 (Debian 4.4.5-10)
cc_compile_by          : waldi
cc_compile_domain      : debian.org
cc_compile_date        : Wed Jan 12 14:04:06 UTC 2011
xend_config_format     : 4

$ uname -a
Linux soleus01.soleus.nu 2.6.32-5-xen-amd64 #1 SMP Mon Oct 3 07:53:54 UTC 2011 x86_64 GNU/Linux

Jeroen.

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 3049 bytes --]

On Thu, May 30, 2013 at 5:15 AM, Jeroen van der Ham <jeroen@dckd.nl> wrote:

> Hi,
>
> On 30 May 2013, at 11:04, Roger Pau Monné <roger.pau@citrix.com> wrote:
> > So it looks like the system rebooted (but it was not a crash or a
> > sporadic reboot? the kernel seems to be aware of the reboot request). It
> > would be interesting if you could provide the output of the serial
> > console when this happens, that might be helpful. Did you enable
> > xenconsoled logging?
>
> Unfortunately I did not.
>
> > Also, could you provide more info about your system, Xen version, what
> > workload was the DomU running, Dom0 kernel version?
>
> There was no one logged in at the time of the reboot according to the last
> log.
> I did do some sysbench tests during the day, but that was way before it
> rebooted. The only thing that could be running during that time was daily
> periodic.
>
> $ sudo xm info
> host                   : soleus01.soleus.nu
> release                : 2.6.32-5-xen-amd64
> version                : #1 SMP Mon Oct 3 07:53:54 UTC 2011
> machine                : x86_64
> nr_cpus                : 8
> nr_nodes               : 2
> cores_per_socket       : 4
> threads_per_core       : 1
> cpu_mhz                : 2200
> hw_caps                :
> 178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000037ff:00000000
> virt_caps              : hvm
> total_memory           : 65534
> free_memory            : 6866
> node_to_cpu            : node0:0-3
>                          node1:4-7
> node_to_memory         : node0:3128
>                          node1:3737
> node_to_dma32_mem      : node0:3128
>                          node1:0
> max_node_id            : 1
> xen_major              : 4
> xen_minor              : 0
> xen_extra              : .1
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32
> hvm-3.0-x86_32p hvm-3.0-x86_64
> xen_scheduler          : credit
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : unavailable
> xen_commandline        : placeholder dom0_mem=1852M
> cc_compiler            : gcc version 4.4.5 (Debian 4.4.5-10)
> cc_compile_by          : waldi
> cc_compile_domain      : debian.org
> cc_compile_date        : Wed Jan 12 14:04:06 UTC 2011
> xend_config_format     : 4
>
> $ uname -a
> Linux soleus01.soleus.nu 2.6.32-5-xen-amd64 #1 SMP Mon Oct 3 07:53:54 UTC
> 2011 x86_64 GNU/Linux
>
> Jeroen.
>
>
first is this a public vm ? and if so who is??
May 29 23:42:30 image01 sshd[31227]: error: Received disconnect from
150.165.15.175: 11: Bye Bye [preauth]

because it is after this potential ssh login attempt, so is this you, has
there been a breach ? only thing i noticed, but it might be nothing.




> _______________________________________________
> freebsd-xen@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-xen
> To unsubscribe, send any mail to "freebsd-xen-unsubscribe@freebsd.org"
>

[-- Attachment #1.2: Type: text/html, Size: 4411 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 30 May 2013, at 16:56, Outback Dingo <outbackdingo@gmail.com> wrote:
> first is this a public vm ? and if so who is??
> May 29 23:42:30 image01 sshd[31227]: error: Received disconnect from
> 150.165.15.175: 11: Bye Bye [preauth]
> 
> because it is after this potential ssh login attempt, so is this you, has
> there been a breach ? only thing i noticed, but it might be nothing.

This VM is on a public IP indeed, and SSH connectivity is enabled. As with any publicly accessible host this then becomes the target of ssh scans.

I included the message just to show that between it and the reboot nothing had been logged.

AFAICT there has not been a breach, and I have not seen any indications at all that there may be one.

Jeroen.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

Hello,

I've pushed a new branch, pvhvm_v14 that contains support for live
migration. While there I've also rebased the changes on top of current
HEAD, so now it contains the recent fixes to blkfront and netfront.

http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v14

Some notes on this branch, I've mainly tested it with Xen 4.3 (unstable)
because previous Xen versions have problems with the PV clock used in
PVHVM when migrating. In order to be able to migrate a PVHVM guest you
will need to add tsc_mode="native_paravirt" to your config file or apply
the following patch to Xen:

http://marc.info/?l=xen-devel&m=137036010517331

I would say that migration across the 4.x series will work without
problems (because they all have support for vector callback injection),
but migrating from 4.x to 3.x will certainly not work. On the other
hand, migrating a guest started on 3.4 to 4.0 should work, although I
have not tested it.

Also, if the migration process fails for some reason, resuming the
original guest on the sender side will leave the VM without working nics
and disks, this is a problem with netfront and blkfront not being able
to resume after suspension if the guest was not actually migrated.

Roger.

Re: FreeBSD PVHVM call for testing

[Outback Dingo]

[-- Attachment #1.1: Type: text/plain, Size: 577 bytes --]

On Mon, Jun 10, 2013 at 10:48 AM, Roger Pau Monné <roger.pau@citrix.com>wrote:

> Hello,
>
> I've pushed a new branch, pvhvm_v14 that contains support for live
> migration. While there I've also rebased the changes on top of current
> HEAD, so now it contains the recent fixes to blkfront and netfront.
>
>
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v14


looking at your master branch your 2 weeks behind current... so where did
you rebase your changes to head, or are you referring to your HEAD, and not
FreeBSD

[-- Attachment #1.2: Type: text/html, Size: 1056 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 10/06/13 17:09, Outback Dingo wrote:
> 
> 
> 
> On Mon, Jun 10, 2013 at 10:48 AM, Roger Pau Monné <roger.pau@citrix.com
> <mailto:roger.pau@citrix.com>> wrote:
> 
>     Hello,
> 
>     I've pushed a new branch, pvhvm_v14 that contains support for live
>     migration. While there I've also rebased the changes on top of current
>     HEAD, so now it contains the recent fixes to blkfront and netfront.
> 
>     http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v14
> 
> 
> looking at your master branch your 2 weeks behind current... so where
> did you rebase your changes to head, or are you referring to your HEAD,
> and not FreeBSD 

No, my HEAD commit from FreeBSD master repository is from 3 days ago:

http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=commit;h=5311e12c931df9b67b64913670eab76a994317b9

This is the commit where I rebased my pvhvm_v14 branch.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 10/06/13 16:48, Roger Pau Monné wrote:
> Hello,
> 
> I've pushed a new branch, pvhvm_v14 that contains support for live
> migration. While there I've also rebased the changes on top of current
> HEAD, so now it contains the recent fixes to blkfront and netfront.
> 
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v14

Hello,

There where some issues with the previous branch (pvhvm_v14), I've
pushed a new one (pvhvm_v16) that fixes the following bugs:

 * Make sure there are no IPIs in flight while the VM is migrated,
having in-flight IPIs is a problem because on resume the event channels
are re-initialized, so all pending events are lost, including IPIs.

 * Reset the clock after migration, this prevent clock drifts when the
VM is migrated.

 * blkfront was not correctly freeing the old event channel port.

The following two commits are needed for Xen:

f8e8fd56bd7d5675e8331b4ec74bae76c9dbf24e x86/HVM: fix initialization of
wallclock time for PVHVM on migration

32c864a35ece2c24a336d183869a546798a4b241 x86/vtsc: update vcpu_time in
hvm_set_guest_time

With this branch I've been able to successfully local migrate a busy VM
400 times consecutively.

As usual, the branch can be found here:

http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v16

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

I've just built a new kernel based on pvhvm_v17, but it panicked on boot.

I still have a xen console attached, so I can provide additional information if someone gives me the right commands.

Jeroen.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 19/06/13 13:13, Jeroen van der Ham wrote:
> Hi,
> 
> I've just built a new kernel based on pvhvm_v17, but it panicked on boot.
> 
> I still have a xen console attached, so I can provide additional information if someone gives me the right commands.
> 
> Jeroen.
> 

Could you provide the boot log of the DomU, backtrace, Xen version and
Dom0 kernel version?

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

On 19 Jun 2013, at 13:34, Roger Pau Monné <roger.pau@citrix.com> wrote:
> 
> Could you provide the boot log of the DomU, backtrace, Xen version and
> Dom0 kernel version?

I did not have a console attached when it rebooted, so I did not have a log of the initial boot. Now that I did, I see that it fails to mount its root volume.

It had been running previously on pvhvm_v10 for about two weeks without problems. I updated my local git, and recompiled the kernel and rebooted. Then this happened.


In order:

Booting...
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 10.0-CURRENT #2 r+6ff8d00-dirty: Tue Jun 18 12:55:16 CEST 2013
    root@image01:/usr/obj/root/freebsd/sys/XENHVM amd64
FreeBSD clang version 3.3 (trunk 178860) 20130405
WARNING: WITNESS option enabled, expect reduced performance.
XEN: Hypervisor version 4.0 detected.
CPU: Quad-Core AMD Opteron(tm) Processor 2374 HE (2200.07-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x100f42  Family = 0x10  Model = 0x4  Stepping = 2
  Features=0x1781fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x80802001<SSE3,CX16,POPCNT,HV>
  AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow!>
  AMD Features2=0x1f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch>
real memory  = 536870912 (512 MB)
avail memory = 472260608 (450 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <Xen HVM>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  2
random device not loaded; using insecure entropy
ioapic0: Changing APIC ID to 1
MADT: Forcing active-low polarity and level trigger for SCI
ioapic0 <Version 1.1> irqs 0-47 on motherboard
kbd1 at kbdmux0
xen_et0: <Xen PV Clock> on motherboard
Event timer "XENTIMER" frequency 1000000000 Hz quality 950
Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
acpi0: <Xen> on motherboard
acpi0: Power Button (fixed)
acpi0: Sleep Button (fixed)
acpi0: reservation of 0, a0000 (3) failed
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x1f48-0x1f4b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc300-0xc30f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
pci0: <bridge> at device 1.3 (no driver attached)
vgapci0: <VGA-compatible display> mem 0xf0000000-0xf1ffffff,0xf3000000-0xf3000fff at device 2.0 on pci0
xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xf2000000-0xf2ffffff irq 28 at device 3.0 on pci0
xenstore0: <XenStore> on xenpci0
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (9600,n,8,1)
ppc0: <Parallel port> port 0x378-0x37f irq 7 on acpi0
ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
ppbus0: <Parallel port bus> on ppc0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
orm0: <ISA Option ROM> at iomem 0xc9000-0xc97ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
fdc0: No FDOUT register!
Timecounters tick every 10.000 msec
xenbusb_front0: <Xen Frontend Devices> on xenstore0
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: <QEMU QEMU DVD-ROM 0.10> Removable CD-ROM SCSI-0 device
cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: cd present [360385 x 2048 byte records]
xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
xn0: Ethernet address: 00:16:3e:2f:b7:22
xn1: <Virtual Network Interface> at device/vif/1 on xenbusb_front0
xn1: Ethernet address: 00:16:3e:3e:64:c5
xenbusb_back0: <Xen Backend Devices> on xenstore0
xctrl0: <Xen Control Device> on xenstore0
xn0: backend features: feature-sg feature-gso-tcp4
xn1: backend features: feature-sg feature-gso-tcp4
xbd0: 20480MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
xbd0: attaching as ada0
xbd0: disk supports cache flush using: barriers
xbd1: 703MB <Virtual Block Device> at device/vbd/5632 on xenbusb_front0
xbd1: attaching as ada2
xbd1: disk supports cache flush using: barriers
SMP: AP CPU #1 Launched!
WARNING: WITNESS option enabled, expect reduced performance.
Trying to mount root from ufs:/dev/ad0p2 [rw]...
mountroot: waiting for device /dev/ad0p2 ...
Mounting from ufs:/dev/ad0p2 failed with error 19.
Loader variables:
  vfs.root.mountfrom=ufs:/dev/ad0p2
  vfs.root.mountfrom.options=rw

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:tank
        cd9660:/dev/acd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot>
panic: mountroot: unable to (re-)mount root.
cpuid = 1
KDB: enter: panic
[ thread pid 1 tid 100002 ]
Stopped at      kdb_enter+0x3e: movq    $0,kdb_why
db> trace
Tracing pid 1 tid 100002 td 0xfffffe0003384920
kdb_enter() at kdb_enter+0x3e/frame 0xffffff80002347d0
vpanic() at vpanic+0x146/frame 0xffffff8000234810
panic() at panic+0x43/frame 0xffffff8000234870
vfs_mountroot() at vfs_mountroot+0x1dc7/frame 0xffffff8000234b20
start_init() at start_init+0x62/frame 0xffffff8000234bb0
fork_exit() at fork_exit+0x84/frame 0xffffff8000234bf0
fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000234bf0
--- trap 0, rip = 0, rsp = 0xffffff8000234cb0, rbp = 0 ---

jeroen@soleus01 ~]$ sudo xm info
host                   : soleus01.soleus.nu
release                : 2.6.32-5-xen-amd64
version                : #1 SMP Mon Oct 3 07:53:54 UTC 2011
machine                : x86_64
nr_cpus                : 8
nr_nodes               : 2
cores_per_socket       : 4
threads_per_core       : 1
cpu_mhz                : 2200
hw_caps                : 178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000037ff:00000000
virt_caps              : hvm
total_memory           : 65534
free_memory            : 6865
node_to_cpu            : node0:0-3
                         node1:4-7
node_to_memory         : node0:3134
                         node1:3731
node_to_dma32_mem      : node0:3128
                         node1:0
max_node_id            : 1
xen_major              : 4
xen_minor              : 0
xen_extra              : .1
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
xen_commandline        : placeholder dom0_mem=1852M
cc_compiler            : gcc version 4.4.5 (Debian 4.4.5-10)
cc_compile_by          : waldi
cc_compile_domain      : debian.org
cc_compile_date        : Wed Jan 12 14:04:06 UTC 2011
xend_config_format     : 4


[jeroen@soleus01 ~]$ uname -a
Linux soleus01.soleus.nu 2.6.32-5-xen-amd64 #1 SMP Mon Oct 3 07:53:54 UTC 2011 x86_64 GNU/Linux

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 19/06/13 14:16, Jeroen van der Ham wrote:
> 
> On 19 Jun 2013, at 13:34, Roger Pau Monné <roger.pau@citrix.com> wrote:
>>
>> Could you provide the boot log of the DomU, backtrace, Xen version and
>> Dom0 kernel version?
> 
> I did not have a console attached when it rebooted, so I did not have a log of the initial boot. Now that I did, I see that it fails to mount its root volume.
> 
> It had been running previously on pvhvm_v10 for about two weeks without problems. I updated my local git, and recompiled the kernel and rebooted. Then this happened.
> 
> 
> In order:
> 
> Booting...
> GDB: no debug ports present
> KDB: debugger backends: ddb
> KDB: current backend: ddb
> Copyright (c) 1992-2013 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
> 	The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 10.0-CURRENT #2 r+6ff8d00-dirty: Tue Jun 18 12:55:16 CEST 2013
>     root@image01:/usr/obj/root/freebsd/sys/XENHVM amd64
> FreeBSD clang version 3.3 (trunk 178860) 20130405
> WARNING: WITNESS option enabled, expect reduced performance.
> XEN: Hypervisor version 4.0 detected.
> CPU: Quad-Core AMD Opteron(tm) Processor 2374 HE (2200.07-MHz K8-class CPU)
>   Origin = "AuthenticAMD"  Id = 0x100f42  Family = 0x10  Model = 0x4  Stepping = 2
>   Features=0x1781fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,MMX,FXSR,SSE,SSE2,HTT>
>   Features2=0x80802001<SSE3,CX16,POPCNT,HV>
>   AMD Features=0xe2500800<SYSCALL,NX,MMX+,FFXSR,LM,3DNow!+,3DNow!>
>   AMD Features2=0x1f3<LAHF,CMP,CR8,ABM,SSE4A,MAS,Prefetch>
> real memory  = 536870912 (512 MB)
> avail memory = 472260608 (450 MB)
> Event timer "LAPIC" quality 400
> ACPI APIC Table: <Xen HVM>
> FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
> FreeBSD/SMP: 1 package(s) x 2 core(s)
>  cpu0 (BSP): APIC ID:  0
>  cpu1 (AP): APIC ID:  2
> random device not loaded; using insecure entropy
> ioapic0: Changing APIC ID to 1
> MADT: Forcing active-low polarity and level trigger for SCI
> ioapic0 <Version 1.1> irqs 0-47 on motherboard
> kbd1 at kbdmux0
> xen_et0: <Xen PV Clock> on motherboard
> Event timer "XENTIMER" frequency 1000000000 Hz quality 950
> Timecounter "XENTIMER" frequency 1000000000 Hz quality 950
> acpi0: <Xen> on motherboard
> acpi0: Power Button (fixed)
> acpi0: Sleep Button (fixed)
> acpi0: reservation of 0, a0000 (3) failed
> cpu0: <ACPI CPU> on acpi0
> cpu1: <ACPI CPU> on acpi0
> attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
> Timecounter "i8254" frequency 1193182 Hz quality 0
> Event timer "i8254" frequency 1193182 Hz quality 100
> atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
> Event timer "RTC" frequency 32768 Hz quality 0
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x1f48-0x1f4b on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pci0: <ACPI PCI bus> on pcib0
> isab0: <PCI-ISA bridge> at device 1.0 on pci0
> isa0: <ISA bus> on isab0
> atapci0: <Intel PIIX3 WDMA2 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc300-0xc30f at device 1.1 on pci0
> ata0: <ATA channel> at channel 0 on atapci0
> ata1: <ATA channel> at channel 1 on atapci0
> pci0: <bridge> at device 1.3 (no driver attached)
> vgapci0: <VGA-compatible display> mem 0xf0000000-0xf1ffffff,0xf3000000-0xf3000fff at device 2.0 on pci0
> xenpci0: <Xen Platform Device> port 0xc000-0xc0ff mem 0xf2000000-0xf2ffffff irq 28 at device 3.0 on pci0
> xenstore0: <XenStore> on xenpci0
> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> psm0: <PS/2 Mouse> irq 12 on atkbdc0
> psm0: [GIANT-LOCKED]
> psm0: model IntelliMouse Explorer, device ID 4
> fdc0: <floppy drive controller> port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0
> fdc0: does not respond
> device_attach: fdc0 attach returned 6
> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
> uart0: console (9600,n,8,1)
> ppc0: <Parallel port> port 0x378-0x37f irq 7 on acpi0
> ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode
> ppbus0: <Parallel port bus> on ppc0
> lpt0: <Printer> on ppbus0
> lpt0: Interrupt-driven port
> ppi0: <Parallel I/O> on ppbus0
> orm0: <ISA Option ROM> at iomem 0xc9000-0xc97ff on isa0
> sc0: <System console> at flags 0x100 on isa0
> sc0: VGA <16 virtual consoles, flags=0x300>
> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
> fdc0: No FDOUT register!
> Timecounters tick every 10.000 msec
> xenbusb_front0: <Xen Frontend Devices> on xenstore0
> cd0 at ata1 bus 0 scbus1 target 0 lun 0
> cd0: <QEMU QEMU DVD-ROM 0.10> Removable CD-ROM SCSI-0 device
> cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
> cd0: cd present [360385 x 2048 byte records]
> xn0: <Virtual Network Interface> at device/vif/0 on xenbusb_front0
> xn0: Ethernet address: 00:16:3e:2f:b7:22
> xn1: <Virtual Network Interface> at device/vif/1 on xenbusb_front0
> xn1: Ethernet address: 00:16:3e:3e:64:c5
> xenbusb_back0: <Xen Backend Devices> on xenstore0
> xctrl0: <Xen Control Device> on xenstore0
> xn0: backend features: feature-sg feature-gso-tcp4
> xn1: backend features: feature-sg feature-gso-tcp4
> xbd0: 20480MB <Virtual Block Device> at device/vbd/768 on xenbusb_front0
> xbd0: attaching as ada0
> xbd0: disk supports cache flush using: barriers
> xbd1: 703MB <Virtual Block Device> at device/vbd/5632 on xenbusb_front0
> xbd1: attaching as ada2
> xbd1: disk supports cache flush using: barriers
> SMP: AP CPU #1 Launched!
> WARNING: WITNESS option enabled, expect reduced performance.
> Trying to mount root from ufs:/dev/ad0p2 [rw]...
> mountroot: waiting for device /dev/ad0p2 ...
> Mounting from ufs:/dev/ad0p2 failed with error 19.
> Loader variables:
>   vfs.root.mountfrom=ufs:/dev/ad0p2
>   vfs.root.mountfrom.options=rw
> 
> Manual root filesystem specification:
>   <fstype>:<device> [options]
>       Mount <device> using filesystem <fstype>
>       and with the specified (optional) option list.
> 
>     eg. ufs:/dev/da0s1a
>         zfs:tank
>         cd9660:/dev/acd0 ro
>           (which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)
> 
>   ?               List valid disk boot devices
>   .               Yield 1 second (for background tasks)
>   <empty line>    Abort manual input
> 
> mountroot>
> panic: mountroot: unable to (re-)mount root.
> cpuid = 1
> KDB: enter: panic
> [ thread pid 1 tid 100002 ]
> Stopped at      kdb_enter+0x3e: movq    $0,kdb_why
> db> trace
> Tracing pid 1 tid 100002 td 0xfffffe0003384920
> kdb_enter() at kdb_enter+0x3e/frame 0xffffff80002347d0
> vpanic() at vpanic+0x146/frame 0xffffff8000234810
> panic() at panic+0x43/frame 0xffffff8000234870
> vfs_mountroot() at vfs_mountroot+0x1dc7/frame 0xffffff8000234b20
> start_init() at start_init+0x62/frame 0xffffff8000234bb0
> fork_exit() at fork_exit+0x84/frame 0xffffff8000234bf0
> fork_trampoline() at fork_trampoline+0xe/frame 0xffffff8000234bf0
> --- trap 0, rip = 0, rsp = 0xffffff8000234cb0, rbp = 0 ---
> 
> jeroen@soleus01 ~]$ sudo xm info
> host                   : soleus01.soleus.nu
> release                : 2.6.32-5-xen-amd64
> version                : #1 SMP Mon Oct 3 07:53:54 UTC 2011
> machine                : x86_64
> nr_cpus                : 8
> nr_nodes               : 2
> cores_per_socket       : 4
> threads_per_core       : 1
> cpu_mhz                : 2200
> hw_caps                : 178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000037ff:00000000
> virt_caps              : hvm
> total_memory           : 65534
> free_memory            : 6865
> node_to_cpu            : node0:0-3
>                          node1:4-7
> node_to_memory         : node0:3134
>                          node1:3731
> node_to_dma32_mem      : node0:3128
>                          node1:0
> max_node_id            : 1
> xen_major              : 4
> xen_minor              : 0
> xen_extra              : .1
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
> xen_scheduler          : credit
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : unavailable
> xen_commandline        : placeholder dom0_mem=1852M
> cc_compiler            : gcc version 4.4.5 (Debian 4.4.5-10)
> cc_compile_by          : waldi
> cc_compile_domain      : debian.org
> cc_compile_date        : Wed Jan 12 14:04:06 UTC 2011
> xend_config_format     : 4
> 
> 
> [jeroen@soleus01 ~]$ uname -a
> Linux soleus01.soleus.nu 2.6.32-5-xen-amd64 #1 SMP Mon Oct 3 07:53:54 UTC 2011 x86_64 GNU/Linux
> 

That's because Justin recently pushed a commit that changed the ad
translation to ada, you should change your /etc/fstab to ada0p2. It's
commit 526f3ad11acb296481215d7c2915b3f30f1844f6.

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

On 19 Jun 2013, at 14:20, Roger Pau Monné <roger.pau@citrix.com> wrote:
> That's because Justin recently pushed a commit that changed the ad
> translation to ada, you should change your /etc/fstab to ada0p2. It's
> commit 526f3ad11acb296481215d7c2915b3f30f1844f6.


Ah, you may want to update the wiki page also to warn for that. :)

Jeroen.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 19/06/13 14:33, Jeroen van der Ham wrote:
> 
> On 19 Jun 2013, at 14:20, Roger Pau Monné <roger.pau@citrix.com> wrote:
>> That's because Justin recently pushed a commit that changed the ad
>> translation to ada, you should change your /etc/fstab to ada0p2. It's
>> commit 526f3ad11acb296481215d7c2915b3f30f1844f6.
> 
> 
> Ah, you may want to update the wiki page also to warn for that. :)

D'oh, I've completely forgot about the wiki page, it's updated now,
thanks for the pointer.

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 19 Jun 2013, at 14:44, Roger Pau Monné <roger.pau@citrix.com> wrote:

> On 19/06/13 14:33, Jeroen van der Ham wrote:
>> 
>> On 19 Jun 2013, at 14:20, Roger Pau Monné <roger.pau@citrix.com> wrote:
>>> That's because Justin recently pushed a commit that changed the ad
>>> translation to ada, you should change your /etc/fstab to ada0p2. It's
>>> commit 526f3ad11acb296481215d7c2915b3f30f1844f6.
>> 
>> 
>> Ah, you may want to update the wiki page also to warn for that. :)
> 
> D'oh, I've completely forgot about the wiki page, it's updated now,
> thanks for the pointer.

Okay, everything works again now.

Additionally, I've applied the patches from FreeBSD-SA-13:06.mmap, rebuilt the kernel and rebooted, and the system now works fine.

I did note however that rebuilding the kernel takes an awful lot more time than on a FreeBSD9 system. As in it took about 2 hours longer.

Jeroen.

Re: FreeBSD PVHVM call for testing

[Justin T. Gibbs]

On Jun 19, 2013, at 10:07 AM, Jeroen van der Ham <jeroen@dckd.nl> wrote:

> Hi,
> 
> On 19 Jun 2013, at 14:44, Roger Pau Monné <roger.pau@citrix.com> wrote:
> 
>> On 19/06/13 14:33, Jeroen van der Ham wrote:
>>> 
>>> On 19 Jun 2013, at 14:20, Roger Pau Monné <roger.pau@citrix.com> wrote:
>>>> That's because Justin recently pushed a commit that changed the ad
>>>> translation to ada, you should change your /etc/fstab to ada0p2. It's
>>>> commit 526f3ad11acb296481215d7c2915b3f30f1844f6.
>>> 
>>> 
>>> Ah, you may want to update the wiki page also to warn for that. :)
>> 
>> D'oh, I've completely forgot about the wiki page, it's updated now,
>> thanks for the pointer.
> 
> Okay, everything works again now.

Should we encourage folks to just configure their VMs to use xbd?  I hope some day that the system will just report "da" devices, so the ada name may change again.  We could also suggest specifying SCSI disks in the VM config since I don't think "da" will ever change.

> Additionally, I've applied the patches from FreeBSD-SA-13:06.mmap, rebuilt the kernel and rebooted, and the system now works fine.
> 
> I did note however that rebuilding the kernel takes an awful lot more time than on a FreeBSD9 system. As in it took about 2 hours longer.
> 
> Jeroen.

I've never seen a kernel build take 2 hours, much less 2 hours *longer*.  Are you talking about buildworld?  It would be interesting to know your results building stable/9 sources in your 10 environment to see if this is just due to "build bloat" or a true performance regression.

--
Justin

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 19 Jun 2013, at 18:15, Justin T. Gibbs <gibbs@FreeBSD.org> wrote:
> 
> I've never seen a kernel build take 2 hours, much less 2 hours *longer*.  Are you talking about buildworld?  It would be interesting to know your results building stable/9 sources in your 10 environment to see if this is just due to "build bloat" or a true performance regression.
> 

I copy/pasted the command from the wiki:

> # make kernel-toolchain && make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM

On the stable/9 I only did 

> make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM


I guess the kernel-toolchain takes a long time to build…and from what I can see it does a clean before rebuilding also.

I'm doing the kernel-toolchain step only now and will report how long it took.

Jeroen.

Re: FreeBSD PVHVM call for testing

[Justin T. Gibbs]

On Jun 19, 2013, at 10:50 AM, Jeroen van der Ham <jeroen@dckd.nl> wrote:

> Hi,
> 
> On 19 Jun 2013, at 18:15, Justin T. Gibbs <gibbs@FreeBSD.org> wrote:
>> 
>> I've never seen a kernel build take 2 hours, much less 2 hours *longer*.  Are you talking about buildworld?  It would be interesting to know your results building stable/9 sources in your 10 environment to see if this is just due to "build bloat" or a true performance regression.
>> 
> 
> I copy/pasted the command from the wiki:
> 
>> # make kernel-toolchain && make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM
> 
> On the stable/9 I only did 
> 
>> make buildkernel KERNCONF=XENHVM && make installkernel KERNCONF=XENHVM
> 
> 
> I guess the kernel-toolchain takes a long time to build…and from what I can see it does a clean before rebuilding also.
> 
> I'm doing the kernel-toolchain step only now and will report how long it took.
> 
> Jeroen.

Oh.  Without any parallelism (-j X), the build will take a really long time.  Even with only one core, you'll get a large speedup by performing a parallel build since many steps of the build are I/O bound.

--
Justin

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 19 Jun 2013, at 18:52, Justin T. Gibbs <gibbs@FreeBSD.org> wrote:
>> I guess the kernel-toolchain takes a long time to build…and from what I can see it does a clean before rebuilding also.
>> 
>> I'm doing the kernel-toolchain step only now and will report how long it took.

This seems to be it, that took roughly 2 hours to build. The actual kernel is probably pretty fast in building.

> Oh.  Without any parallelism (-j X), the build will take a really long time.  Even with only one core, you'll get a large speedup by performing a parallel build since many steps of the build are I/O bound.

Neither of the systems actually had parallelism defined, either as flag or in /etc/make.conf

Jeroen.

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

I have this running for a day or so now, but I'm noticing that the load averages seem a bit off:

$ uptime
11:17AM  up 17:14, 1 user, load averages: 0.31, 0.27, 0.21

This is for a clean install, with just enough installed to compile this kernel. In top I'm seeing that the machine is idling >98% of the time. But this does not correlate to the load displayed above.

Jeroen.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 20/06/13 11:20, Jeroen van der Ham wrote:
> Hi,
> 
> I have this running for a day or so now, but I'm noticing that the load averages seem a bit off:
> 
> $ uptime
> 11:17AM  up 17:14, 1 user, load averages: 0.31, 0.27, 0.21
> 
> This is for a clean install, with just enough installed to compile this kernel. In top I'm seeing that the machine is idling >98% of the time. But this does not correlate to the load displayed above.

This is probably due to the fact that we are not properly accounting for
blocked/runnable/offline time. Did you see the same when running the
XENHVM kernel without my patches?

Roger.

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 20 Jun 2013, at 11:33, Roger Pau Monné <roger.pau@citrix.com> wrote:
> 
> This is probably due to the fact that we are not properly accounting for
> blocked/runnable/offline time. Did you see the same when running the
> XENHVM kernel without my patches?
> 

I have a different system on the same platform running FreeBSD9 with XENHVM. This server is running (web)mail, smokeping and irssi.

That gives:

 11:35AM  up 20:07, 1 user, load averages: 0.06, 0.06, 0.07

Jeroen.

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

After some more testing I thought it would be good to put this into production for my personal server. I've used pvhvm_v19 and built it without debugging options and installed it on a FreeBSD 9.1 system.

I've run into some hiccups with 9.1 user land and a 10-CURRENT kernel, but that's all solvable[0].

My VPS has some very limited memory (256M), but I've compensated with swap space (1G)

Now anytime I'm putting the system under stress, by building ports or by running a git clone on the kernel repository here, I'm seeing a lot of messages about swap_pager:

> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 132545, size: 4096

The system also becomes very sluggish and sometimes unresponsive.
The weird thing was that one of these messages happened right after a reboot when I rebuilt an outdated port and on the main console was checking the swap memory:

> jeroen:~/ $ swapinfo                                                  [8:13:29]
> Device          1K-blocks     Used    Avail Capacity
> /dev/ada0p2        524288     2484   521804     0%
> /dev/md0          1048576     2364  1046212     0%
> Total             1572864     4848  1568016     0%
> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 131424, size: 4096


Is anyone else seeing something similar?
I certainly did not experience something like this on 9.0 with a XENHVM kernel.

If necessary I can rebuild a kernel with debugging support and do some more recording of what is actually going on.

Jeroen.


[0]: I have edited bsd.port.mk to always apply the FBSD10_FIX, and for version checking I am  running "pkg version" with UNAME_r=9.1-RELEASE.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 22/07/13 09:18, Jeroen van der Ham wrote:
> Hi,
> 
> After some more testing I thought it would be good to put this into production for my personal server. I've used pvhvm_v19 and built it without debugging options and installed it on a FreeBSD 9.1 system.
> 
> I've run into some hiccups with 9.1 user land and a 10-CURRENT kernel, but that's all solvable[0].
> 
> My VPS has some very limited memory (256M), but I've compensated with swap space (1G)

Is your guest running a 32bit or a 64bit kernel?

Could you also provide the config file used to launch your guest and the
Xen and Dom0 kernel versions?

> 
> Now anytime I'm putting the system under stress, by building ports or by running a git clone on the kernel repository here, I'm seeing a lot of messages about swap_pager:
> 
>> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 132545, size: 4096
> 
> The system also becomes very sluggish and sometimes unresponsive.
> The weird thing was that one of these messages happened right after a reboot when I rebuilt an outdated port and on the main console was checking the swap memory:
> 
>> jeroen:~/ $ swapinfo                                                  [8:13:29]
>> Device          1K-blocks     Used    Avail Capacity
>> /dev/ada0p2        524288     2484   521804     0%
>> /dev/md0          1048576     2364  1046212     0%
>> Total             1572864     4848  1568016     0%
>> swap_pager: indefinite wait buffer: bufobj: 0, blkno: 131424, size: 4096
> 
> 
> Is anyone else seeing something similar?

Could you also try a HEAD XENHVM kernel (without my patches), to see if
the issue is related to my changes or to some bug already present in HEAD?

> I certainly did not experience something like this on 9.0 with a XENHVM kernel.
> 
> If necessary I can rebuild a kernel with debugging support and do some more recording of what is actually going on.
> 
> Jeroen.
> 
> 
> [0]: I have edited bsd.port.mk to always apply the FBSD10_FIX, and for version checking I am  running "pkg version" with UNAME_r=9.1-RELEASE.
> 

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 22 Jul 2013, at 10:29, Roger Pau Monné <roger.pau@citrix.com> wrote:
> Is your guest running a 32bit or a 64bit kernel?

$ uname -a
FreeBSD positron.dckd.nl 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r+a09eac7-dirty: Wed Jul 17 17:51:10 CEST 2013     root@image01:/usr/obj/usr/home/jeroen/freebsd/sys/XENHVM  amd64

> 
> Could you also provide the config file used to launch your guest and the
> Xen and Dom0 kernel versions?

Guest config:

kernel = '/usr/lib/xen-4.0/boot/hvmloader'
device_model = '/usr/lib/xen-4.0/bin/qemu-dm'
builder = 'hvm'
shadow_memory = 8
memory = 512
name = "positron"
vcpus = 2
cpus = "2-7"
maxvcpus = 4
xen_shell = 'root, jeroen'

vif = [
'type=vifname=positron.wan,bridge=br-wan,mac=00:16:3E:2F:AD:99,ip=94.142.246.99'
,
'type=vifname=positron.lan,bridge=br-lan,mac=00:16:3E:0D:96:5C,ip=10.20.0.99'
]

disk = ['phy:/xen/domains/positron/positron-disk1,hda,w']

xen_platform_pci=1
boot = 'c'
sdl=0
stdvga=0
serial='pty'


Xen info:

host                   : soleus01.soleus.nu
release                : 2.6.32-5-xen-amd64
version                : #1 SMP Mon Oct 3 07:53:54 UTC 2011
machine                : x86_64
nr_cpus                : 8
nr_nodes               : 2
cores_per_socket       : 4
threads_per_core       : 1
cpu_mhz                : 2200
hw_caps                : 178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000037ff:00000000
virt_caps              : hvm
total_memory           : 65534
free_memory            : 6865
node_to_cpu            : node0:0-3
                         node1:4-7
node_to_memory         : node0:3128
                         node1:3737
node_to_dma32_mem      : node0:3128
                         node1:0
max_node_id            : 1
xen_major              : 4
xen_minor              : 0
xen_extra              : .1
xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler          : credit
xen_pagesize           : 4096
platform_params        : virt_start=0xffff800000000000
xen_changeset          : unavailable
xen_commandline        : placeholder dom0_mem=1852M
cc_compiler            : gcc version 4.4.5 (Debian 4.4.5-10)
cc_compile_by          : waldi
cc_compile_domain      : debian.org
cc_compile_date        : Wed Jan 12 14:04:06 UTC 2011
xend_config_format     : 4

> Could you also try a HEAD XENHVM kernel (without my patches), to see if
> the issue is related to my changes or to some bug already present in HEAD?

Will do.


Jeroen.

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 22/07/13 10:40, Jeroen van der Ham wrote:
> Hi,
> 
> On 22 Jul 2013, at 10:29, Roger Pau Monné <roger.pau@citrix.com> wrote:
>> Is your guest running a 32bit or a 64bit kernel?
> 
> $ uname -a
> FreeBSD positron.dckd.nl 10.0-CURRENT FreeBSD 10.0-CURRENT #0 r+a09eac7-dirty: Wed Jul 17 17:51:10 CEST 2013     root@image01:/usr/obj/usr/home/jeroen/freebsd/sys/XENHVM  amd64
> 
>>
>> Could you also provide the config file used to launch your guest and the
>> Xen and Dom0 kernel versions?
> 
> Guest config:
> 
> kernel = '/usr/lib/xen-4.0/boot/hvmloader'
> device_model = '/usr/lib/xen-4.0/bin/qemu-dm'
> builder = 'hvm'
> shadow_memory = 8

Are you setting the shadow memory size manually because your hardware
lacks HAP support?

> memory = 512
> name = "positron"
> vcpus = 2
> cpus = "2-7"
> maxvcpus = 4
> xen_shell = 'root, jeroen'

This doesn't seem like a standard xl config option.

> 
> vif = [
> 'type=vifname=positron.wan,bridge=br-wan,mac=00:16:3E:2F:AD:99,ip=94.142.246.99'
> ,
> 'type=vifname=positron.lan,bridge=br-lan,mac=00:16:3E:0D:96:5C,ip=10.20.0.99'
> ]
> 
> disk = ['phy:/xen/domains/positron/positron-disk1,hda,w']
> 
> xen_platform_pci=1
> boot = 'c'
> sdl=0
> stdvga=0
> serial='pty'
> 
> 
> Xen info:
> 
> host                   : soleus01.soleus.nu
> release                : 2.6.32-5-xen-amd64
> version                : #1 SMP Mon Oct 3 07:53:54 UTC 2011
> machine                : x86_64
> nr_cpus                : 8
> nr_nodes               : 2
> cores_per_socket       : 4
> threads_per_core       : 1
> cpu_mhz                : 2200
> hw_caps                : 178bf3ff:efd3fbff:00000000:00001310:00802001:00000000:000037ff:00000000
> virt_caps              : hvm
> total_memory           : 65534
> free_memory            : 6865
> node_to_cpu            : node0:0-3
>                          node1:4-7
> node_to_memory         : node0:3128
>                          node1:3737
> node_to_dma32_mem      : node0:3128
>                          node1:0
> max_node_id            : 1
> xen_major              : 4
> xen_minor              : 0
> xen_extra              : .1
> xen_caps               : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
> xen_scheduler          : credit
> xen_pagesize           : 4096
> platform_params        : virt_start=0xffff800000000000
> xen_changeset          : unavailable
> xen_commandline        : placeholder dom0_mem=1852M
> cc_compiler            : gcc version 4.4.5 (Debian 4.4.5-10)
> cc_compile_by          : waldi
> cc_compile_domain      : debian.org
> cc_compile_date        : Wed Jan 12 14:04:06 UTC 2011
> xend_config_format     : 4

I've set up a XENHVM system with 256MB of RAM and swap and this is what
I see when doing a make buildkernel:

[root@ ~]# swapinfo
Device          1K-blocks     Used    Avail Capacity
/dev/ada0p3       1048540   351116   697424    33%

I don't see any messages on the console or anything else, the system
seems to be sluggish while doing the build, but that's quite normal when
using only 256MB of RAM.

This test was done using the pvhvm_20 branch, but it should not contain
any significant code changes in comparison with pvhvm_v19 (it's just a
rebase on top of HEAD and a reorder of patches). Could this be because
you are using a 9 userland with a 10 kernel?

Roger.

Re: FreeBSD PVHVM call for testing

[Jeroen van der Ham]

Hi,

On 22 Jul 2013, at 10:40, Jeroen van der Ham <jeroen@dckd.nl> wrote:
>> Could you also try a HEAD XENHVM kernel (without my patches), to see if
>> the issue is related to my changes or to some bug already present in HEAD?

It seems I was worrying too soon.

I have been putting the system through the wringer some more, and I now believe that it has been caused by adding a new swap file. Just before I rebooted my system I created a larger swap file to be used by /etc/rc.d/add_swap.
Right after I rebooted I started compiling and doing other things. And I am getting the feeling that the system was still initialising that swap file and was unable to provide swap space at that time.

I've rebooted my system again with the PVHVM system, abused it even more than I did before and I'm not seeing the same messages again, nor getting any exaggerated sluggishness.

So my apologies for the false alarm.

Jeroen.

Re: FreeBSD PVHVM call for testing

[Eggert, Lars]

[-- Attachment #1.1: Type: text/plain, Size: 318 bytes --]

Hi,

On May 13, 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
> Right now the code is in a state where it can be tested by users, so we
> would like to encourage FreeBSD and Xen users to test it and provide
> feedback.

any idea if/when this code will be merged into -CURRENT?

Thanks,
Lars

[-- Attachment #1.2: Message signed with OpenPGP using GPGMail --]
[-- Type: application/pgp-signature, Size: 273 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: FreeBSD PVHVM call for testing

[Roger Pau Monné]

On 11/10/13 11:42, Eggert, Lars wrote:
> Hi,
> 
> On May 13, 2013, at 20:32, Roger Pau Monné <roger.pau@citrix.com> wrote:
>> Right now the code is in a state where it can be tested by users, so we
>> would like to encourage FreeBSD and Xen users to test it and provide
>> feedback.
> 
> any idea if/when this code will be merged into -CURRENT?

It's already in HEAD, and will hopefully be part of the 10 release.

Roger.

FreeBSD PVHVM call for testing

[Roger Pau Monné]

Hello,

Recently Justin T Gibbs, Will Andrews and myself have been working on
improving the Xen support in FreeBSD. The main goal of this was to bring
full PVHVM support to FreeBSD, right now FreeBSD is only using PV
interfaces for disk and network interfaces when running as a HVM guest.
The main benefits of this changes are that Xen virtual interrupts (event
channels) are now delivered to the guest using a vector callback
injection, that is a per-cpu mechanism that allows each vCPU to have
different interrupts assigned, so for example network and disk
interrupts are delivered to different vCPUs in order to improve
performance. With this changes FreeBSD also uses PV timers when running
as an HVM guest, which should provide better time keeping and reduce the
virtualization overhead, since emulated timers are no longer used. PV
IPIs can also be used inside a HVM guest, but this will be implemented
later.

Right now the code is in a state where it can be tested by users, so we
would like to encourage FreeBSD and Xen users to test it and provide
feedback.

The code is available in the following git repository, under the branch
pvhvm_v5:

http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=summary

Also, I've created a wiki page that explains how to set up a FreeBSD
PVHVM for testing:

http://wiki.xen.org/wiki/Testing_FreeBSD_PVHVM