VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Michael MacLeod]

[-- Attachment #1.1: Type: text/plain, Size: 6772 bytes --]

I've been trying to get VT-D working with Xen and my Asus P5E-VM DO
motherboard for the last week. This board is listed on the VTdHowTo xen wiki
page as supported, and I've found another subscriber of the xen-users list
who has this configuration working, which makes my predicament all the more
odd.

I'm using debian lenny as my dom0 OS, though I've built Xen 3.4.1 from
source and can replicate this problem with both the lenny 2.6.26-2-xen-amd64
kernel and the Xen 2.6.18-xen.hg kernel I built. I've tried several
different BIOS revisions, and the problem is consistent across them.

Here's some output from my bootup and from acpidump:

# xm dmesg | grep -C1 VT-D
(XEN) ACPI: HPET id: 0xffffffff base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed90000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed92000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed93000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff
(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.

# acpidump -t DMAR
Wrong checksum for OEMB!
Wrong checksum for !

# acpidump | grep -B3 -A20 DMAR
Wrong checksum for OEMB!
Wrong checksum for !

 @ 0xcff601b0
  0000: 00 4d 41 52 48 01 00 00 01 f5 41 4d 49 00 00 00  .MARH.....AMI...
  0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
  0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
  0030: 00 00 18 00 00 00 00 00 00 00 d9 fe 00 00 00 00  ................
  0040: 01 08 00 00 00 00 1b 00 00 00 28 00 00 00 00 00  ..........(.....
  0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............
  0060: 01 08 00 00 00 00 03 02 01 08 00 00 00 00 03 03  ................
  0070: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
  0080: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
  0090: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
  00a0: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
  00b0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
  00c0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
  00d0: 01 08 00 00 00 00 1a 07 01 00 18 00 00 00 00 00  ................
  00e0: 00 00 00 d0 00 00 00 00 ff ff ff cf 00 00 00 00  ................
  00f0: 01 00 58 00 00 00 00 00 00 00 fe cf 00 00 00 00  ..X.............
  0100: ff ff fe cf 00 00 00 00 01 08 00 00 00 00 1d 00  ................
  0110: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
  0120: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
  0130: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
  0140: 01 08 00 00 00 00 1a 07                          ........

I've posted the my full xm dmesg here: http://pastebin.com/m32beff18 and you
can find my full dmesg output here: http://pastebin.com/m74a7dc2a

The other user of the xen-users list (Christian Tramnitz) who has VT-D
working with the same motherboard and BIOS revision listed this as the
output he gets:

(XEN) [VT-D]iommu.c:722: iommu_page_fault: iommu->reg = ffff828bfff56000
(XEN) [VT-D]iommu.c:691: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:694: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:676: iommu_fault:DMA Write: 0:2.0 addr 400000000 REASON
5 iommu->reg = ffff828bfff56000
(XEN) print_vtd_entries: iommu = ffff83022bde62d0 bdf = 0:2:0 gmfn = 400000
(XEN)     root_entry = ffff83022bdcf000
(XEN)     root_entry[0] = 224cc8001
(XEN)     context = ffff830224cc8000
(XEN)     context[10] = 101_22bdb3001
(XEN)     l3 = ffff83022bdb3000
(XEN)     l3_index = 10
(XEN)     l3[10] = 0
(XEN)     l3[10] not present

# acpidump | grep -B3 -A20 DMAR
Wrong checksum for OEMB
Wrong checksum for
Wrong checksum for OEMB!

@ 0xcf5601b0
 0000: 00 4d 41 52 40 01 00 00 01 01 41 4d 49 00 00 00  .MAR@.....AMI...
 0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
 0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
 0030: 00 00 18 00 00 00 00 00 00 10 d9 fe 00 00 00 00  ................
 0040: 01 08 00 00 00 00 02 00 00 00 18 00 00 00 00 00  ................
 0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............
 0060: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
 0070: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
 0080: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
 0090: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
 00a0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
 00b0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
 00c0: 01 08 00 00 00 00 1a 07 01 00 20 00 00 00 00 00  .......... .....
 00d0: 00 00 60 cf 00 00 00 00 ff ff ff cf 00 00 00 00  ..`.............
 00e0: 01 08 00 00 00 00 02 00 01 00 58 00 00 00 00 00  ..........X.....
 00f0: 00 00 5e cf 00 00 00 00 ff ff 5e cf 00 00 00 00  ..^.......^.....
 0100: 01 08 00 00 00 00 1d 00 01 08 00 00 00 00 1d 01  ................
 0110: 01 08 00 00 00 00 1d 02 01 08 00 00 00 00 1d 07  ................
 0120: 01 08 00 00 00 00 1a 00 01 08 00 00 00 00 1a 01  ................
 0130: 01 08 00 00 00 00 1a 02 01 08 00 00 00 00 1a 07  ................

XSDT @ 0xcf550100
Wrong checksum for !

# acpidump -t DMAR
Wrong checksum for OEMB
Wrong checksum for
Wrong checksum for OEMB!
Wrong checksum for !

I've also, on the suggestion of Christian I tried disabling USB in the BIOS
completely, as he said that he had to do that with a different Asus
motherboard that also supported VT-D before it would work, but that didn't
resolve the problem for me.

Please let me know if there is anything else I can try or any other info I
can provide. I'd be quite happy to work with a dev to determine the exact
source of the problem and test patches, etc.

Thanks,
Mike

[-- Attachment #1.2: Type: text/html, Size: 7529 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Han, Weidong]

[-- Attachment #1.1: Type: text/plain, Size: 7302 bytes --]

Hi Mike,

Obviously your BIOS is broken. Pls contact your vendor for help.

Regards,
Weidong

________________________________
From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Michael MacLeod
Sent: 2009年9月27日 4:18
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

I've been trying to get VT-D working with Xen and my Asus P5E-VM DO motherboard for the last week. This board is listed on the VTdHowTo xen wiki page as supported, and I've found another subscriber of the xen-users list who has this configuration working, which makes my predicament all the more odd.

I'm using debian lenny as my dom0 OS, though I've built Xen 3.4.1 from source and can replicate this problem with both the lenny 2.6.26-2-xen-amd64 kernel and the Xen 2.6.18-xen.hg kernel I built. I've tried several different BIOS revisions, and the problem is consistent across them.

Here's some output from my bootup and from acpidump:

# xm dmesg | grep -C1 VT-D
(XEN) ACPI: HPET id: 0xffffffff base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed90000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed92000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed93000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff
(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.

# acpidump -t DMAR
Wrong checksum for OEMB!
Wrong checksum for !

# acpidump | grep -B3 -A20 DMAR
Wrong checksum for OEMB!
Wrong checksum for !

 @ 0xcff601b0
  0000: 00 4d 41 52 48 01 00 00 01 f5 41 4d 49 00 00 00  .MARH.....AMI...
  0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
  0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
  0030: 00 00 18 00 00 00 00 00 00 00 d9 fe 00 00 00 00  ................
  0040: 01 08 00 00 00 00 1b 00 00 00 28 00 00 00 00 00  ..........(.....
  0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............
  0060: 01 08 00 00 00 00 03 02 01 08 00 00 00 00 03 03  ................
  0070: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
  0080: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
  0090: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
  00a0: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
  00b0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
  00c0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
  00d0: 01 08 00 00 00 00 1a 07 01 00 18 00 00 00 00 00  ................
  00e0: 00 00 00 d0 00 00 00 00 ff ff ff cf 00 00 00 00  ................
  00f0: 01 00 58 00 00 00 00 00 00 00 fe cf 00 00 00 00  ..X.............
  0100: ff ff fe cf 00 00 00 00 01 08 00 00 00 00 1d 00  ................
  0110: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
  0120: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
  0130: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
  0140: 01 08 00 00 00 00 1a 07                          ........

I've posted the my full xm dmesg here: http://pastebin.com/m32beff18 and you can find my full dmesg output here: http://pastebin.com/m74a7dc2a

The other user of the xen-users list (Christian Tramnitz) who has VT-D working with the same motherboard and BIOS revision listed this as the output he gets:

(XEN) [VT-D]iommu.c:722: iommu_page_fault: iommu->reg = ffff828bfff56000
(XEN) [VT-D]iommu.c:691: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:694: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:676: iommu_fault:DMA Write: 0:2.0 addr 400000000 REASON 5 iommu->reg = ffff828bfff56000
(XEN) print_vtd_entries: iommu = ffff83022bde62d0 bdf = 0:2:0 gmfn = 400000
(XEN)     root_entry = ffff83022bdcf000
(XEN)     root_entry[0] = 224cc8001
(XEN)     context = ffff830224cc8000
(XEN)     context[10] = 101_22bdb3001
(XEN)     l3 = ffff83022bdb3000
(XEN)     l3_index = 10
(XEN)     l3[10] = 0
(XEN)     l3[10] not present

# acpidump | grep -B3 -A20 DMAR
Wrong checksum for OEMB
Wrong checksum for
Wrong checksum for OEMB!

@ 0xcf5601b0
 0000: 00 4d 41 52 40 01 00 00 01 01 41 4d 49 00 00 00  .MAR@.....AMI...

 0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
 0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
 0030: 00 00 18 00 00 00 00 00 00 10 d9 fe 00 00 00 00  ................
 0040: 01 08 00 00 00 00 02 00 00 00 18 00 00 00 00 00  ................

 0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............
 0060: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
 0070: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
 0080: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
 0090: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
 00a0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
 00b0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
 00c0: 01 08 00 00 00 00 1a 07 01 00 20 00 00 00 00 00  .......... .....
 00d0: 00 00 60 cf 00 00 00 00 ff ff ff cf 00 00 00 00  ..`.............
 00e0: 01 08 00 00 00 00 02 00 01 00 58 00 00 00 00 00  ..........X.....
 00f0: 00 00 5e cf 00 00 00 00 ff ff 5e cf 00 00 00 00  ..^.......^.....
 0100: 01 08 00 00 00 00 1d 00 01 08 00 00 00 00 1d 01  ................
 0110: 01 08 00 00 00 00 1d 02 01 08 00 00 00 00 1d 07  ................
 0120: 01 08 00 00 00 00 1a 00 01 08 00 00 00 00 1a 01  ................
 0130: 01 08 00 00 00 00 1a 02 01 08 00 00 00 00 1a 07  ................

XSDT @ 0xcf550100
Wrong checksum for !

# acpidump -t DMAR
Wrong checksum for OEMB
Wrong checksum for
Wrong checksum for OEMB!
Wrong checksum for !

I've also, on the suggestion of Christian I tried disabling USB in the BIOS completely, as he said that he had to do that with a different Asus motherboard that also supported VT-D before it would work, but that didn't resolve the problem for me.

Please let me know if there is anything else I can try or any other info I can provide. I'd be quite happy to work with a dev to determine the exact source of the problem and test patches, etc.

Thanks,
Mike

[-- Attachment #1.2: Type: text/html, Size: 9691 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Michael MacLeod]

[-- Attachment #1.1: Type: text/plain, Size: 504 bytes --]

2009/9/27 Han, Weidong <weidong.han@intel.com>

>  Hi Mike,
>
> Obviously your BIOS is broken. Pls contact your vendor for help.
>
> Regards,
> Weidong
>

"Dear Valued Customer,
It seems that you are using the Linux system.P5E-VM DO don't Claim support
for Linux.
Best Regards,
Johnson
ASUS Customer Service"

What's odd is that another user with the same motherboard and the same BIOS
revision (0702) is having complete success with VT-D. I'm hoping for some
insight into that little abnormality.

Mike

[-- Attachment #1.2: Type: text/html, Size: 1676 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Han, Weidong]

[-- Attachment #1.1: Type: text/plain, Size: 1139 bytes --]

Really? I found following lines in your messages. The base address of RMRR is larger than the end address. It's an obvious error of your BIOS. Thus VT-d is disabled.

(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff
(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.

________________________________
From: Michael MacLeod [mailto:mikemacleod@gmail.com]
Sent: 2009年9月27日 15:35
To: Han, Weidong
Cc: xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

2009/9/27 Han, Weidong <weidong.han@intel.com<mailto:weidong.han@intel.com>>
Hi Mike,

Obviously your BIOS is broken. Pls contact your vendor for help.

Regards,
Weidong

"Dear Valued Customer,
It seems that you are using the Linux system.P5E-VM DO don't Claim support for Linux.
Best Regards,
Johnson
ASUS Customer Service"

What's odd is that another user with the same motherboard and the same BIOS revision (0702) is having complete success with VT-D. I'm hoping for some insight into that little abnormality.

Mike

[-- Attachment #1.2: Type: text/html, Size: 2961 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Michael MacLeod]

[-- Attachment #1.1: Type: text/plain, Size: 695 bytes --]

2009/9/27 Han, Weidong <weidong.han@intel.com>

>  Really? I found following lines in your messages. The base address of
> RMRR is larger than the end address. It's an obvious error of your BIOS.
> Thus VT-d is disabled.
>

Han,

I'm not denying that my BIOS has an error. The question is whether it's an
error that can be worked around, and why does it affect my particular
motherboard, but not other people who have the same motherboard and the same
BIOS revision? These same other users have demonstrated that the DMAR table
in their BIOS is also corrupt, but that their installation of Xen is able to
cope with it anyway. I'm hoping that Xen can be made to cope with mine too.

Cheers,
Mike

[-- Attachment #1.2: Type: text/html, Size: 1098 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Keir Fraser]

On 27/09/2009 16:35, "Michael MacLeod" <mikemacleod@gmail.com> wrote:

> 2009/9/27 Han, Weidong <weidong.han@intel.com>
>> Really? I found following lines in your messages. The base address of RMRR is
>> larger than the end address. It's an obvious error of your BIOS. Thus VT-d is
>> disabled. 
> 
> Han,
> 
> I'm not denying that my BIOS has an error. The question is whether it's an
> error that can be worked around, and why does it affect my particular
> motherboard, but not other people who have the same motherboard and the same
> BIOS revision? These same other users have demonstrated that the DMAR table in
> their BIOS is also corrupt, but that their installation of Xen is able to cope
> with it anyway. I'm hoping that Xen can be made to cope with mine too.

The particular check that you are failing on in Xen has been in since at
least Xen 3.3.0. If others have the same mobo and BIOS, perhaps they are
using an even earlier version of Xen than that? You can try removing the
check that prints that error message, and see how far you get. But you're
not likely to get ongoing support for that here.

 -- Keir

RE: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Han, Weidong]

Keir Fraser wrote:
> On 27/09/2009 16:35, "Michael MacLeod" <mikemacleod@gmail.com> wrote:
> 
>> 2009/9/27 Han, Weidong <weidong.han@intel.com>
>>> Really? I found following lines in your messages. The base address
>>> of RMRR is larger than the end address. It's an obvious error of
>>> your BIOS. Thus VT-d is disabled.
>> 
>> Han,
>> 
>> I'm not denying that my BIOS has an error. The question is whether
>> it's an error that can be worked around, and why does it affect my
>> particular motherboard, but not other people who have the same
>> motherboard and the same BIOS revision? These same other users have
>> demonstrated that the DMAR table in their BIOS is also corrupt, but
>> that their installation of Xen is able to cope with it anyway. I'm
>> hoping that Xen can be made to cope with mine too. 
> 
> The particular check that you are failing on in Xen has been in since
> at least Xen 3.3.0. If others have the same mobo and BIOS, perhaps
> they are using an even earlier version of Xen than that? You can try
> removing the check that prints that error message, and see how far
> you get. But you're not likely to get ongoing support for that here.
> 
>  -- Keir

Yes, the other people should not succeed on the same motherboard with the same BIOS, unless they used an old xen version without this check. USB controllers and integrated graphics card will use RMRR. I suspect it can work with this broken BIOS even you remove the check and don't disable VT-d.

Regards,
Weidong

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Christian Tramnitz]

[-- Attachment #1: Type: text/plain, Size: 929 bytes --]

Han, Weidong wrote:
> Yes, the other people should not succeed on the same motherboard with the same BIOS, unless they used an old xen version without this check. USB controllers and integrated graphics card will use RMRR. I suspect it can work with this broken BIOS even you remove the check and don't disable VT-d.
> 
> Regards,
> Weidong


I'm the one with working VT-d on this board (using the same BIOS version 
than Michael). I was using an older 3.4.0-pre but now switched to the 
latest 3.5-unstable and its still working. I haven't tried to actually 
pass-through a device for some time but it was already working in the past.

Attached my xm dmesg.

I also agree with Sander's comment in this thread, that we should mark 
"unwilling" vendors in the VT-d section of the Wiki. I'll add notes to 
all Asus boards (basically this one and the P6T Deluxe that I also own 
and had problems with).


Best regards,
   Christian

[-- Attachment #2: xm.dmesg --]
[-- Type: text/plain, Size: 16397 bytes --]

 __  __            _____  ____                     _        _     _      
 \ \/ /___ _ __   |___ / | ___|    _   _ _ __  ___| |_ __ _| |__ | | ___ 
  \  // _ \ \047_ \    |_ \ |___ \ __| | | | \047_ \/ __| __/ _` | \047_ \| |/ _ \
  /  \  __/ | | |  ___) | ___) |__| |_| | | | \__ \ || (_| | |_) | |  __/
 /_/\_\___|_| |_| |____(_)____/    \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
                                                                         
(XEN) Xen version 3.5-unstable (root@epp.trasec.de) (gcc version 4.1.2 (Gentoo 4.1.2 p1.0.2)) Sat Sep 12 11:57:50 CEST 2009
(XEN) Latest ChangeSet: Tue Sep 22 14:19:38 2009 +0100 20247:8b1567102cf3
(XEN) Command line: iommu=1 iommu_inclusive_mapping=1
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000009ec00 (usable)
(XEN)  000000000009ec00 - 00000000000a0000 (reserved)
(XEN)  00000000000e4000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 00000000cf550000 (usable)
(XEN)  00000000cf550000 - 00000000cf55e000 (ACPI data)
(XEN)  00000000cf55e000 - 00000000cf5e0000 (ACPI NVS)
(XEN)  00000000cf5e0000 - 00000000cf600000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ffc00000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000022c000000 (usable)
(XEN) ACPI: RSDP 000F9A80, 0024 (r2 ACPIAM)
(XEN) ACPI: XSDT CF550100, 0064 (r1 A_M_I_ OEMXSDT   3000831 MSFT       97)
(XEN) ACPI: FACP CF550290, 00F4 (r3 A_M_I_ OEMFACP   3000831 MSFT       97)
(XEN) ACPI: DSDT CF5505C0, 843B (r1  A0869 A0869001        1 INTL 20060113)
(XEN) ACPI: FACS CF55E000, 0040
(XEN) ACPI: APIC CF550390, 006C (r1 A_M_I_ OEMAPIC   3000831 MSFT       97)
(XEN) ACPI: MCFG CF550400, 003C (r1 A_M_I_ OEMMCFG   3000831 MSFT       97)
(XEN) ACPI: OEMB CF55E040, 0081 (r1 A_M_I_ AMI_OEM   3000831 MSFT       97)
(XEN) ACPI: HPET CF558A00, 0038 (r1 A_M_I_ OEMHPET   3000831 MSFT       97)
(XEN) ACPI: GSCI CF55E0D0, 2024 (r1 A_M_I_ GMCHSCI   3000831 MSFT       97)
(XEN) ACPI: iEIT CF560100, 00B0 (r1 A_M_I_ EITTABLE  3000831 MSFT       97)
(XEN) ACPI: DMAR CF5601B0, 0140 (r1    AMI  OEMDMAR        1 MSFT       97)
(XEN) System RAM: 8105MB (8299984kB)
(XEN) NUMA turned off
(XEN) Faking a node at 0000000000000000-000000022c000000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000ff780
(XEN) DMI 2.4 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x808
(XEN) ACPI: ACPI SLEEP INFO: pm1x_cnt[804,0], pm1x_evt[800,0]
(XEN) ACPI:                  wakeup_vec[cf55e00c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) Processor #0 6:15 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
(XEN) Processor #1 6:15 APIC version 20
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x82] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x83] disabled)
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0xffffffff base: 0xfed00000
(XEN) [VT-D]dmar.c:537: Host address width 36
(XEN) [VT-D]dmar.c:546: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:373: dmaru->address = fed91000
(XEN) [VT-D]dmar.c:325: endpoint: 0:2.0
(XEN) [VT-D]dmar.c:546: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:373: dmaru->address = fed92000
(XEN) [VT-D]dmar.c:325: endpoint: 0:3.0
(XEN) [VT-D]dmar.c:546: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:373: dmaru->address = fed93000
(XEN) [VT-D]dmar.c:385: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:550: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:550: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:428: RMRR address range not in reserved memory base = cf600000 end = cfffffff; iommu_inclusive_mapping=1 parameter may be needed.
(XEN) [VT-D]dmar.c:325: endpoint: 0:2.0
(XEN) [VT-D]dmar.c:550: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.7
(XEN) Intel VT-d DMAR tables have been parsed.
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 255
(XEN) PCI: Not using MMCONFIG.
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 3000.062 MHz processor.
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 4096K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN) HVM: VMX enabled
(XEN) Intel machine check reporting enabled on CPU#0.
(XEN) CPU0: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU0 has no CMCI support
(XEN) [VT-D]iommu.c:943: drhd->address = fed92000
(XEN) [VT-D]iommu.c:944: iommu->reg = ffff82c3fff57000
(XEN) [VT-D]iommu.c:943: drhd->address = fed91000
(XEN) [VT-D]iommu.c:944: iommu->reg = ffff82c3fff56000
(XEN) [VT-D]iommu.c:943: drhd->address = fed93000
(XEN) [VT-D]iommu.c:944: iommu->reg = ffff82c3fff55000
(XEN) Intel VT-d Snoop Control not supported.
(XEN) Intel VT-d DMA Passthrough not supported.
(XEN) Intel VT-d Queued Invalidation not supported.
(XEN) Intel VT-d Interrupt Remapping not supported.
(XEN) I/O virtualisation enabled
(XEN) I/O virtualisation for PV guests disabled
(XEN) CPU0: Intel(R) Core(TM)2 Duo CPU     E6850  @ 3.00GHz stepping 0b
(XEN) Booting processor 1/1 eip 8c000
(XEN) Initializing CPU#1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 4096K
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) Intel machine check reporting enabled on CPU#1.
(XEN) CPU1: Thermal monitoring enabled (TM2)
(XEN) CMCI: CPU1 has no CMCI support
(XEN) CPU1: Intel(R) Core(TM)2 Duo CPU     E6850  @ 3.00GHz stepping 0b
(XEN) Total of 2 processors activated.
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) checking TSC synchronization across 2 CPUs: passed.
(XEN) Platform timer is 14.318MHz HPET
(XEN) Brought up 2 CPUs
(XEN) microcode.c:73:d32767 microcode: CPU1 resumed
(XEN) HPET: 4 timers in total, 0 timers will be used for broadcast
(XEN) ACPI sleep modes: S3
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:0.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:3.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:19.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1f.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1f.2
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1f.3
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1f.6
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 1:1.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 1:1.1
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 1:1.2
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 1:1.3
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 1:1.4
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 1:1.5
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 1:1.6
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 1:1.7
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:2.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.1
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.2
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1d.7
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.0
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.1
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.2
(XEN) [VT-D]iommu.c:1182:d32767 domain_context_mapping:PCI: bdf = 0:1a.7
(XEN) [VT-D]iommu.c:632: iommu_enable_translation: iommu->reg = ffff82c3fff57000
(XEN) [VT-D]iommu.c:632: iommu_enable_translation: iommu->reg = ffff82c3fff56000
(XEN) [VT-D]iommu.c:716: iommu_page_fault: iommu->reg = ffff82c3fff56000
(XEN) [VT-D]iommu.c:685: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:688: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:670: iommu_fault:DMA Write: 0:2.0 addr 400000000 REASON 5 iommu->reg = ffff82c3fff56000
(XEN) print_vtd_entries: iommu = ffff8302263f30d0 bdf = 0:2:0 gmfn = 400000
(XEN)     root_entry = ffff8302263f0000
(XEN)     root_entry[0] = 225255001
(XEN)     context = ffff830225255000
(XEN)     context[10] = 101_2263c0001
(XEN)     l3 = ffff8302263c0000
(XEN)     l3_index = 10
(XEN)     l3[10] = 0
(XEN)     l3[10] not present
(XEN) [VT-D]iommu.c:632: iommu_enable_translation: iommu->reg = ffff82c3fff55000
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x200000 memsz=0x403000
(XEN) elf_parse_binary: phdr: paddr=0x603000 memsz=0x6ca40
(XEN) elf_parse_binary: phdr: paddr=0x670000 memsz=0x888
(XEN) elf_parse_binary: phdr: paddr=0x672000 memsz=0x1b92d4
(XEN) elf_parse_binary: memory: 0x200000 -> 0x82b2d4
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff80200000
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff80207000
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: INIT_P2M = 0xffffe20000000000
(XEN) elf_xen_parse_note: FEATURES = "writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_addr_calc_check: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0x0
(XEN)     virt_offset      = 0xffffffff80000000
(XEN)     virt_kstart      = 0xffffffff80200000
(XEN)     virt_kend        = 0xffffffff8082b2d4
(XEN)     virt_entry       = 0xffffffff80200000
(XEN)     p2m_base         = 0xffffe20000000000
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, lsb, paddr 0x200000 -> 0x82b2d4
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000224000000->0000000225000000 (2012285 pages to be allocated)
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff80200000->ffffffff8082b2d4
(XEN)  Init. ramdisk: ffffffff8082c000->ffffffff8082c000
(XEN)  Phys-Mach map: ffffe20000000000->ffffe20000f623e8
(XEN)  Start info:    ffffffff8082c000->ffffffff8082c4b4
(XEN)  Page tables:   ffffffff8082d000->ffffffff80836000
(XEN)  Boot stack:    ffffffff80836000->ffffffff80837000
(XEN)  TOTAL:         ffffffff80000000->ffffffff80c00000
(XEN)  ENTRY ADDRESS: ffffffff80200000
(XEN) Dom0 has maximum 2 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff80200000 -> 0xffffffff80603000
(XEN) elf_load_binary: phdr 1 at 0xffffffff80603000 -> 0xffffffff8066fa40
(XEN) elf_load_binary: phdr 2 at 0xffffffff80670000 -> 0xffffffff80670888
(XEN) elf_load_binary: phdr 3 at 0xffffffff80672000 -> 0xffffffff806baa20
(XEN) Scrubbing Free RAM: .done.
(XEN) Xen trace buffers: disabled
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type \047CTRL-a\047 three times to switch input to Xen)
(XEN) Freed 152kB init memory.
(XEN) io_apic.c:2208: 
(XEN) ioapic_guest_write: apic=0, pin=2, irq=0
(XEN) ioapic_guest_write: new_entry=00000900
(XEN) ioapic_guest_write: Attempt to modify IO-APIC pin for in-use IRQ!
(XEN) PCI add device 00:00.0
(XEN) PCI add device 00:02.0
(XEN) PCI add device 00:03.0
(XEN) PCI add device 00:19.0
(XEN) PCI add device 00:1a.0
(XEN) PCI add device 00:1a.1
(XEN) PCI add device 00:1a.2
(XEN) PCI add device 00:1a.7
(XEN) PCI add device 00:1d.0
(XEN) PCI add device 00:1d.1
(XEN) PCI add device 00:1d.2
(XEN) PCI add device 00:1d.7
(XEN) PCI add device 00:1e.0
(XEN) PCI add device 00:1f.0
(XEN) PCI add device 00:1f.2
(XEN) PCI add device 00:1f.3
(XEN) PCI add device 00:1f.6
(XEN) PCI add device 01:01.0
(XEN) Set CPU acpi_id(1) cpuid(0) Px State info:
(XEN) 	_PCT: descriptor=130, length=12, space_id=127, bit_width=64, bit_offset=0, reserved=0, address=409
(XEN) 	_PCT: descriptor=130, length=12, space_id=127, bit_width=16, bit_offset=0, reserved=0, address=408
(XEN) 	_PSS: state_count=2
(XEN) 	State0: 2997MHz 88000mW 10us 10us 0x928 0x928
(XEN) 	State1: 1998MHz 52800mW 10us 10us 0x618 0x618
(XEN) 	_PSD: num_entries=5 rev=0 domain=0 coord_type=254 num_processors=2
(XEN) 	_PPC: 0
(XEN) xen_pminfo: @acpi_cpufreq_cpu_init,HARDWARE addr space
(XEN) CPU 0 initialization completed
(XEN) Set CPU acpi_id(2) cpuid(1) Px State info:
(XEN) 	_PCT: descriptor=130, length=12, space_id=127, bit_width=64, bit_offset=0, reserved=0, address=409
(XEN) 	_PCT: descriptor=130, length=12, space_id=127, bit_width=16, bit_offset=0, reserved=0, address=408
(XEN) 	_PSS: state_count=2
(XEN) 	State0: 2997MHz 88000mW 10us 10us 0x928 0x928
(XEN) 	State1: 1998MHz 52800mW 10us 10us 0x618 0x618
(XEN) 	_PSD: num_entries=5 rev=0 domain=0 coord_type=254 num_processors=2
(XEN) 	_PPC: 0
(XEN) xen_pminfo: @acpi_cpufreq_cpu_init,HARDWARE addr space
(XEN) CPU 1 initialization completed
(XEN) no cpu_id for acpi_id 3
(XEN) no cpu_id for acpi_id 4
(XEN) PCI add device 00:00.0
(XEN) PCI add device 00:19.0
(XEN) allocated vector for irq:20
(XEN) allocated vector for irq:17
(XEN) PCI add device 00:1f.2
(XEN) allocated vector for irq:22
(XEN) PCI add device 00:1a.7
(XEN) allocated vector for irq:18
(XEN) PCI add device 00:1d.7
(XEN) allocated vector for irq:23
(XEN) PCI add device 00:1a.0
(XEN) allocated vector for irq:16
(XEN) PCI add device 00:1a.1
(XEN) allocated vector for irq:21
(XEN) PCI add device 00:1a.2
(XEN) PCI add device 00:1d.0
(XEN) PCI add device 00:1d.1
(XEN) allocated vector for irq:19
(XEN) PCI add device 00:1d.2

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Keir Fraser]

On 28/09/2009 11:38, "Christian Tramnitz" <chris.ace@gmx.net> wrote:

> I'm the one with working VT-d on this board (using the same BIOS version
> than Michael). I was using an older 3.4.0-pre but now switched to the
> latest 3.5-unstable and its still working. I haven't tried to actually
> pass-through a device for some time but it was already working in the past.
> 
> Attached my xm dmesg.

Your BIOS has declared an RMRR range of cf600000-cfffffff, compared with
Sander's invalid d0000000-cfffffff. So the BIOSes must be different, or
differently configured.

 -- Keir

RE: Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Tim Moore]

If I recall from my testing of an ASUS P6T (before I bought an Intel board) enabling/disabling legacy USB, Firewire and the External SATA controller changed the RMRR ranges and produced different results ... suggest try disable legacy USB and Firewire to see.

-----Original Message-----
From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Keir Fraser
Sent: 28 September 2009 13:55
To: Christian Tramnitz; xen-devel@lists.xensource.com
Subject: Re: [Xen-devel] Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

On 28/09/2009 11:38, "Christian Tramnitz" <chris.ace@gmx.net> wrote:

> I'm the one with working VT-d on this board (using the same BIOS version
> than Michael). I was using an older 3.4.0-pre but now switched to the
> latest 3.5-unstable and its still working. I haven't tried to actually
> pass-through a device for some time but it was already working in the past.
> 
> Attached my xm dmesg.

Your BIOS has declared an RMRR range of cf600000-cfffffff, compared with
Sander's invalid d0000000-cfffffff. So the BIOSes must be different, or
differently configured.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Christian Tramnitz]

Tim Moore wrote:
> If I recall from my testing of an ASUS P6T (before I bought an Intel board) enabling/disabling legacy USB, Firewire and the External SATA controller changed the RMRR ranges and produced different results ... suggest try disable legacy USB and Firewire to see.
> 

I already suggested that to Michael, but it didn't help.
I'll just stop by the lab later today and write down my BIOS settings 
for reference. I'm pretty sure I had USB and SATA enabled, Firewire 
disabled but we'll see.


Best regards,
    Christian

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Christian Tramnitz]

Christian Tramnitz wrote:
> I already suggested that to Michael, but it didn't help.
> I'll just stop by the lab later today and write down my BIOS settings 
> for reference. I'm pretty sure I had USB and SATA enabled, Firewire 
> disabled but we'll see.
> 
> 
> Best regards,
>    Christian

Out of curiosity I did it right away, here my details:

Asus P5E-VM DO (product number: 90-MBB7E0-G0EAY00Z)
BIOS: 0702 (dated 03/31/08)
CPU: Intel Core2Duo 6850
RAM: 8GB (reported as 8118MB)
only extension card: an old PCI (not PCIe!) Radeon 7000/VE that is one 
of my test cases for gfx passthrough.


BIOS settings
(this is probably with a lot of unrelated settings, but I wanted to make 
sure the interesting stuff is really there, so I wrote down everything)

MAIN:
Legacy Disk A: disabled
SATA:
- Configure SATA as: AHCI
- HD Write Protect: disabled
- SATA Detect Timeout: 0
AHCI:
- AHCI CD/DVD Boot Timeout: 0
- Ch1 + Ch2: HDD, all other unconnected

ADVANCED:
Jumper Free Settings:
- Ai Overclocking: standard
- all others (CPU Ratio, DRAM Frequency, DRAM Timimg, CPU Spread, PCIE 
Spread): auto
USB:
- USB Functions: enabled
- USB 2.0 Controller: enabled
- USB 2.0 Controller Mode: hispeed
- BIOS EHCI Handoff: enabled
- Port 64/60 Emulation: disabled
- Legacy USB: auto (Tested with USB Keyboard connected and nothing 
connected)
TPM: no
TXT: disabled
VT-d: enabled
CPU:
- Ratio Setting: auto
- C1E Support: enabled
- Max CPUID Limit: disabled
- all others (Vanderpool, CPU TM, Execute bit, Speedstep): enaled
Chipset:
- Northbridge
-- Memory Remap: enabled
-- Initiate Graphic: IGD
-- Internal Mode Select: enabled, 8MB
-- PEG Port Control: auto
-- PEG Force x1: disabled
-- VID Function: DVMT Mode, 256MB
- ME
-- all (ASF, ME Device, ME Bios Extension): disabled
Onboard Devices:
- HD Audio: disabled
- LAN: enabled
- LAN Boot ROM: disabled
- FireWire: disabled
- JMicron PATA: disabled
- Serial1: 3F8/IRQ4
- Parallel: disabled
PCIPNP: Plug&Play OS: yes

POWER:
Suspend: auto
Report on S3: disabled
ACPI 2.0: enabled
ACPI APIC: enabled





Hope that helps.
If you have some settings different dont ask me why I made a choice, it 
was either default or accidental or I intended to disable the specific 
functionality... but it works ;-)


Best regards,
   Christian

Re: Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Michael MacLeod]

[-- Attachment #1.1: Type: text/plain, Size: 3355 bytes --]

On Mon, Sep 28, 2009 at 9:47 AM, Christian Tramnitz <chris.ace@gmx.net>wrote:

> Christian Tramnitz wrote:
>
>> I already suggested that to Michael, but it didn't help.
>> I'll just stop by the lab later today and write down my BIOS settings for
>> reference. I'm pretty sure I had USB and SATA enabled, Firewire disabled but
>> we'll see.
>>
>>
>> Best regards,
>>   Christian
>>
>
> Out of curiosity I did it right away, here my details:
>
> Asus P5E-VM DO (product number: 90-MBB7E0-G0EAY00Z)
> BIOS: 0702 (dated 03/31/08)
> CPU: Intel Core2Duo 6850
> RAM: 8GB (reported as 8118MB)
> only extension card: an old PCI (not PCIe!) Radeon 7000/VE that is one of
> my test cases for gfx passthrough.
>
>
> BIOS settings
> (this is probably with a lot of unrelated settings, but I wanted to make
> sure the interesting stuff is really there, so I wrote down everything)
>
> MAIN:
> Legacy Disk A: disabled
> SATA:
> - Configure SATA as: AHCI
> - HD Write Protect: disabled
> - SATA Detect Timeout: 0
> AHCI:
> - AHCI CD/DVD Boot Timeout: 0
> - Ch1 + Ch2: HDD, all other unconnected
>
> ADVANCED:
> Jumper Free Settings:
> - Ai Overclocking: standard
> - all others (CPU Ratio, DRAM Frequency, DRAM Timimg, CPU Spread, PCIE
> Spread): auto
> USB:
> - USB Functions: enabled
> - USB 2.0 Controller: enabled
> - USB 2.0 Controller Mode: hispeed
> - BIOS EHCI Handoff: enabled
> - Port 64/60 Emulation: disabled
> - Legacy USB: auto (Tested with USB Keyboard connected and nothing
> connected)
> TPM: no
> TXT: disabled
> VT-d: enabled
> CPU:
> - Ratio Setting: auto
> - C1E Support: enabled
> - Max CPUID Limit: disabled
> - all others (Vanderpool, CPU TM, Execute bit, Speedstep): enaled
> Chipset:
> - Northbridge
> -- Memory Remap: enabled
> -- Initiate Graphic: IGD
> -- Internal Mode Select: enabled, 8MB
> -- PEG Port Control: auto
> -- PEG Force x1: disabled
> -- VID Function: DVMT Mode, 256MB
> - ME
> -- all (ASF, ME Device, ME Bios Extension): disabled
> Onboard Devices:
> - HD Audio: disabled
> - LAN: enabled
> - LAN Boot ROM: disabled
> - FireWire: disabled
> - JMicron PATA: disabled
> - Serial1: 3F8/IRQ4
> - Parallel: disabled
> PCIPNP: Plug&Play OS: yes
>
> POWER:
> Suspend: auto
> Report on S3: disabled
> ACPI 2.0: enabled
> ACPI APIC: enabled
>
> Hope that helps.
> If you have some settings different dont ask me why I made a choice, it was
> either default or accidental or I intended to disable the specific
> functionality... but it works ;-)
>
> Best regards,
>  Christian
>

We have a bit of joy now folks. It appears to have been the Initiate
Graphics Adapter setting in the BIOS. It was set to use my PCIe graphics
card as the first option (which is the default value). I set it to use
integrated graphics (Intel 82Q35 Express) and now VT-d is enabled at boot
(even without Dexuan's patch). It's still a BIOS bug, as I believe I should
be able to use a PCIe graphics card and have VT-d work correctly at the same
time.

Unfortunately, 2.6.18.8-xen does not appear to have support for the Intel
graphics card that is the integrated graphics chip, so I lose my X display.
Oh, and when I try and actually launch a domU, I get this: Error: pci: PCI
Backend and pci-stub don't own device 0000:03:00.0. I haven't had time to
play with any of this yet though, so hopefully I can surmount these
obstacles as well.

Thanks everyone,
Mike.

[-- Attachment #1.2: Type: text/html, Size: 4094 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Christian Tramnitz]

Michael MacLeod wrote:
> We have a bit of joy now folks. It appears to have been the Initiate 
> Graphics Adapter setting in the BIOS. It was set to use my PCIe graphics 
> card as the first option (which is the default value). I set it to use 
> integrated graphics (Intel 82Q35 Express) and now VT-d is enabled at 
> boot (even without Dexuan's patch). It's still a BIOS bug, as I believe 
> I should be able to use a PCIe graphics card and have VT-d work 
> correctly at the same time.
> 
> Unfortunately, 2.6.18.8-xen does not appear to have support for the 
> Intel graphics card that is the integrated graphics chip, so I lose my X 
> display. Oh, and when I try and actually launch a domU, I get this: 
> Error: pci: PCI Backend and pci-stub don't own device 0000:03:00.0. I 
> haven't had time to play with any of this yet though, so hopefully I can 
> surmount these obstacles as well.
> 
> Thanks everyone,
> Mike.


I could reproduce the error when completely disabling the IGD, but not 
by changing the init order.
I.e. I could successfully but with my Radeon as primary VGA device (with 
the IGD enabled but as secondary and unused device) and VT-d was still 
enabled.

This could be different when using PCIe because due to the limit of PCIe 
lanes the IGD might be disabled completely when using a PCIe video 
card... not sure about this though, so you may give it a try with IGD 
enabled but PEG/IGD init order.


Best regards,
    Christian

Re: Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Michael MacLeod]

[-- Attachment #1.1: Type: text/plain, Size: 2059 bytes --]

On Wed, Sep 30, 2009 at 7:12 AM, Christian Tramnitz <chris.ace@gmx.net>wrote:

> Michael MacLeod wrote:
>
>> We have a bit of joy now folks. It appears to have been the Initiate
>> Graphics Adapter setting in the BIOS. It was set to use my PCIe graphics
>> card as the first option (which is the default value). I set it to use
>> integrated graphics (Intel 82Q35 Express) and now VT-d is enabled at boot
>> (even without Dexuan's patch). It's still a BIOS bug, as I believe I should
>> be able to use a PCIe graphics card and have VT-d work correctly at the same
>> time.
>>
>> Unfortunately, 2.6.18.8-xen does not appear to have support for the Intel
>> graphics card that is the integrated graphics chip, so I lose my X display.
>> Oh, and when I try and actually launch a domU, I get this: Error: pci: PCI
>> Backend and pci-stub don't own device 0000:03:00.0. I haven't had time to
>> play with any of this yet though, so hopefully I can surmount these
>> obstacles as well.
>>
>> Thanks everyone,
>> Mike.
>>
>
>
> I could reproduce the error when completely disabling the IGD, but not by
> changing the init order.
> I.e. I could successfully but with my Radeon as primary VGA device (with
> the IGD enabled but as secondary and unused device) and VT-d was still
> enabled.
>
> This could be different when using PCIe because due to the limit of PCIe
> lanes the IGD might be disabled completely when using a PCIe video card...
> not sure about this though, so you may give it a try with IGD enabled but
> PEG/IGD init order.
> Best regards,
>   Christian
>

I just tested on my system by setting the bootup option to PEG/IGD and to
PEG/PCI and my DMAR table was corrupt. I haven't checked if PEG/IGD will
result is a corrupt DMAR table is there is no PCIe graphics card present
(thus making IGD the active graphics adapter), as I don't really feel
inclined to pull my system apart.

In summary, with an Asus P5E-VM DO, PEG + PCIe graphics card will result in
VT-d not working correctly.

Anyone have any contact info for people at Asus?

Cheers,
Mike

[-- Attachment #1.2: Type: text/html, Size: 2569 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Han, Weidong]

Christian Tramnitz wrote:
> Han, Weidong wrote:
>> Yes, the other people should not succeed on the same motherboard
>> with the same BIOS, unless they used an old xen version without this
>> check. USB controllers and integrated graphics card will use RMRR. I
>> suspect it can work with this broken BIOS even you remove the check
>> and don't disable VT-d.    
>> 
>> Regards,
>> Weidong
> 
> 
> I'm the one with working VT-d on this board (using the same BIOS
> version than Michael). I was using an older 3.4.0-pre but now
> switched to the latest 3.5-unstable and its still working. I haven't
> tried to actually pass-through a device for some time but it was
> already working in the past. 

As Keir pointed out, RMRR range on your machine is different from Michael's.

> 
> Attached my xm dmesg.
> 
> I also agree with Sander's comment in this thread, that we should mark
> "unwilling" vendors in the VT-d section of the Wiki. I'll add notes to
> all Asus boards (basically this one and the P6T Deluxe that I also own
> and had problems with).

Yes, you can do it. I think it's BIOS issue, pls include BIOS version.

Regards,
Weidong

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Sander Eikelenboom]

Hi Michael,

I have had the same with a asus P5Q-VM DO
It  seems  they  fuck  up  all bioses, and there response is that they
don't  support  anything  other than microsoft stuff. And are not very
sensitive to the fact that it doesn't adhere to the intel vt-d specs.

Commenting  out  a  check on the RMRR address having an "end address >
start address" helps.
But i don't have this problem in xen-unstable anymore, so i think the
check has been relaxt there.

Regards,

Sander


Hello Michael,

Saturday, September 26, 2009, 10:17:52 PM, you wrote:

> I've been trying to get VT-D working with Xen and my Asus P5E-VM DO
> motherboard for the last week. This board is listed on the VTdHowTo xen wiki
> page as supported, and I've found another subscriber of the xen-users list
> who has this configuration working, which makes my predicament all the more
> odd.

> I'm using debian lenny as my dom0 OS, though I've built Xen 3.4.1 from
> source and can replicate this problem with both the lenny 2.6.26-2-xen-amd64
> kernel and the Xen 2.6.18-xen.hg kernel I built. I've tried several
> different BIOS revisions, and the problem is consistent across them.

> Here's some output from my bootup and from acpidump:

> # xm dmesg | grep -C1 VT-D
> (XEN) ACPI: HPET id: 0xffffffff base: 0xfed00000
> (XEN) [VT-D]dmar.c:485: Host address width 36
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed90000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed92000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed93000
> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
> (XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff
> (XEN) Failed to parse ACPI DMAR.  Disabling VT-d.

> # acpidump -t DMAR
> Wrong checksum for OEMB!
> Wrong checksum for !

> # acpidump | grep -B3 -A20 DMAR
> Wrong checksum for OEMB!
> Wrong checksum for !

>  @ 0xcff601b0
>   0000: 00 4d 41 52 48 01 00 00 01 f5 41 4d 49 00 00 00  .MARH.....AMI...
>   0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
>   0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
>   0030: 00 00 18 00 00 00 00 00 00 00 d9 fe 00 00 00 00  ................
>   0040: 01 08 00 00 00 00 1b 00 00 00 28 00 00 00 00 00  ..........(.....
>   0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............
>   0060: 01 08 00 00 00 00 03 02 01 08 00 00 00 00 03 03  ................
>   0070: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
>   0080: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
>   0090: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
>   00a0: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
>   00b0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
>   00c0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
>   00d0: 01 08 00 00 00 00 1a 07 01 00 18 00 00 00 00 00  ................
>   00e0: 00 00 00 d0 00 00 00 00 ff ff ff cf 00 00 00 00  ................
>   00f0: 01 00 58 00 00 00 00 00 00 00 fe cf 00 00 00 00  ..X.............
>   0100: ff ff fe cf 00 00 00 00 01 08 00 00 00 00 1d 00  ................
>   0110: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
>   0120: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
>   0130: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
>   0140: 01 08 00 00 00 00 1a 07                          ........

> I've posted the my full xm dmesg here:
> http://pastebin.com/m32beff18 and you
> can find my full dmesg output here: http://pastebin.com/m74a7dc2a

> The other user of the xen-users list (Christian Tramnitz) who has VT-D
> working with the same motherboard and BIOS revision listed this as the
> output he gets:

> (XEN) [VT-D]iommu.c:722: iommu_page_fault: iommu->reg = ffff828bfff56000
> (XEN) [VT-D]iommu.c:691: iommu_fault_status: Fault Overflow
> (XEN) [VT-D]iommu.c:694: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]iommu.c:676: iommu_fault:DMA Write: 0:2.0 addr 400000000 REASON
5 iommu->>reg = ffff828bfff56000
> (XEN) print_vtd_entries: iommu = ffff83022bde62d0 bdf = 0:2:0 gmfn = 400000
> (XEN)     root_entry = ffff83022bdcf000
> (XEN)     root_entry[0] = 224cc8001
> (XEN)     context = ffff830224cc8000
> (XEN)     context[10] = 101_22bdb3001
> (XEN)     l3 = ffff83022bdb3000
> (XEN)     l3_index = 10
> (XEN)     l3[10] = 0
> (XEN)     l3[10] not present

> # acpidump | grep -B3 -A20 DMAR
> Wrong checksum for OEMB
> Wrong checksum for
> Wrong checksum for OEMB!

> @ 0xcf5601b0
>  0000: 00 4d 41 52 40 01 00 00 01 01 41 4d 49 00 00 00  .MAR@.....AMI...
>  0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
>  0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
>  0030: 00 00 18 00 00 00 00 00 00 10 d9 fe 00 00 00 00  ................
>  0040: 01 08 00 00 00 00 02 00 00 00 18 00 00 00 00 00  ................
>  0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............
>  0060: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
>  0070: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
>  0080: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
>  0090: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
>  00a0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
>  00b0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
>  00c0: 01 08 00 00 00 00 1a 07 01 00 20 00 00 00 00 00  .......... .....
>  00d0: 00 00 60 cf 00 00 00 00 ff ff ff cf 00 00 00 00  ..`.............
>  00e0: 01 08 00 00 00 00 02 00 01 00 58 00 00 00 00 00  ..........X.....
>  00f0: 00 00 5e cf 00 00 00 00 ff ff 5e cf 00 00 00 00  ..^.......^.....
>  0100: 01 08 00 00 00 00 1d 00 01 08 00 00 00 00 1d 01  ................
>  0110: 01 08 00 00 00 00 1d 02 01 08 00 00 00 00 1d 07  ................
>  0120: 01 08 00 00 00 00 1a 00 01 08 00 00 00 00 1a 01  ................
>  0130: 01 08 00 00 00 00 1a 02 01 08 00 00 00 00 1a 07  ................

> XSDT @ 0xcf550100
> Wrong checksum for !

> # acpidump -t DMAR
> Wrong checksum for OEMB
> Wrong checksum for
> Wrong checksum for OEMB!
> Wrong checksum for !

> I've also, on the suggestion of Christian I tried disabling USB in the BIOS
> completely, as he said that he had to do that with a different Asus
> motherboard that also supported VT-D before it would work, but that didn't
> resolve the problem for me.

> Please let me know if there is anything else I can try or any other info I
> can provide. I'd be quite happy to work with a dev to determine the exact
> source of the problem and test patches, etc.

> Thanks,
> Mike



-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Michael MacLeod]

[-- Attachment #1.1: Type: text/plain, Size: 2066 bytes --]

On Sun, Sep 27, 2009 at 3:39 AM, Sander Eikelenboom <linux@eikelenboom.it>wrote:

> Hi Michael,
>
> I have had the same with a asus P5Q-VM DO
> It  seems  they  fuck  up  all bioses, and there response is that they
> don't  support  anything  other than microsoft stuff. And are not very
> sensitive to the fact that it doesn't adhere to the intel vt-d specs.
>
> Commenting  out  a  check on the RMRR address having an "end address >
> start address" helps.
> But i don't have this problem in xen-unstable anymore, so i think the
> check has been relaxt there.
>
> Regards,
>
> Sander


Sander,

I just built xen-unstable from source and gave it a go, but I get pretty
much the same output and still no VT-d. Not being particularly familiar with
the code in question, what's the best way to try and comment out the check
completely?

(XEN) [VT-D]dmar.c:537: Host address width 36
(XEN) [VT-D]dmar.c:546: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:373: dmaru->address = fed90000
(XEN) [VT-D]dmar.c:325: endpoint: 0:1b.0
(XEN) [VT-D]dmar.c:546: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:373: dmaru->address = fed92000
(XEN) [VT-D]dmar.c:325: endpoint: 0:3.0
(XEN) [VT-D]dmar.c:325: endpoint: 0:3.2
(XEN) [VT-D]dmar.c:325: endpoint: 0:3.3
(XEN) [VT-D]dmar.c:546: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:373: dmaru->address = fed93000
(XEN) [VT-D]dmar.c:385: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:550: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.0
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.1
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.2
(XEN) [VT-D]dmar.c:325: endpoint: 0:1d.7
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.0
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.1
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.2
(XEN) [VT-D]dmar.c:325: endpoint: 0:1a.7
(XEN) [VT-D]dmar.c:550: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:415: RMRR error: base_addr d0000000 end_address cfffffff
(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.

Also, what Windows based virtualization software supports VT-d? Something so
I can try leveraging Asus into actually fixing this problem.

Cheers,
Mike

[-- Attachment #1.2: Type: text/html, Size: 2523 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Cui, Dexuan]

Hi Michael,
You can try the workaround patch below (for xen-unstable.hg). Hope it would help.
However, please remember the right solution is pushing your BIOS vendor to fix the buggy BIOS. :-)

Thanks,
-- Dexuan 

diff -r 623aa5c2eaa4 xen/drivers/passthrough/vtd/dmar.c
--- a/xen/drivers/passthrough/vtd/dmar.c    Fri Sep 25 15:20:58 2009 +0100
+++ b/xen/drivers/passthrough/vtd/dmar.c    Mon Sep 28 10:25:33 2009 +0800
@@ -413,7 +413,7 @@ acpi_parse_one_rmrr(struct acpi_dmar_ent
         dprintk(XENLOG_ERR VTDPREFIX,
                 "RMRR error: base_addr %"PRIx64" end_address %"PRIx64"\n",
                 rmrr->base_address, rmrr->end_address);
-        return -EFAULT;
+        rmrr->end_address = 0xFFFFFFFF;
     }
 
 #ifdef CONFIG_X86
diff -r 623aa5c2eaa4 xen/drivers/passthrough/vtd/x86/vtd.c
--- a/xen/drivers/passthrough/vtd/x86/vtd.c Fri Sep 25 15:20:58 2009 +0100
+++ b/xen/drivers/passthrough/vtd/x86/vtd.c Mon Sep 28 10:26:16 2009 +0800
@@ -31,7 +31,7 @@
  * iommu_inclusive_mapping: when set, all memory below 4GB is included in dom0
  * 1:1 iommu mappings except xen and unusable regions.
  */
-static int iommu_inclusive_mapping;
+static int iommu_inclusive_mapping = 1;
 boolean_param("iommu_inclusive_mapping", iommu_inclusive_mapping);
 
 void *map_vtd_domain_page(u64 maddr)


________________________________

From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Michael MacLeod
Sent: 2009?9?27? 4:18
To: xen-devel@lists.xensource.com
Subject: [Xen-devel] VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1


I've been trying to get VT-D working with Xen and my Asus P5E-VM DO motherboard for the last week. This board is listed on the VTdHowTo xen wiki page as supported, and I've found another subscriber of the xen-users list who has this configuration working, which makes my predicament all the more odd.

I'm using debian lenny as my dom0 OS, though I've built Xen 3.4.1 from source and can replicate this problem with both the lenny 2.6.26-2-xen-amd64 kernel and the Xen 2.6.18-xen.hg kernel I built. I've tried several different BIOS revisions, and the problem is consistent across them.

Here's some output from my bootup and from acpidump:

# xm dmesg | grep -C1 VT-D
(XEN) ACPI: HPET id: 0xffffffff base: 0xfed00000
(XEN) [VT-D]dmar.c:485: Host address width 36
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed90000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed92000
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:349: dmaru->address = fed93000
(XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
(XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR

(XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff
(XEN) Failed to parse ACPI DMAR.  Disabling VT-d.


# acpidump -t DMAR
Wrong checksum for OEMB!
Wrong checksum for !

# acpidump | grep -B3 -A20 DMAR
Wrong checksum for OEMB!
Wrong checksum for !

 @ 0xcff601b0
  0000: 00 4d 41 52 48 01 00 00 01 f5 41 4d 49 00 00 00  .MARH.....AMI...
  0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
  0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
  0030: 00 00 18 00 00 00 00 00 00 00 d9 fe 00 00 00 00  ................
  0040: 01 08 00 00 00 00 1b 00 00 00 28 00 00 00 00 00  ..........(.....
  0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............
  0060: 01 08 00 00 00 00 03 02 01 08 00 00 00 00 03 03  ................
  0070: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
  0080: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
  0090: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
  00a0: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
  00b0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
  00c0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
  00d0: 01 08 00 00 00 00 1a 07 01 00 18 00 00 00 00 00  ................
  00e0: 00 00 00 d0 00 00 00 00 ff ff ff cf 00 00 00 00  ................
  00f0: 01 00 58 00 00 00 00 00 00 00 fe cf 00 00 00 00  ..X.............
  0100: ff ff fe cf 00 00 00 00 01 08 00 00 00 00 1d 00  ................
  0110: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
  0120: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
  0130: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
  0140: 01 08 00 00 00 00 1a 07                          ........

I've posted the my full xm dmesg here: http://pastebin.com/m32beff18 and you can find my full dmesg output here: http://pastebin.com/m74a7dc2a

The other user of the xen-users list (Christian Tramnitz) who has VT-D working with the same motherboard and BIOS revision listed this as the output he gets:

(XEN) [VT-D]iommu.c:722: iommu_page_fault: iommu->reg = ffff828bfff56000
(XEN) [VT-D]iommu.c:691: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:694: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]iommu.c:676: iommu_fault:DMA Write: 0:2.0 addr 400000000 REASON 5 iommu->reg = ffff828bfff56000
(XEN) print_vtd_entries: iommu = ffff83022bde62d0 bdf = 0:2:0 gmfn = 400000
(XEN)     root_entry = ffff83022bdcf000
(XEN)     root_entry[0] = 224cc8001
(XEN)     context = ffff830224cc8000
(XEN)     context[10] = 101_22bdb3001
(XEN)     l3 = ffff83022bdb3000
(XEN)     l3_index = 10
(XEN)     l3[10] = 0
(XEN)     l3[10] not present

# acpidump | grep -B3 -A20 DMAR

Wrong checksum for OEMB
Wrong checksum for

Wrong checksum for OEMB!

@ 0xcf5601b0
 0000: 00 4d 41 52 40 01 00 00 01 01 41 4d 49 00 00 00  .MAR@.....AMI... 

 0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
 0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
 0030: 00 00 18 00 00 00 00 00 00 10 d9 fe 00 00 00 00  ................

 0040: 01 08 00 00 00 00 02 00 00 00 18 00 00 00 00 00  ................ 

 0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............

 0060: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
 0070: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
 0080: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
 0090: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
 00a0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
 00b0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
 00c0: 01 08 00 00 00 00 1a 07 01 00 20 00 00 00 00 00  .......... .....
 00d0: 00 00 60 cf 00 00 00 00 ff ff ff cf 00 00 00 00  ..`.............
 00e0: 01 08 00 00 00 00 02 00 01 00 58 00 00 00 00 00  ..........X.....
 00f0: 00 00 5e cf 00 00 00 00 ff ff 5e cf 00 00 00 00  ..^.......^.....
 0100: 01 08 00 00 00 00 1d 00 01 08 00 00 00 00 1d 01  ................
 0110: 01 08 00 00 00 00 1d 02 01 08 00 00 00 00 1d 07  ................
 0120: 01 08 00 00 00 00 1a 00 01 08 00 00 00 00 1a 01  ................
 0130: 01 08 00 00 00 00 1a 02 01 08 00 00 00 00 1a 07  ................

XSDT @ 0xcf550100
Wrong checksum for !

# acpidump -t DMAR
Wrong checksum for OEMB
Wrong checksum for
Wrong checksum for OEMB!
Wrong checksum for !

I've also, on the suggestion of Christian I tried disabling USB in the BIOS completely, as he said that he had to do that with a different Asus motherboard that also supported VT-D before it would work, but that didn't resolve the problem for me.

Please let me know if there is anything else I can try or any other info I can provide. I'd be quite happy to work with a dev to determine the exact source of the problem and test patches, etc.

Thanks,
Mike

Re: VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1

[Sander Eikelenboom]

Hello Dexuan,

Although  it  would be the best, as one little tiny customer you don't
make a very big impression on firms like ASUS (believe me I have tried
..)  and  bios  manufacturers  like  award  phoenix etc. You don't get
passed  customer  support  with there standard denial: only supporting
microsoft windows (so no virtualization what so ever).

So what perhaps could make an impression:

1)  Naming  and  shaming  on  the  VT-D wiki, that these boards do NOT
support it properly. This also warns potential customers.

2)  Pressure  from  Intel, that manufactures who do not implement VT-D
properly  (and/or  not correct it properly if there are problems) are not allowed to mention
it as features of there motherboards anymore. This makes sense because
i'm at the point i'm waiting for AMD to get there IOMMU on the market.
Hope  they  are  somehow  less dependant on bios manufacturers, and it
actually works without these problems.

At  the  moment  buying  an Intel motherboard should be the safest bet,
because although there have been reports of problems with them as well
(according  to  mailing  lists)  they  have  a  larger chance of being
resolved.  But  most  of the times they are not as feature complete as
other boards.

Perhaps  a boot option "relax_iommu_checks" would be handy in the mean
time.

For  the record, ASUS P5Q-EM DO has the same problems, also tested all
available  bioses,  contacted  both  ASUS and there BIOS manufacturer,
with no result what so ever.

Regards,

Sander




Monday, September 28, 2009, 4:33:14 AM, you wrote:

> Hi Michael,
> You can try the workaround patch below (for xen-unstable.hg). Hope it would help.
> However, please remember the right solution is pushing your BIOS vendor to fix the buggy BIOS. :-)

> Thanks,
> -- Dexuan 

> diff -r 623aa5c2eaa4 xen/drivers/passthrough/vtd/dmar.c
> --- a/xen/drivers/passthrough/vtd/dmar.c    Fri Sep 25 15:20:58 2009 +0100
> +++ b/xen/drivers/passthrough/vtd/dmar.c    Mon Sep 28 10:25:33 2009 +0800
> @@ -413,7 +413,7 @@ acpi_parse_one_rmrr(struct acpi_dmar_ent
>          dprintk(XENLOG_ERR VTDPREFIX,
>                  "RMRR error: base_addr %"PRIx64" end_address %"PRIx64"\n",
>                  rmrr->base_address, rmrr->end_address);
> -        return -EFAULT;
+        rmrr->>end_address = 0xFFFFFFFF;
>      }
>  
>  #ifdef CONFIG_X86
> diff -r 623aa5c2eaa4 xen/drivers/passthrough/vtd/x86/vtd.c
> --- a/xen/drivers/passthrough/vtd/x86/vtd.c Fri Sep 25 15:20:58 2009 +0100
> +++ b/xen/drivers/passthrough/vtd/x86/vtd.c Mon Sep 28 10:26:16 2009 +0800
> @@ -31,7 +31,7 @@
>   * iommu_inclusive_mapping: when set, all memory below 4GB is included in dom0
>   * 1:1 iommu mappings except xen and unusable regions.
>   */
> -static int iommu_inclusive_mapping;
> +static int iommu_inclusive_mapping = 1;
>  boolean_param("iommu_inclusive_mapping", iommu_inclusive_mapping);
>  
>  void *map_vtd_domain_page(u64 maddr)


> ________________________________

> From: xen-devel-bounces@lists.xensource.com
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Michael MacLeod
> Sent: 2009?9?27? 4:18
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] VT-D On An Asus P5E-VM DO Motherboard Not Working, Xen 3.4.1


> I've been trying to get VT-D working with Xen and my Asus P5E-VM DO
> motherboard for the last week. This board is listed on the VTdHowTo
> xen wiki page as supported, and I've found another subscriber of the
> xen-users list who has this configuration working, which makes my predicament all the more odd.

> I'm using debian lenny as my dom0 OS, though I've built Xen 3.4.1
> from source and can replicate this problem with both the lenny
> 2.6.26-2-xen-amd64 kernel and the Xen 2.6.18-xen.hg kernel I built.
> I've tried several different BIOS revisions, and the problem is consistent across them.

> Here's some output from my bootup and from acpidump:

> # xm dmesg | grep -C1 VT-D
> (XEN) ACPI: HPET id: 0xffffffff base: 0xfed00000
> (XEN) [VT-D]dmar.c:485: Host address width 36
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed90000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1b.0
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed92000
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:3.3
> (XEN) [VT-D]dmar.c:494: found ACPI_DMAR_DRHD
> (XEN) [VT-D]dmar.c:349: dmaru->address = fed93000
> (XEN) [VT-D]dmar.c:358: found INCLUDE_ALL
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1d.7
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.0
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.1
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.2
> (XEN) [VT-D]dmar.c:300: found endpoint: bdf = 0:1a.7
> (XEN) [VT-D]dmar.c:498: found ACPI_DMAR_RMRR

> (XEN) [VT-D]dmar.c:388: RMRR error: base_addr d0000000 end_address cfffffff
> (XEN) Failed to parse ACPI DMAR.  Disabling VT-d.


> # acpidump -t DMAR
> Wrong checksum for OEMB!
> Wrong checksum for !

> # acpidump | grep -B3 -A20 DMAR
> Wrong checksum for OEMB!
> Wrong checksum for !

>  @ 0xcff601b0
>   0000: 00 4d 41 52 48 01 00 00 01 f5 41 4d 49 00 00 00  .MARH.....AMI...
>   0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
>   0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
>   0030: 00 00 18 00 00 00 00 00 00 00 d9 fe 00 00 00 00  ................
>   0040: 01 08 00 00 00 00 1b 00 00 00 28 00 00 00 00 00  ..........(.....
>   0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............
>   0060: 01 08 00 00 00 00 03 02 01 08 00 00 00 00 03 03  ................
>   0070: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
>   0080: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
>   0090: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
>   00a0: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
>   00b0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
>   00c0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
>   00d0: 01 08 00 00 00 00 1a 07 01 00 18 00 00 00 00 00  ................
>   00e0: 00 00 00 d0 00 00 00 00 ff ff ff cf 00 00 00 00  ................
>   00f0: 01 00 58 00 00 00 00 00 00 00 fe cf 00 00 00 00  ..X.............
>   0100: ff ff fe cf 00 00 00 00 01 08 00 00 00 00 1d 00  ................
>   0110: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
>   0120: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
>   0130: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
>   0140: 01 08 00 00 00 00 1a 07                          ........

> I've posted the my full xm dmesg here:
> http://pastebin.com/m32beff18 and you can find my full dmesg output
> here: http://pastebin.com/m74a7dc2a

> The other user of the xen-users list (Christian Tramnitz) who has
> VT-D working with the same motherboard and BIOS revision listed this as the output he gets:

> (XEN) [VT-D]iommu.c:722: iommu_page_fault: iommu->reg = ffff828bfff56000
> (XEN) [VT-D]iommu.c:691: iommu_fault_status: Fault Overflow
> (XEN) [VT-D]iommu.c:694: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]iommu.c:676: iommu_fault:DMA Write: 0:2.0 addr
> 400000000 REASON 5 iommu->reg = ffff828bfff56000
> (XEN) print_vtd_entries: iommu = ffff83022bde62d0 bdf = 0:2:0 gmfn = 400000
> (XEN)     root_entry = ffff83022bdcf000
> (XEN)     root_entry[0] = 224cc8001
> (XEN)     context = ffff830224cc8000
> (XEN)     context[10] = 101_22bdb3001
> (XEN)     l3 = ffff83022bdb3000
> (XEN)     l3_index = 10
> (XEN)     l3[10] = 0
> (XEN)     l3[10] not present

> # acpidump | grep -B3 -A20 DMAR

> Wrong checksum for OEMB
> Wrong checksum for

> Wrong checksum for OEMB!

> @ 0xcf5601b0
>  0000: 00 4d 41 52 40 01 00 00 01 01 41 4d 49 00 00 00  .MAR@.....AMI...

>  0010: 4f 45 4d 44 4d 41 52 00 01 00 00 00 4d 53 46 54  OEMDMAR.....MSFT
>  0020: 97 00 00 00 23 00 00 00 00 00 00 00 00 00 00 00  ....#...........
>  0030: 00 00 18 00 00 00 00 00 00 10 d9 fe 00 00 00 00  ................

>  0040: 01 08 00 00 00 00 02 00 00 00 18 00 00 00 00 00  ................

>  0050: 00 20 d9 fe 00 00 00 00 01 08 00 00 00 00 03 00  . ..............

>  0060: 00 00 10 00 01 00 00 00 00 30 d9 fe 00 00 00 00  .........0......
>  0070: 01 00 58 00 00 00 00 00 00 d0 0e 00 00 00 00 00  ..X.............
>  0080: ff ff 0e 00 00 00 00 00 01 08 00 00 00 00 1d 00  ................
>  0090: 01 08 00 00 00 00 1d 01 01 08 00 00 00 00 1d 02  ................
>  00a0: 01 08 00 00 00 00 1d 07 01 08 00 00 00 00 1a 00  ................
>  00b0: 01 08 00 00 00 00 1a 01 01 08 00 00 00 00 1a 02  ................
>  00c0: 01 08 00 00 00 00 1a 07 01 00 20 00 00 00 00 00  .......... .....
>  00d0: 00 00 60 cf 00 00 00 00 ff ff ff cf 00 00 00 00  ..`.............
>  00e0: 01 08 00 00 00 00 02 00 01 00 58 00 00 00 00 00  ..........X.....
>  00f0: 00 00 5e cf 00 00 00 00 ff ff 5e cf 00 00 00 00  ..^.......^.....
>  0100: 01 08 00 00 00 00 1d 00 01 08 00 00 00 00 1d 01  ................
>  0110: 01 08 00 00 00 00 1d 02 01 08 00 00 00 00 1d 07  ................
>  0120: 01 08 00 00 00 00 1a 00 01 08 00 00 00 00 1a 01  ................
>  0130: 01 08 00 00 00 00 1a 02 01 08 00 00 00 00 1a 07  ................

> XSDT @ 0xcf550100
> Wrong checksum for !

> # acpidump -t DMAR
> Wrong checksum for OEMB
> Wrong checksum for
> Wrong checksum for OEMB!
> Wrong checksum for !

> I've also, on the suggestion of Christian I tried disabling USB in
> the BIOS completely, as he said that he had to do that with a
> different Asus motherboard that also supported VT-D before it would
> work, but that didn't resolve the problem for me.

> Please let me know if there is anything else I can try or any other
> info I can provide. I'd be quite happy to work with a dev to
> determine the exact source of the problem and test patches, etc.

> Thanks,
> Mike






-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it