Unable to boot Xen 4.8 with iommu=0

Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

Hi all,
I'm having some strange issues with Xen 4.8 when I try to boot with
iommu disabled.

(XEN) Xen version 4.8.0 (root@lan) (gcc (Debian 5.4.1-4) 5.4.1
20161202) debug=n  Tue Feb 14 15:32:55 MST 2017
(XEN) Latest ChangeSet:
(XEN) Bootloader: GRUB 2.02~beta3-4
(XEN) Command line: placeholder dom0_mem=4096M,max:4096M
dom0_max_vcpus=4 dom0_vcpus_pin=true hap_1gb=false hap_2mb=false
altp2m=1 com1=115200,8n1,amt loglvl=all guest_loglvl=all console=com1
flask_enforcing=1 iommu=0
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 3 MBR signatures
(XEN)  Found 3 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)  0000000000000000 - 000000000008f000 (usable)
(XEN)  000000000008f000 - 00000000000a0000 (reserved)
(XEN)  00000000000e0000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 0000000020000000 (usable)
(XEN)  0000000020000000 - 0000000020200000 (reserved)
(XEN)  0000000020200000 - 0000000040000000 (usable)
(XEN)  0000000040000000 - 0000000040200000 (reserved)
(XEN)  0000000040200000 - 000000007a556000 (usable)
(XEN)  000000007a556000 - 000000007a5a5000 (ACPI NVS)
(XEN)  000000007a5a5000 - 000000007a5ae000 (ACPI data)
(XEN)  000000007a5ae000 - 000000007a6a1000 (reserved)
(XEN)  000000007a6a1000 - 000000007a6a3000 (usable)
(XEN)  000000007a6a3000 - 000000007a8c2000 (reserved)
(XEN)  000000007a8c2000 - 000000007a8c3000 (usable)
(XEN)  000000007a8c3000 - 000000007a8d3000 (reserved)
(XEN)  000000007a8d3000 - 000000007a8f1000 (ACPI NVS)
(XEN)  000000007a8f1000 - 000000007a915000 (reserved)
(XEN)  000000007a915000 - 000000007a958000 (ACPI NVS)
(XEN)  000000007a958000 - 000000007ab78000 (reserved)
(XEN)  000000007ab78000 - 000000007ad00000 (usable)
(XEN)  000000007ad00000 - 000000007b000000 (reserved)
(XEN)  000000007b800000 - 000000007fa00000 (reserved)
(XEN)  00000000fed1c000 - 00000000fed40000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000047e600000 (usable)
(XEN) ACPI: RSDP 000F0450, 0024 (r2  INTEL)
(XEN) ACPI: XSDT 7A5A5070, 0064 (r1 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: FACP 7A5ACB50, 00F4 (r4 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: DSDT 7A5A5168, 79E1 (r2 INTEL  DQ67SW         16 INTL 20051117)
(XEN) ACPI: FACS 7A8D7F80, 0040
(XEN) ACPI: APIC 7A5ACC48, 0092 (r3 INTEL  DQ67SW    1072009 AMI     10013)
(XEN) ACPI: TCPA 7A5ACCE0, 0032 (r2 INTEL  DQ67SW          1 MSFT  1000013)
(XEN) ACPI: SSDT 7A5ACD18, 01D6 (r1 INTEL  DQ67SW          1 MSFT  3000001)
(XEN) ACPI: MCFG 7A5ACEF0, 003C (r1 INTEL  DQ67SW    1072009 MSFT       97)
(XEN) ACPI: HPET 7A5ACF30, 0038 (r1 INTEL  DQ67SW    1072009 AMI.        4)
(XEN) ACPI: ASF! 7A5ACF68, 00A0 (r32 INTEL  DQ67SW          1 TFSM    F4240)
(XEN) ACPI: DMAR 7A5AD008, 00E8 (r1 INTEL  DQ67SW          1 INTL        1)
(XEN) System RAM: 16264MB (16654784kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000047e600000
(XEN) Domain heap initialised
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 42 (0x2a), Stepping 7
(raw 000206a7)
(XEN) found SMP MP-table at 000fda00
(XEN) DMI 2.6 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408 (32 bits)
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:404,1:0], pm1x_evt[1:400,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
7a8d7f80/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[7a8d7f8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) XSM Framework v1.0.0 initialized
(XEN) Flask: 128 avtab hash slots, 394 rules.
(XEN) Flask: 128 avtab hash slots, 394 rules.
(XEN) Flask:  5 users, 3 roles, 43 types, 2 bools
(XEN) Flask:  12 classes, 394 rules
(XEN) Flask:  Starting in enforcing mode.
(XEN) xstate: size: 0x340 and states: 0x7
(XEN) Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 3392.326 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802d3f38 -> ffff82d0802d5564
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - 3f
(XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Couldn't enable IOMMU and iommu=required/force
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.

As seen in the command line iommu is not set to required or forced.
Yet Xen thinks it is set to required/force. Has anyone else run into
this issue?

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Andrew Cooper]

On 14/02/2017 22:47, Tamas K Lengyel wrote:
> (XEN) Switched to APIC driver x2apic_cluster.
> (XEN) XSM Framework v1.0.0 initialized
> (XEN) Flask: 128 avtab hash slots, 394 rules.
> (XEN) Flask: 128 avtab hash slots, 394 rules.
> (XEN) Flask:  5 users, 3 roles, 43 types, 2 bools
> (XEN) Flask:  12 classes, 394 rules
> (XEN) Flask:  Starting in enforcing mode.
> (XEN) xstate: size: 0x340 and states: 0x7
> (XEN) Intel machine check reporting enabled
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Platform timer is 14.318MHz HPET
> (XEN) Detected 3392.326 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) alt table ffff82d0802d3f38 -> ffff82d0802d5564
> (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - 3f
> (XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Couldn't enable IOMMU and iommu=required/force
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>
> As seen in the command line iommu is not set to required or forced.
> Yet Xen thinks it is set to required/force. Has anyone else run into
> this issue?

This area is a rats nest :(

The problem is that the APIC setup has chosen to use the x2apic_cluster
driver, which in the Xen code depends on x2APIC, which depends on
interrupt remapping, which depends on an IOMMU.

(I could have sworn we fixed this before), but the bug is that the APIC
setup shouldn't choose any of the x2apic modes if there isn't an
interrupt remapping capable IOMMU.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Jan Beulich]

>>> On 15.02.17 at 00:21, <andrew.cooper3@citrix.com> wrote:
> On 14/02/2017 22:47, Tamas K Lengyel wrote:
>> (XEN) Switched to APIC driver x2apic_cluster.
>> (XEN) XSM Framework v1.0.0 initialized
>> (XEN) Flask: 128 avtab hash slots, 394 rules.
>> (XEN) Flask: 128 avtab hash slots, 394 rules.
>> (XEN) Flask:  5 users, 3 roles, 43 types, 2 bools
>> (XEN) Flask:  12 classes, 394 rules
>> (XEN) Flask:  Starting in enforcing mode.
>> (XEN) xstate: size: 0x340 and states: 0x7
>> (XEN) Intel machine check reporting enabled
>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>> (XEN) Platform timer is 14.318MHz HPET
>> (XEN) Detected 3392.326 MHz processor.
>> (XEN) Initing memory sharing.
>> (XEN) alt table ffff82d0802d3f38 -> ffff82d0802d5564
>> (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - 3f
>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
>> (XEN)
>> (XEN) ****************************************
>> (XEN) Panic on CPU 0:
>> (XEN) Couldn't enable IOMMU and iommu=required/force
>> (XEN) ****************************************
>> (XEN)
>> (XEN) Reboot in five seconds...
>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>>
>> As seen in the command line iommu is not set to required or forced.
>> Yet Xen thinks it is set to required/force. Has anyone else run into
>> this issue?
> 
> This area is a rats nest :(
> 
> The problem is that the APIC setup has chosen to use the x2apic_cluster
> driver, which in the Xen code depends on x2APIC, which depends on
> interrupt remapping, which depends on an IOMMU.
> 
> (I could have sworn we fixed this before), but the bug is that the APIC
> setup shouldn't choose any of the x2apic modes if there isn't an
> interrupt remapping capable IOMMU.

And from going over the code I can't see how this would happen,
so Tamas, it would be nice if you could add some verbosity to the
functions involved. In particular x2apic_bsp_setup() must be
getting success (zero) from iommu_enable_x2apic_IR(), yet that
function calls iommu_supports_eim(), which ought to produce
false through its very first exit path in your case.

Or wait - do you have the same issue if you use
"iommu=no,no-intremap"? In which case the problem would be
that "iommu=no" should clear more than just "iommu_enable", or
code checking iommu_intremap early (before iommu_setup()
manages to clear it in the case here) would need to made look at
both variables. Oddly enough acpi_parse_dmar() only bails if
both variables are clear, which suggests to me that
iommu_enable is intended to have two different meanings in
different contexts (master flag vs. controlling just DMA
remapping). Kevin, Feng - any thoughts here?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

On Wed, Feb 15, 2017 at 3:03 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 15.02.17 at 00:21, <andrew.cooper3@citrix.com> wrote:
>> On 14/02/2017 22:47, Tamas K Lengyel wrote:
>>> (XEN) Switched to APIC driver x2apic_cluster.
>>> (XEN) XSM Framework v1.0.0 initialized
>>> (XEN) Flask: 128 avtab hash slots, 394 rules.
>>> (XEN) Flask: 128 avtab hash slots, 394 rules.
>>> (XEN) Flask:  5 users, 3 roles, 43 types, 2 bools
>>> (XEN) Flask:  12 classes, 394 rules
>>> (XEN) Flask:  Starting in enforcing mode.
>>> (XEN) xstate: size: 0x340 and states: 0x7
>>> (XEN) Intel machine check reporting enabled
>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>>> (XEN) Platform timer is 14.318MHz HPET
>>> (XEN) Detected 3392.326 MHz processor.
>>> (XEN) Initing memory sharing.
>>> (XEN) alt table ffff82d0802d3f38 -> ffff82d0802d5564
>>> (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - 3f
>>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
>>> (XEN)
>>> (XEN) ****************************************
>>> (XEN) Panic on CPU 0:
>>> (XEN) Couldn't enable IOMMU and iommu=required/force
>>> (XEN) ****************************************
>>> (XEN)
>>> (XEN) Reboot in five seconds...
>>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>>>
>>> As seen in the command line iommu is not set to required or forced.
>>> Yet Xen thinks it is set to required/force. Has anyone else run into
>>> this issue?
>>
>> This area is a rats nest :(
>>
>> The problem is that the APIC setup has chosen to use the x2apic_cluster
>> driver, which in the Xen code depends on x2APIC, which depends on
>> interrupt remapping, which depends on an IOMMU.
>>
>> (I could have sworn we fixed this before), but the bug is that the APIC
>> setup shouldn't choose any of the x2apic modes if there isn't an
>> interrupt remapping capable IOMMU.
>
> And from going over the code I can't see how this would happen,
> so Tamas, it would be nice if you could add some verbosity to the
> functions involved. In particular x2apic_bsp_setup() must be
> getting success (zero) from iommu_enable_x2apic_IR(), yet that
> function calls iommu_supports_eim(), which ought to produce
> false through its very first exit path in your case.
>
> Or wait - do you have the same issue if you use
> "iommu=no,no-intremap"? In which case the problem would be
> that "iommu=no" should clear more than just "iommu_enable", or
> code checking iommu_intremap early (before iommu_setup()
> manages to clear it in the case here) would need to made look at
> both variables. Oddly enough acpi_parse_dmar() only bails if
> both variables are clear, which suggests to me that
> iommu_enable is intended to have two different meanings in
> different contexts (master flag vs. controlling just DMA
> remapping). Kevin, Feng - any thoughts here?

iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

On Wed, Feb 15, 2017 at 12:30 PM, Tamas K Lengyel
<tamas.k.lengyel@gmail.com> wrote:
> On Wed, Feb 15, 2017 at 3:03 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 15.02.17 at 00:21, <andrew.cooper3@citrix.com> wrote:
>>> On 14/02/2017 22:47, Tamas K Lengyel wrote:
>>>> (XEN) Switched to APIC driver x2apic_cluster.
>>>> (XEN) XSM Framework v1.0.0 initialized
>>>> (XEN) Flask: 128 avtab hash slots, 394 rules.
>>>> (XEN) Flask: 128 avtab hash slots, 394 rules.
>>>> (XEN) Flask:  5 users, 3 roles, 43 types, 2 bools
>>>> (XEN) Flask:  12 classes, 394 rules
>>>> (XEN) Flask:  Starting in enforcing mode.
>>>> (XEN) xstate: size: 0x340 and states: 0x7
>>>> (XEN) Intel machine check reporting enabled
>>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>>>> (XEN) Platform timer is 14.318MHz HPET
>>>> (XEN) Detected 3392.326 MHz processor.
>>>> (XEN) Initing memory sharing.
>>>> (XEN) alt table ffff82d0802d3f38 -> ffff82d0802d5564
>>>> (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - 3f
>>>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
>>>> (XEN)
>>>> (XEN) ****************************************
>>>> (XEN) Panic on CPU 0:
>>>> (XEN) Couldn't enable IOMMU and iommu=required/force
>>>> (XEN) ****************************************
>>>> (XEN)
>>>> (XEN) Reboot in five seconds...
>>>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>>>>
>>>> As seen in the command line iommu is not set to required or forced.
>>>> Yet Xen thinks it is set to required/force. Has anyone else run into
>>>> this issue?
>>>
>>> This area is a rats nest :(
>>>
>>> The problem is that the APIC setup has chosen to use the x2apic_cluster
>>> driver, which in the Xen code depends on x2APIC, which depends on
>>> interrupt remapping, which depends on an IOMMU.
>>>
>>> (I could have sworn we fixed this before), but the bug is that the APIC
>>> setup shouldn't choose any of the x2apic modes if there isn't an
>>> interrupt remapping capable IOMMU.
>>
>> And from going over the code I can't see how this would happen,
>> so Tamas, it would be nice if you could add some verbosity to the
>> functions involved. In particular x2apic_bsp_setup() must be
>> getting success (zero) from iommu_enable_x2apic_IR(), yet that
>> function calls iommu_supports_eim(), which ought to produce
>> false through its very first exit path in your case.
>>
>> Or wait - do you have the same issue if you use
>> "iommu=no,no-intremap"? In which case the problem would be
>> that "iommu=no" should clear more than just "iommu_enable", or
>> code checking iommu_intremap early (before iommu_setup()
>> manages to clear it in the case here) would need to made look at
>> both variables. Oddly enough acpi_parse_dmar() only bails if
>> both variables are clear, which suggests to me that
>> iommu_enable is intended to have two different meanings in
>> different contexts (master flag vs. controlling just DMA
>> remapping). Kevin, Feng - any thoughts here?
>
> iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"

And just to clarify, x2apic is disabled in this case "(XEN) Not
enabling x2APIC: depends on iommu_supports_eim."

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Jan Beulich]

>>> On 15.02.17 at 20:30, <tamas.k.lengyel@gmail.com> wrote:
> On Wed, Feb 15, 2017 at 3:03 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>>> On 15.02.17 at 00:21, <andrew.cooper3@citrix.com> wrote:
>>> On 14/02/2017 22:47, Tamas K Lengyel wrote:
>>>> (XEN) Switched to APIC driver x2apic_cluster.
>>>> (XEN) XSM Framework v1.0.0 initialized
>>>> (XEN) Flask: 128 avtab hash slots, 394 rules.
>>>> (XEN) Flask: 128 avtab hash slots, 394 rules.
>>>> (XEN) Flask:  5 users, 3 roles, 43 types, 2 bools
>>>> (XEN) Flask:  12 classes, 394 rules
>>>> (XEN) Flask:  Starting in enforcing mode.
>>>> (XEN) xstate: size: 0x340 and states: 0x7
>>>> (XEN) Intel machine check reporting enabled
>>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
>>>> (XEN) Platform timer is 14.318MHz HPET
>>>> (XEN) Detected 3392.326 MHz processor.
>>>> (XEN) Initing memory sharing.
>>>> (XEN) alt table ffff82d0802d3f38 -> ffff82d0802d5564
>>>> (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - 3f
>>>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
>>>> (XEN)
>>>> (XEN) ****************************************
>>>> (XEN) Panic on CPU 0:
>>>> (XEN) Couldn't enable IOMMU and iommu=required/force
>>>> (XEN) ****************************************
>>>> (XEN)
>>>> (XEN) Reboot in five seconds...
>>>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>>>>
>>>> As seen in the command line iommu is not set to required or forced.
>>>> Yet Xen thinks it is set to required/force. Has anyone else run into
>>>> this issue?
>>>
>>> This area is a rats nest :(
>>>
>>> The problem is that the APIC setup has chosen to use the x2apic_cluster
>>> driver, which in the Xen code depends on x2APIC, which depends on
>>> interrupt remapping, which depends on an IOMMU.
>>>
>>> (I could have sworn we fixed this before), but the bug is that the APIC
>>> setup shouldn't choose any of the x2apic modes if there isn't an
>>> interrupt remapping capable IOMMU.
>>
>> And from going over the code I can't see how this would happen,
>> so Tamas, it would be nice if you could add some verbosity to the
>> functions involved. In particular x2apic_bsp_setup() must be
>> getting success (zero) from iommu_enable_x2apic_IR(), yet that
>> function calls iommu_supports_eim(), which ought to produce
>> false through its very first exit path in your case.
>>
>> Or wait - do you have the same issue if you use
>> "iommu=no,no-intremap"? In which case the problem would be
>> that "iommu=no" should clear more than just "iommu_enable", or
>> code checking iommu_intremap early (before iommu_setup()
>> manages to clear it in the case here) would need to made look at
>> both variables. Oddly enough acpi_parse_dmar() only bails if
>> both variables are clear, which suggests to me that
>> iommu_enable is intended to have two different meanings in
>> different contexts (master flag vs. controlling just DMA
>> remapping). Kevin, Feng - any thoughts here?
> 
> iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"

Thanks for confirming.

Kevin, Feng, we now depend on your input regarding the intentions
with the two variables.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tian, Kevin]

> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Thursday, February 16, 2017 5:32 PM
> 
> >>> On 15.02.17 at 20:30, <tamas.k.lengyel@gmail.com> wrote:
> > On Wed, Feb 15, 2017 at 3:03 AM, Jan Beulich <JBeulich@suse.com> wrote:
> >>>>> On 15.02.17 at 00:21, <andrew.cooper3@citrix.com> wrote:
> >>> On 14/02/2017 22:47, Tamas K Lengyel wrote:
> >>>> (XEN) Switched to APIC driver x2apic_cluster.
> >>>> (XEN) XSM Framework v1.0.0 initialized
> >>>> (XEN) Flask: 128 avtab hash slots, 394 rules.
> >>>> (XEN) Flask: 128 avtab hash slots, 394 rules.
> >>>> (XEN) Flask:  5 users, 3 roles, 43 types, 2 bools
> >>>> (XEN) Flask:  12 classes, 394 rules
> >>>> (XEN) Flask:  Starting in enforcing mode.
> >>>> (XEN) xstate: size: 0x340 and states: 0x7
> >>>> (XEN) Intel machine check reporting enabled
> >>>> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> >>>> (XEN) Platform timer is 14.318MHz HPET
> >>>> (XEN) Detected 3392.326 MHz processor.
> >>>> (XEN) Initing memory sharing.
> >>>> (XEN) alt table ffff82d0802d3f38 -> ffff82d0802d5564
> >>>> (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - 3f
> >>>> (XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
> >>>> (XEN)
> >>>> (XEN) ****************************************
> >>>> (XEN) Panic on CPU 0:
> >>>> (XEN) Couldn't enable IOMMU and iommu=required/force
> >>>> (XEN) ****************************************
> >>>> (XEN)
> >>>> (XEN) Reboot in five seconds...
> >>>> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
> >>>>
> >>>> As seen in the command line iommu is not set to required or forced.
> >>>> Yet Xen thinks it is set to required/force. Has anyone else run into
> >>>> this issue?
> >>>
> >>> This area is a rats nest :(
> >>>
> >>> The problem is that the APIC setup has chosen to use the x2apic_cluster
> >>> driver, which in the Xen code depends on x2APIC, which depends on
> >>> interrupt remapping, which depends on an IOMMU.
> >>>
> >>> (I could have sworn we fixed this before), but the bug is that the APIC
> >>> setup shouldn't choose any of the x2apic modes if there isn't an
> >>> interrupt remapping capable IOMMU.
> >>
> >> And from going over the code I can't see how this would happen,
> >> so Tamas, it would be nice if you could add some verbosity to the
> >> functions involved. In particular x2apic_bsp_setup() must be
> >> getting success (zero) from iommu_enable_x2apic_IR(), yet that
> >> function calls iommu_supports_eim(), which ought to produce
> >> false through its very first exit path in your case.
> >>
> >> Or wait - do you have the same issue if you use
> >> "iommu=no,no-intremap"? In which case the problem would be
> >> that "iommu=no" should clear more than just "iommu_enable", or
> >> code checking iommu_intremap early (before iommu_setup()
> >> manages to clear it in the case here) would need to made look at
> >> both variables. Oddly enough acpi_parse_dmar() only bails if
> >> both variables are clear, which suggests to me that
> >> iommu_enable is intended to have two different meanings in
> >> different contexts (master flag vs. controlling just DMA
> >> remapping). Kevin, Feng - any thoughts here?
> >
> > iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"
> 
> Thanks for confirming.
> 
> Kevin, Feng, we now depend on your input regarding the intentions
> with the two variables.
> 

Feng just left Intel. Let me take a look at code to understand the
rationale behind.

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tian, Kevin]

> From: Tian, Kevin
> Sent: Friday, February 17, 2017 11:35 AM
> > >>
> > >> Or wait - do you have the same issue if you use
> > >> "iommu=no,no-intremap"? In which case the problem would be
> > >> that "iommu=no" should clear more than just "iommu_enable", or
> > >> code checking iommu_intremap early (before iommu_setup()
> > >> manages to clear it in the case here) would need to made look at
> > >> both variables. Oddly enough acpi_parse_dmar() only bails if
> > >> both variables are clear, which suggests to me that
> > >> iommu_enable is intended to have two different meanings in
> > >> different contexts (master flag vs. controlling just DMA
> > >> remapping). Kevin, Feng - any thoughts here?
> > >
> > > iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"
> >
> > Thanks for confirming.
> >
> > Kevin, Feng, we now depend on your input regarding the intentions
> > with the two variables.
> >
> 
> Feng just left Intel. Let me take a look at code to understand the
> rationale behind.
> 

Jan, looks it's caused by your change back to 2012:

commit 7a8f6d0607a38c64506b4e8b473d955bf8e2a71f
Author: Jan Beulich <jbeulich@suse.com>
Date:   Fri Nov 2 17:15:30 2012 +0100

Before that iommu_enable was the master flag consistently. I'm still
trying to understand the background and you may help elaborate if
still something in your memory.

I agree we should stick to one meaning after clearing up above issue.

Given this commit is pretty old, I'm also curious why it's only reported
on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
to be the one upon which you first tried iommu=0 on a platform supporting
interrupt remapping?

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Jan Beulich]

>>> On 17.02.17 at 04:35, <kevin.tian@intel.com> wrote:
> Feng just left Intel. Let me take a look at code to understand the
> rationale behind.

Do you know if he means to stay VT-d maintainer? If yes, do you
have a new email address of him? If not, would you mind sending
an update to ./MAINTAINERS removing him?

Thanks, Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Jan Beulich]

>>> On 17.02.17 at 07:53, <kevin.tian@intel.com> wrote:
>>  From: Tian, Kevin
>> Sent: Friday, February 17, 2017 11:35 AM
>> > >>
>> > >> Or wait - do you have the same issue if you use
>> > >> "iommu=no,no-intremap"? In which case the problem would be
>> > >> that "iommu=no" should clear more than just "iommu_enable", or
>> > >> code checking iommu_intremap early (before iommu_setup()
>> > >> manages to clear it in the case here) would need to made look at
>> > >> both variables. Oddly enough acpi_parse_dmar() only bails if
>> > >> both variables are clear, which suggests to me that
>> > >> iommu_enable is intended to have two different meanings in
>> > >> different contexts (master flag vs. controlling just DMA
>> > >> remapping). Kevin, Feng - any thoughts here?
>> > >
>> > > iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"
>> >
>> > Thanks for confirming.
>> >
>> > Kevin, Feng, we now depend on your input regarding the intentions
>> > with the two variables.
>> >
>> 
>> Feng just left Intel. Let me take a look at code to understand the
>> rationale behind.
>> 
> 
> Jan, looks it's caused by your change back to 2012:
> 
> commit 7a8f6d0607a38c64506b4e8b473d955bf8e2a71f
> Author: Jan Beulich <jbeulich@suse.com>
> Date:   Fri Nov 2 17:15:30 2012 +0100

Oh, I see. That's been a while; I'm sorry for not having remembered
and bugging you.

> Before that iommu_enable was the master flag consistently. I'm still
> trying to understand the background and you may help elaborate if
> still something in your memory.

I think the commit description is pretty clear. Especially this part

"This could have the nice side effect of allowing to use "iommu=off"
 even when x2APIC was pre-enabled by the BIOS (in which case interrupt
 remapping is a requirement, but DMA translation [obviously] isn't), but
 that doesn't currently work (and hence x2apic_bsp_setup() forces the
 IOMMU on rather than just interrupt remapping)."

is quite relevant in the context here. Just like Linux, we really ought
to have a way to run with interrupt remapping, but without DMA
remapping. So I think this old commit was a half-hearted step into
that direction, not recognizing that it would break some other case.

So I think the only reasonable way forward to address Tamas's
issue is to fully disentangle DMA and interrupt remapping setup,
which I'm sure is going to take some time. As a minimal adjustment,
though, I wonder whether that old commit's adjustment to apic.c
shouldn't rather have set force_intremap. Tamas - could you check
whether

--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -975,7 +975,7 @@ void __init x2apic_bsp_setup(void)
         goto restore_out;
     }
 
-    force_iommu = 1;
+    force_intremap = 1;
 
     genapic = apic_x2apic_probe();
     printk("Switched to APIC driver %s.\n", genapic->name);

makes things any better?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

On Fri, Feb 17, 2017 at 1:20 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 17.02.17 at 07:53, <kevin.tian@intel.com> wrote:
>>>  From: Tian, Kevin
>>> Sent: Friday, February 17, 2017 11:35 AM
>>> > >>
>>> > >> Or wait - do you have the same issue if you use
>>> > >> "iommu=no,no-intremap"? In which case the problem would be
>>> > >> that "iommu=no" should clear more than just "iommu_enable", or
>>> > >> code checking iommu_intremap early (before iommu_setup()
>>> > >> manages to clear it in the case here) would need to made look at
>>> > >> both variables. Oddly enough acpi_parse_dmar() only bails if
>>> > >> both variables are clear, which suggests to me that
>>> > >> iommu_enable is intended to have two different meanings in
>>> > >> different contexts (master flag vs. controlling just DMA
>>> > >> remapping). Kevin, Feng - any thoughts here?
>>> > >
>>> > > iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"
>>> >
>>> > Thanks for confirming.
>>> >
>>> > Kevin, Feng, we now depend on your input regarding the intentions
>>> > with the two variables.
>>> >
>>>
>>> Feng just left Intel. Let me take a look at code to understand the
>>> rationale behind.
>>>
>>
>> Jan, looks it's caused by your change back to 2012:
>>
>> commit 7a8f6d0607a38c64506b4e8b473d955bf8e2a71f
>> Author: Jan Beulich <jbeulich@suse.com>
>> Date:   Fri Nov 2 17:15:30 2012 +0100
>
> Oh, I see. That's been a while; I'm sorry for not having remembered
> and bugging you.
>
>> Before that iommu_enable was the master flag consistently. I'm still
>> trying to understand the background and you may help elaborate if
>> still something in your memory.
>
> I think the commit description is pretty clear. Especially this part
>
> "This could have the nice side effect of allowing to use "iommu=off"
>  even when x2APIC was pre-enabled by the BIOS (in which case interrupt
>  remapping is a requirement, but DMA translation [obviously] isn't), but
>  that doesn't currently work (and hence x2apic_bsp_setup() forces the
>  IOMMU on rather than just interrupt remapping)."
>
> is quite relevant in the context here. Just like Linux, we really ought
> to have a way to run with interrupt remapping, but without DMA
> remapping. So I think this old commit was a half-hearted step into
> that direction, not recognizing that it would break some other case.
>
> So I think the only reasonable way forward to address Tamas's
> issue is to fully disentangle DMA and interrupt remapping setup,
> which I'm sure is going to take some time. As a minimal adjustment,
> though, I wonder whether that old commit's adjustment to apic.c
> shouldn't rather have set force_intremap. Tamas - could you check
> whether
>
> --- a/xen/arch/x86/apic.c
> +++ b/xen/arch/x86/apic.c
> @@ -975,7 +975,7 @@ void __init x2apic_bsp_setup(void)
>          goto restore_out;
>      }
>
> -    force_iommu = 1;
> +    force_intremap = 1;
>
>      genapic = apic_x2apic_probe();
>      printk("Switched to APIC driver %s.\n", genapic->name);
>
> makes things any better?
>
> Jan

Just making this change won't compile:

apic.c: In function ‘x2apic_bsp_setup’:
apic.c:947:5: error: ‘force_intremap’ undeclared (first use in this function)
     force_intremap = 1;
     ^

Had to also add

diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index d793f5de1a..a45578608b 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -52,6 +52,7 @@ custom_param("iommu", parse_iommu_param);
 bool_t __initdata iommu_enable = 1;
 bool_t __read_mostly iommu_enabled;
 bool_t __read_mostly force_iommu;
+bool_t __read_mostly force_intremap;
 bool_t __hwdom_initdata iommu_dom0_strict;
 bool_t __read_mostly iommu_verbose;
 bool_t __read_mostly iommu_workaround_bios_bug;
@@ -364,7 +365,7 @@ int iommu_iotlb_flush_all(struct domain *d)
 int __init iommu_setup(void)
 {
     int rc = -ENODEV;
-    bool_t force_intremap = force_iommu && iommu_intremap;
+    force_intremap = force_iommu && iommu_intremap;

     if ( iommu_dom0_strict )
         iommu_passthrough = 0;
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 5803e3f95b..770e4f6282 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -29,6 +29,7 @@

 extern bool_t iommu_enable, iommu_enabled;
 extern bool_t force_iommu, iommu_verbose;
+extern bool_t force_intremap;
 extern bool_t iommu_workaround_bios_bug, iommu_igfx, iommu_passthrough;
 extern bool_t iommu_snoop, iommu_qinval, iommu_intremap, iommu_intpost;
 extern bool_t iommu_hap_pt_share;

With these changes in place the system starts to boot but dom0 fails
to assemble the mdraid disk and find root:

Begin: Running /scripts/local-block ... mdadm: No devices listed in
conf file were found.
  WARNING: Failed to connect to lvmetad. Falling back to device scanning.
  Volume group "vg" not found
  Cannot process volume group vg
done.
done.
Gave up waiting for root device.  Common problems:
 - Boot args (cat /proc/cmdline)
   - Check rootdelay= (did the system wait long enough?)
   - Check root= (did the system wait for the right device?)
 - Missing modules (cat /proc/modules; ls /dev)
ALERT!  /dev/mapper/vg-root does not exist.  Dropping to a shell!

Also, I see Xen still using the x2apic_cluster:

(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) Switched to APIC driver x2apic_cluster.

With iommu=1 everything works as normal.

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

On Thu, Feb 16, 2017 at 11:53 PM, Tian, Kevin <kevin.tian@intel.com> wrote:
>> From: Tian, Kevin
>> Sent: Friday, February 17, 2017 11:35 AM
>> > >>
>> > >> Or wait - do you have the same issue if you use
>> > >> "iommu=no,no-intremap"? In which case the problem would be
>> > >> that "iommu=no" should clear more than just "iommu_enable", or
>> > >> code checking iommu_intremap early (before iommu_setup()
>> > >> manages to clear it in the case here) would need to made look at
>> > >> both variables. Oddly enough acpi_parse_dmar() only bails if
>> > >> both variables are clear, which suggests to me that
>> > >> iommu_enable is intended to have two different meanings in
>> > >> different contexts (master flag vs. controlling just DMA
>> > >> remapping). Kevin, Feng - any thoughts here?
>> > >
>> > > iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"
>> >
>> > Thanks for confirming.
>> >
>> > Kevin, Feng, we now depend on your input regarding the intentions
>> > with the two variables.
>> >
>>
>> Feng just left Intel. Let me take a look at code to understand the
>> rationale behind.
>>
>
> Jan, looks it's caused by your change back to 2012:
>
> commit 7a8f6d0607a38c64506b4e8b473d955bf8e2a71f
> Author: Jan Beulich <jbeulich@suse.com>
> Date:   Fri Nov 2 17:15:30 2012 +0100
>
> Before that iommu_enable was the master flag consistently. I'm still
> trying to understand the background and you may help elaborate if
> still something in your memory.
>
> I agree we should stick to one meaning after clearing up above issue.
>
> Given this commit is pretty old, I'm also curious why it's only reported
> on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
> to be the one upon which you first tried iommu=0 on a platform supporting
> interrupt remapping?

It just happens to be the one I tried. VT-d was spamming my console
with faults like this:

(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
277e28000, iommu reg = ffff82c000201000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set

So I figured I can just turn it off to clean up my console. I still
don't know what these VT-d faults are about..

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Konrad Rzeszutek Wilk]

On Fri, Feb 17, 2017 at 11:42:20AM -0700, Tamas K Lengyel wrote:
> On Thu, Feb 16, 2017 at 11:53 PM, Tian, Kevin <kevin.tian@intel.com> wrote:
> >> From: Tian, Kevin
> >> Sent: Friday, February 17, 2017 11:35 AM
> >> > >>
> >> > >> Or wait - do you have the same issue if you use
> >> > >> "iommu=no,no-intremap"? In which case the problem would be
> >> > >> that "iommu=no" should clear more than just "iommu_enable", or
> >> > >> code checking iommu_intremap early (before iommu_setup()
> >> > >> manages to clear it in the case here) would need to made look at
> >> > >> both variables. Oddly enough acpi_parse_dmar() only bails if
> >> > >> both variables are clear, which suggests to me that
> >> > >> iommu_enable is intended to have two different meanings in
> >> > >> different contexts (master flag vs. controlling just DMA
> >> > >> remapping). Kevin, Feng - any thoughts here?
> >> > >
> >> > > iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"
> >> >
> >> > Thanks for confirming.
> >> >
> >> > Kevin, Feng, we now depend on your input regarding the intentions
> >> > with the two variables.
> >> >
> >>
> >> Feng just left Intel. Let me take a look at code to understand the
> >> rationale behind.
> >>
> >
> > Jan, looks it's caused by your change back to 2012:
> >
> > commit 7a8f6d0607a38c64506b4e8b473d955bf8e2a71f
> > Author: Jan Beulich <jbeulich@suse.com>
> > Date:   Fri Nov 2 17:15:30 2012 +0100
> >
> > Before that iommu_enable was the master flag consistently. I'm still
> > trying to understand the background and you may help elaborate if
> > still something in your memory.
> >
> > I agree we should stick to one meaning after clearing up above issue.
> >
> > Given this commit is pretty old, I'm also curious why it's only reported
> > on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
> > to be the one upon which you first tried iommu=0 on a platform supporting
> > interrupt remapping?
> 
> It just happens to be the one I tried. VT-d was spamming my console
> with faults like this:
> 
> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
> 277e28000, iommu reg = ffff82c000201000
> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
> 
> So I figured I can just turn it off to clean up my console. I still
> don't know what these VT-d faults are about..

What is the 0:02.0 device? Is it being passed in to your guest?
> 
> Tamas
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

On Fri, Feb 17, 2017 at 11:48 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Fri, Feb 17, 2017 at 11:42:20AM -0700, Tamas K Lengyel wrote:
>> On Thu, Feb 16, 2017 at 11:53 PM, Tian, Kevin <kevin.tian@intel.com> wrote:
>> >> From: Tian, Kevin
>> >> Sent: Friday, February 17, 2017 11:35 AM
>> >> > >>
>> >> > >> Or wait - do you have the same issue if you use
>> >> > >> "iommu=no,no-intremap"? In which case the problem would be
>> >> > >> that "iommu=no" should clear more than just "iommu_enable", or
>> >> > >> code checking iommu_intremap early (before iommu_setup()
>> >> > >> manages to clear it in the case here) would need to made look at
>> >> > >> both variables. Oddly enough acpi_parse_dmar() only bails if
>> >> > >> both variables are clear, which suggests to me that
>> >> > >> iommu_enable is intended to have two different meanings in
>> >> > >> different contexts (master flag vs. controlling just DMA
>> >> > >> remapping). Kevin, Feng - any thoughts here?
>> >> > >
>> >> > > iommu=no,no-intremap boots fine with "(XEN) Using APIC driver default"
>> >> >
>> >> > Thanks for confirming.
>> >> >
>> >> > Kevin, Feng, we now depend on your input regarding the intentions
>> >> > with the two variables.
>> >> >
>> >>
>> >> Feng just left Intel. Let me take a look at code to understand the
>> >> rationale behind.
>> >>
>> >
>> > Jan, looks it's caused by your change back to 2012:
>> >
>> > commit 7a8f6d0607a38c64506b4e8b473d955bf8e2a71f
>> > Author: Jan Beulich <jbeulich@suse.com>
>> > Date:   Fri Nov 2 17:15:30 2012 +0100
>> >
>> > Before that iommu_enable was the master flag consistently. I'm still
>> > trying to understand the background and you may help elaborate if
>> > still something in your memory.
>> >
>> > I agree we should stick to one meaning after clearing up above issue.
>> >
>> > Given this commit is pretty old, I'm also curious why it's only reported
>> > on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
>> > to be the one upon which you first tried iommu=0 on a platform supporting
>> > interrupt remapping?
>>
>> It just happens to be the one I tried. VT-d was spamming my console
>> with faults like this:
>>
>> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
>> 277e28000, iommu reg = ffff82c000201000
>> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
>>
>> So I figured I can just turn it off to clean up my console. I still
>> don't know what these VT-d faults are about..
>
> What is the 0:02.0 device? Is it being passed in to your guest?

Nope, there is no pass-through to any guests. The device is:

00:02.0 Display controller: Intel Corporation 2nd Generation Core
Processor Family Integrated Graphics Controller (rev 09)

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Konrad Rzeszutek Wilk]

. snip..
> >> > Given this commit is pretty old, I'm also curious why it's only reported
> >> > on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
> >> > to be the one upon which you first tried iommu=0 on a platform supporting
> >> > interrupt remapping?
> >>
> >> It just happens to be the one I tried. VT-d was spamming my console
> >> with faults like this:
> >>
> >> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
> >> 277e28000, iommu reg = ffff82c000201000
> >> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
> >>
> >> So I figured I can just turn it off to clean up my console. I still
> >> don't know what these VT-d faults are about..
> >
> > What is the 0:02.0 device? Is it being passed in to your guest?
> 
> Nope, there is no pass-through to any guests. The device is:
> 
> 00:02.0 Display controller: Intel Corporation 2nd Generation Core
> Processor Family Integrated Graphics Controller (rev 09)

Let me guess, you have an SandyBridge motherboard and were using
Intel AMT?

I see those all the time on that box (and I think my Haswell
one too). The best I could narrow it down was that the card decided that
certain area should be in the RMRR regions. With Venu/Elen's patch
in (see 431685e8deb660976d8e986c41a647944e410c6c) you should be able
to provide an rmrr paramater to include these addresses.



I see those
> 
> Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

On Fri, Feb 17, 2017 at 11:56 AM, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> . snip..
>> >> > Given this commit is pretty old, I'm also curious why it's only reported
>> >> > on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
>> >> > to be the one upon which you first tried iommu=0 on a platform supporting
>> >> > interrupt remapping?
>> >>
>> >> It just happens to be the one I tried. VT-d was spamming my console
>> >> with faults like this:
>> >>
>> >> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
>> >> 277e28000, iommu reg = ffff82c000201000
ffff82c000201000
>> >> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
>> >>
>> >> So I figured I can just turn it off to clean up my console. I still
>> >> don't know what these VT-d faults are about..
>> >
>> > What is the 0:02.0 device? Is it being passed in to your guest?
>>
>> Nope, there is no pass-through to any guests. The device is:
>>
>> 00:02.0 Display controller: Intel Corporation 2nd Generation Core
>> Processor Family Integrated Graphics Controller (rev 09)
>
> Let me guess, you have an SandyBridge motherboard and were using
> Intel AMT?

Correct.

>
> I see those all the time on that box (and I think my Haswell
> one too). The best I could narrow it down was that the card decided that
> certain area should be in the RMRR regions. With Venu/Elen's patch
> in (see 431685e8deb660976d8e986c41a647944e410c6c) you should be able
> to provide an rmrr paramater to include these addresses.

Some more information on how to use that flag would be valuable.. I
tried "rmrr=270000000-280000000=00:02.0" to no avail and I really have
no idea what I'm doing here =)

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Konrad Rzeszutek Wilk]

On Fri, Feb 17, 2017 at 12:27:25PM -0700, Tamas K Lengyel wrote:
> On Fri, Feb 17, 2017 at 11:56 AM, Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com> wrote:
> > . snip..
> >> >> > Given this commit is pretty old, I'm also curious why it's only reported
> >> >> > on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
> >> >> > to be the one upon which you first tried iommu=0 on a platform supporting
> >> >> > interrupt remapping?
> >> >>
> >> >> It just happens to be the one I tried. VT-d was spamming my console
> >> >> with faults like this:
> >> >>
> >> >> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
> >> >> 277e28000, iommu reg = ffff82c000201000
> ffff82c000201000
> >> >> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
> >> >>
> >> >> So I figured I can just turn it off to clean up my console. I still
> >> >> don't know what these VT-d faults are about..
> >> >
> >> > What is the 0:02.0 device? Is it being passed in to your guest?
> >>
> >> Nope, there is no pass-through to any guests. The device is:
> >>
> >> 00:02.0 Display controller: Intel Corporation 2nd Generation Core
> >> Processor Family Integrated Graphics Controller (rev 09)
> >
> > Let me guess, you have an SandyBridge motherboard and were using
> > Intel AMT?
> 
> Correct.
> 
> >
> > I see those all the time on that box (and I think my Haswell
> > one too). The best I could narrow it down was that the card decided that
> > certain area should be in the RMRR regions. With Venu/Elen's patch
> > in (see 431685e8deb660976d8e986c41a647944e410c6c) you should be able
> > to provide an rmrr paramater to include these addresses.
> 
> Some more information on how to use that flag would be valuable.. I
> tried "rmrr=270000000-280000000=00:02.0" to no avail and I really have
> no idea what I'm doing here =)

CC-ing Elena and Venu who I hope can help you with this.

You may want to provide the full 'xl dmesg' as the 'rmrr' should have
outputted some details..
> 
> Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Venu Busireddy]

On Fri, Feb 17, 2017 at 02:29:32PM -0500, Konrad Rzeszutek Wilk wrote:
> On Fri, Feb 17, 2017 at 12:27:25PM -0700, Tamas K Lengyel wrote:
> > On Fri, Feb 17, 2017 at 11:56 AM, Konrad Rzeszutek Wilk
> > <konrad.wilk@oracle.com> wrote:
> > > . snip..
> > >> >> > Given this commit is pretty old, I'm also curious why it's only reported
> > >> >> > on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
> > >> >> > to be the one upon which you first tried iommu=0 on a platform supporting
> > >> >> > interrupt remapping?
> > >> >>
> > >> >> It just happens to be the one I tried. VT-d was spamming my console
> > >> >> with faults like this:
> > >> >>
> > >> >> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
> > >> >> 277e28000, iommu reg = ffff82c000201000
> > ffff82c000201000
> > >> >> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
> > >> >>
> > >> >> So I figured I can just turn it off to clean up my console. I still
> > >> >> don't know what these VT-d faults are about..
> > >> >
> > >> > What is the 0:02.0 device? Is it being passed in to your guest?
> > >>
> > >> Nope, there is no pass-through to any guests. The device is:
> > >>
> > >> 00:02.0 Display controller: Intel Corporation 2nd Generation Core
> > >> Processor Family Integrated Graphics Controller (rev 09)
> > >
> > > Let me guess, you have an SandyBridge motherboard and were using
> > > Intel AMT?
> > 
> > Correct.
> > 
> > >
> > > I see those all the time on that box (and I think my Haswell
> > > one too). The best I could narrow it down was that the card decided that
> > > certain area should be in the RMRR regions. With Venu/Elen's patch
> > > in (see 431685e8deb660976d8e986c41a647944e410c6c) you should be able
> > > to provide an rmrr paramater to include these addresses.
> > 
> > Some more information on how to use that flag would be valuable.. I
> > tried "rmrr=270000000-280000000=00:02.0" to no avail and I really have
> > no idea what I'm doing here =)
> 
> CC-ing Elena and Venu who I hope can help you with this.
> 
> You may want to provide the full 'xl dmesg' as the 'rmrr' should have
> outputted some details..

Tamas,

While 'xl dmesg' and the output from the serial console will certainly
help, it appears that you may have already provided the info needed. There
may be other devices than 0:02.0 that have problems, but let us assume
that it is the only device.

Your change to xen command line is incorrect. The correct option to add
to the xen command line is:

rmrr=0x277e28=0:00:02.0

For the RMRR specification to work, apart from changing the Xen command
line appropriately, you will also need the corresponding patch applied to
your Xen image. The patch has been pushed upstream, but is only available
in 4.9. Did you backport that change and rebuilt your Xen image with
it? Unless you do that, the Xen command line changes won't work.

Good luck!

Venu


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

On Fri, Feb 17, 2017 at 1:01 PM, Venu Busireddy
<venu.busireddy@oracle.com> wrote:
> On Fri, Feb 17, 2017 at 02:29:32PM -0500, Konrad Rzeszutek Wilk wrote:
>> On Fri, Feb 17, 2017 at 12:27:25PM -0700, Tamas K Lengyel wrote:
>> > On Fri, Feb 17, 2017 at 11:56 AM, Konrad Rzeszutek Wilk
>> > <konrad.wilk@oracle.com> wrote:
>> > > . snip..
>> > >> >> > Given this commit is pretty old, I'm also curious why it's only reported
>> > >> >> > on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
>> > >> >> > to be the one upon which you first tried iommu=0 on a platform supporting
>> > >> >> > interrupt remapping?
>> > >> >>
>> > >> >> It just happens to be the one I tried. VT-d was spamming my console
>> > >> >> with faults like this:
>> > >> >>
>> > >> >> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
>> > >> >> 277e28000, iommu reg = ffff82c000201000
>> > ffff82c000201000
>> > >> >> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
>> > >> >>
>> > >> >> So I figured I can just turn it off to clean up my console. I still
>> > >> >> don't know what these VT-d faults are about..
>> > >> >
>> > >> > What is the 0:02.0 device? Is it being passed in to your guest?
>> > >>
>> > >> Nope, there is no pass-through to any guests. The device is:
>> > >>
>> > >> 00:02.0 Display controller: Intel Corporation 2nd Generation Core
>> > >> Processor Family Integrated Graphics Controller (rev 09)
>> > >
>> > > Let me guess, you have an SandyBridge motherboard and were using
>> > > Intel AMT?
>> >
>> > Correct.
>> >
>> > >
>> > > I see those all the time on that box (and I think my Haswell
>> > > one too). The best I could narrow it down was that the card decided that
>> > > certain area should be in the RMRR regions. With Venu/Elen's patch
>> > > in (see 431685e8deb660976d8e986c41a647944e410c6c) you should be able
>> > > to provide an rmrr paramater to include these addresses.
>> >
>> > Some more information on how to use that flag would be valuable.. I
>> > tried "rmrr=270000000-280000000=00:02.0" to no avail and I really have
>> > no idea what I'm doing here =)
>>
>> CC-ing Elena and Venu who I hope can help you with this.
>>
>> You may want to provide the full 'xl dmesg' as the 'rmrr' should have
>> outputted some details..
>
> Tamas,
>
> While 'xl dmesg' and the output from the serial console will certainly
> help, it appears that you may have already provided the info needed. There
> may be other devices than 0:02.0 that have problems, but let us assume
> that it is the only device.
>
> Your change to xen command line is incorrect. The correct option to add
> to the xen command line is:
>
> rmrr=0x277e28=0:00:02.0

I see, so it has to be a page and not an address.. OTOH the problem
with specifying 0x277e28 is that the fault address seem to change
every time I reboot the machine. I'm not sure where that range is
coming from but it always seems to be between 270000000-280000000.

>
> For the RMRR specification to work, apart from changing the Xen command
> line appropriately, you will also need the corresponding patch applied to
> your Xen image. The patch has been pushed upstream, but is only available
> in 4.9. Did you backport that change and rebuilt your Xen image with
> it? Unless you do that, the Xen command line changes won't work.

Ah, no, I thought it was already part of Xen 4.8.. I'll check with 4.9
unstable in a bit.

Thanks!
Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Andrew Cooper]

On 17/02/17 21:28, Tamas K Lengyel wrote:
> On Fri, Feb 17, 2017 at 1:01 PM, Venu Busireddy
> <venu.busireddy@oracle.com> wrote:
>> On Fri, Feb 17, 2017 at 02:29:32PM -0500, Konrad Rzeszutek Wilk wrote:
>>> On Fri, Feb 17, 2017 at 12:27:25PM -0700, Tamas K Lengyel wrote:
>>>> On Fri, Feb 17, 2017 at 11:56 AM, Konrad Rzeszutek Wilk
>>>> <konrad.wilk@oracle.com> wrote:
>>>>> . snip..
>>>>>>>>> Given this commit is pretty old, I'm also curious why it's only reported
>>>>>>>>> on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
>>>>>>>>> to be the one upon which you first tried iommu=0 on a platform supporting
>>>>>>>>> interrupt remapping?
>>>>>>>> It just happens to be the one I tried. VT-d was spamming my console
>>>>>>>> with faults like this:
>>>>>>>>
>>>>>>>> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
>>>>>>>> 277e28000, iommu reg = ffff82c000201000
>>>> ffff82c000201000
>>>>>>>> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
>>>>>>>>
>>>>>>>> So I figured I can just turn it off to clean up my console. I still
>>>>>>>> don't know what these VT-d faults are about..
>>>>>>> What is the 0:02.0 device? Is it being passed in to your guest?
>>>>>> Nope, there is no pass-through to any guests. The device is:
>>>>>>
>>>>>> 00:02.0 Display controller: Intel Corporation 2nd Generation Core
>>>>>> Processor Family Integrated Graphics Controller (rev 09)
>>>>> Let me guess, you have an SandyBridge motherboard and were using
>>>>> Intel AMT?
>>>> Correct.
>>>>
>>>>> I see those all the time on that box (and I think my Haswell
>>>>> one too). The best I could narrow it down was that the card decided that
>>>>> certain area should be in the RMRR regions. With Venu/Elen's patch
>>>>> in (see 431685e8deb660976d8e986c41a647944e410c6c) you should be able
>>>>> to provide an rmrr paramater to include these addresses.
>>>> Some more information on how to use that flag would be valuable.. I
>>>> tried "rmrr=270000000-280000000=00:02.0" to no avail and I really have
>>>> no idea what I'm doing here =)
>>> CC-ing Elena and Venu who I hope can help you with this.
>>>
>>> You may want to provide the full 'xl dmesg' as the 'rmrr' should have
>>> outputted some details..
>> Tamas,
>>
>> While 'xl dmesg' and the output from the serial console will certainly
>> help, it appears that you may have already provided the info needed. There
>> may be other devices than 0:02.0 that have problems, but let us assume
>> that it is the only device.
>>
>> Your change to xen command line is incorrect. The correct option to add
>> to the xen command line is:
>>
>> rmrr=0x277e28=0:00:02.0
> I see, so it has to be a page and not an address.. OTOH the problem
> with specifying 0x277e28 is that the fault address seem to change
> every time I reboot the machine. I'm not sure where that range is
> coming from but it always seems to be between 270000000-280000000.

Looking at the parsing code, ranges work fine but you need to use 0x for
hex addresses, or the address will be interpreted as decimal.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tamas K Lengyel]

On Fri, Feb 17, 2017 at 2:31 PM, Andrew Cooper
<andrew.cooper3@citrix.com> wrote:
> On 17/02/17 21:28, Tamas K Lengyel wrote:
>> On Fri, Feb 17, 2017 at 1:01 PM, Venu Busireddy
>> <venu.busireddy@oracle.com> wrote:
>>> On Fri, Feb 17, 2017 at 02:29:32PM -0500, Konrad Rzeszutek Wilk wrote:
>>>> On Fri, Feb 17, 2017 at 12:27:25PM -0700, Tamas K Lengyel wrote:
>>>>> On Fri, Feb 17, 2017 at 11:56 AM, Konrad Rzeszutek Wilk
>>>>> <konrad.wilk@oracle.com> wrote:
>>>>>> . snip..
>>>>>>>>>> Given this commit is pretty old, I'm also curious why it's only reported
>>>>>>>>>> on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
>>>>>>>>>> to be the one upon which you first tried iommu=0 on a platform supporting
>>>>>>>>>> interrupt remapping?
>>>>>>>>> It just happens to be the one I tried. VT-d was spamming my console
>>>>>>>>> with faults like this:
>>>>>>>>>
>>>>>>>>> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
>>>>>>>>> 277e28000, iommu reg = ffff82c000201000
>>>>> ffff82c000201000
>>>>>>>>> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
>>>>>>>>>
>>>>>>>>> So I figured I can just turn it off to clean up my console. I still
>>>>>>>>> don't know what these VT-d faults are about..
>>>>>>>> What is the 0:02.0 device? Is it being passed in to your guest?
>>>>>>> Nope, there is no pass-through to any guests. The device is:
>>>>>>>
>>>>>>> 00:02.0 Display controller: Intel Corporation 2nd Generation Core
>>>>>>> Processor Family Integrated Graphics Controller (rev 09)
>>>>>> Let me guess, you have an SandyBridge motherboard and were using
>>>>>> Intel AMT?
>>>>> Correct.
>>>>>
>>>>>> I see those all the time on that box (and I think my Haswell
>>>>>> one too). The best I could narrow it down was that the card decided that
>>>>>> certain area should be in the RMRR regions. With Venu/Elen's patch
>>>>>> in (see 431685e8deb660976d8e986c41a647944e410c6c) you should be able
>>>>>> to provide an rmrr paramater to include these addresses.
>>>>> Some more information on how to use that flag would be valuable.. I
>>>>> tried "rmrr=270000000-280000000=00:02.0" to no avail and I really have
>>>>> no idea what I'm doing here =)
>>>> CC-ing Elena and Venu who I hope can help you with this.
>>>>
>>>> You may want to provide the full 'xl dmesg' as the 'rmrr' should have
>>>> outputted some details..
>>> Tamas,
>>>
>>> While 'xl dmesg' and the output from the serial console will certainly
>>> help, it appears that you may have already provided the info needed. There
>>> may be other devices than 0:02.0 that have problems, but let us assume
>>> that it is the only device.
>>>
>>> Your change to xen command line is incorrect. The correct option to add
>>> to the xen command line is:
>>>
>>> rmrr=0x277e28=0:00:02.0
>> I see, so it has to be a page and not an address.. OTOH the problem
>> with specifying 0x277e28 is that the fault address seem to change
>> every time I reboot the machine. I'm not sure where that range is
>> coming from but it always seems to be between 270000000-280000000.
>
> Looking at the parsing code, ranges work fine but you need to use 0x for
> hex addresses, or the address will be interpreted as decimal.
>

Ah, yes, makes sense =)

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Tian, Kevin]

> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Friday, February 17, 2017 4:21 PM
> 
> I think the commit description is pretty clear. Especially this part
> 
> "This could have the nice side effect of allowing to use "iommu=off"
>  even when x2APIC was pre-enabled by the BIOS (in which case interrupt
>  remapping is a requirement, but DMA translation [obviously] isn't), but
>  that doesn't currently work (and hence x2apic_bsp_setup() forces the
>  IOMMU on rather than just interrupt remapping)."
> 
> is quite relevant in the context here. Just like Linux, we really ought
> to have a way to run with interrupt remapping, but without DMA
> remapping. So I think this old commit was a half-hearted step into
> that direction, not recognizing that it would break some other case.
> 
> So I think the only reasonable way forward to address Tamas's
> issue is to fully disentangle DMA and interrupt remapping setup,
> which I'm sure is going to take some time. As a minimal adjustment,
> though, I wonder whether that old commit's adjustment to apic.c
> shouldn't rather have set force_intremap. Tamas - could you check
> whether
> 
> --- a/xen/arch/x86/apic.c
> +++ b/xen/arch/x86/apic.c
> @@ -975,7 +975,7 @@ void __init x2apic_bsp_setup(void)
>          goto restore_out;
>      }
> 
> -    force_iommu = 1;
> +    force_intremap = 1;
> 
>      genapic = apic_x2apic_probe();
>      printk("Switched to APIC driver %s.\n", genapic->name);
> 
> makes things any better?
> 
> Jan

Looks there were some conflicting commits moving the policy 
back-and-forth. For example, your another commit in 2013 
changed back to the old way that iommu_intremap depends 
on iommu_enable:

    if ( iommu_enable )
    {
        rc = iommu_hardware_setup();
        iommu_enabled = (rc == 0);
    }
---- (added by fae03721)
    if ( !iommu_enabled )
        iommu_intremap = 0;
----
    if ( (force_iommu && !iommu_enabled) ||
         (force_intremap && !iommu_intremap) )
        panic("Couldn't enable %s and iommu=required/force",
              !iommu_enabled ? "IOMMU" : "Interrupt Remapping");

But I cannot understand why Tamas doesn't hit above panic
again...

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: Unable to boot Xen 4.8 with iommu=0

[Roger Pau Monné]

On Fri, Feb 17, 2017 at 02:01:15PM -0600, Venu Busireddy wrote:
> On Fri, Feb 17, 2017 at 02:29:32PM -0500, Konrad Rzeszutek Wilk wrote:
> > On Fri, Feb 17, 2017 at 12:27:25PM -0700, Tamas K Lengyel wrote:
> > > On Fri, Feb 17, 2017 at 11:56 AM, Konrad Rzeszutek Wilk
> > > <konrad.wilk@oracle.com> wrote:
> > > > . snip..
> > > >> >> > Given this commit is pretty old, I'm also curious why it's only reported
> > > >> >> > on 4.8. Tamas, did you succeed with iommu=0 pre 4.8, or 4.8 happens
> > > >> >> > to be the one upon which you first tried iommu=0 on a platform supporting
> > > >> >> > interrupt remapping?
> > > >> >>
> > > >> >> It just happens to be the one I tried. VT-d was spamming my console
> > > >> >> with faults like this:
> > > >> >>
> > > >> >> (XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr
> > > >> >> 277e28000, iommu reg = ffff82c000201000
> > > ffff82c000201000
> > > >> >> (XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
> > > >> >>
> > > >> >> So I figured I can just turn it off to clean up my console. I still
> > > >> >> don't know what these VT-d faults are about..
> > > >> >
> > > >> > What is the 0:02.0 device? Is it being passed in to your guest?
> > > >>
> > > >> Nope, there is no pass-through to any guests. The device is:
> > > >>
> > > >> 00:02.0 Display controller: Intel Corporation 2nd Generation Core
> > > >> Processor Family Integrated Graphics Controller (rev 09)
> > > >
> > > > Let me guess, you have an SandyBridge motherboard and were using
> > > > Intel AMT?
> > > 
> > > Correct.
> > > 
> > > >
> > > > I see those all the time on that box (and I think my Haswell
> > > > one too). The best I could narrow it down was that the card decided that
> > > > certain area should be in the RMRR regions. With Venu/Elen's patch
> > > > in (see 431685e8deb660976d8e986c41a647944e410c6c) you should be able
> > > > to provide an rmrr paramater to include these addresses.
> > > 
> > > Some more information on how to use that flag would be valuable.. I
> > > tried "rmrr=270000000-280000000=00:02.0" to no avail and I really have
> > > no idea what I'm doing here =)
> > 
> > CC-ing Elena and Venu who I hope can help you with this.
> > 
> > You may want to provide the full 'xl dmesg' as the 'rmrr' should have
> > outputted some details..
> 
> Tamas,
> 
> While 'xl dmesg' and the output from the serial console will certainly
> help, it appears that you may have already provided the info needed. There
> may be other devices than 0:02.0 that have problems, but let us assume
> that it is the only device.
> 
> Your change to xen command line is incorrect. The correct option to add
> to the xen command line is:
> 
> rmrr=0x277e28=0:00:02.0

Could you please improve the documentation by mentioning that 'start' and 'end'
should be frame numbers instead of full memory addresses?

I also stumbled upon this and had to go look at the code in order to figure out
what I should pass to rmrr. Also a couple of examples about it's utilization
wouldn't hurt IMHO.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel