[Qemu-devel] Unusual physical address when using 64-bit BAR

[Qemu-devel] Unusual physical address when using 64-bit BAR

[Cam Macdonell]

Hi,

I'm trying to use a 64-bit BAR for my shared memory device.  In simply
changing the memory type in pci_register_bar() to
PCI_BASE_ADDRESS_MEM_TYPE_64 I get an unusual physical address for
that BAR (and my driver crashes in pci_ioremap).

from lspci:

00:04.0 RAM memory: Qumranet, Inc. Device 1110
        Subsystem: Qumranet, Inc. Device 1100
        Flags: fast devsel, IRQ 10
        Memory at f1020000 (32-bit, non-prefetchable) [size=1K]
        Memory at f1021000 (32-bit, non-prefetchable) [size=4K]
        Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
        Capabilities: <access denied>
00: f4 1a 10 11 03 00 10 00 00 00 00 05 00 00 00 00
10: 00 00 02 f1 00 10 02 f1 04 00 00 00 00 c2 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11
30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00

with DEBUG_MEMREG, I see

kvm_unregister_memory_area:666 Unregistering memory region
c20000000000 (40000000)
kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
IVSHMEM: addr = 3221225472 size = 1073741824
kvm_register_phys_mem:605 memory: gpa: c200c0000000, size: 40000000,
uaddr: 7f6dd7ffe000, slot: 7, flags: 0
kvm_unregister_memory_area:666 Unregistering memory region
c200c0000000 (40000000)
kvm_destroy_phys_mem:649 slot 7 start c200c0000000 len 0 flags 0
IVSHMEM: addr = 0 size = 1073741824
kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
uaddr: 7f6dd7ffe000, slot: 7, flags: 0
kvm_unregister_memory_area:666 Unregistering memory region
c20000000000 (40000000)
kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
IVSHMEM: addr = 0 size = 1073741824
kvm_register_phys_mem:605 memory: gpa: ffffffff00000000, size:
40000000, uaddr: 7f6dd7ffe000, slot: 7, flags: 0
kvm_unregister_memory_area:666 Unregistering memory region
ffffffff00000000 (40000000)
kvm_destroy_phys_mem:649 slot 7 start ffffffff00000000 len 0 flags 0
IVSHMEM: addr = 0 size = 1073741824
kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
uaddr: 7f6dd7ffe000, slot: 7, flags: 0

(the IVSHMEM lines are my debug statements)

address sizes   : 40 bits physical, 48 bits virtual  (guest)
address sizes   : 38 bits physical, 48 bits virtual  (host)

Thanks,
Cam

[Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Mon, Apr 19, 2010 at 10:41 AM, Cam Macdonell <cam@cs.ualberta.ca> wrote:
> Hi,
>
> I'm trying to use a 64-bit BAR for my shared memory device.  In simply
> changing the memory type in pci_register_bar() to
> PCI_BASE_ADDRESS_MEM_TYPE_64 I get an unusual physical address for
> that BAR (and my driver crashes in pci_ioremap).
>
> from lspci:
>
> 00:04.0 RAM memory: Qumranet, Inc. Device 1110
>        Subsystem: Qumranet, Inc. Device 1100
>        Flags: fast devsel, IRQ 10
>        Memory at f1020000 (32-bit, non-prefetchable) [size=1K]
>        Memory at f1021000 (32-bit, non-prefetchable) [size=4K]
>        Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
>        Capabilities: <access denied>
> 00: f4 1a 10 11 03 00 10 00 00 00 00 05 00 00 00 00
> 10: 00 00 02 f1 00 10 02 f1 04 00 00 00 00 c2 00 00
> 20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11
> 30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00
>
> with DEBUG_MEMREG, I see
>
> kvm_unregister_memory_area:666 Unregistering memory region
> c20000000000 (40000000)
> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
> IVSHMEM: addr = 3221225472 size = 1073741824
> kvm_register_phys_mem:605 memory: gpa: c200c0000000, size: 40000000,
> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
> kvm_unregister_memory_area:666 Unregistering memory region
> c200c0000000 (40000000)
> kvm_destroy_phys_mem:649 slot 7 start c200c0000000 len 0 flags 0
> IVSHMEM: addr = 0 size = 1073741824
> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
> kvm_unregister_memory_area:666 Unregistering memory region
> c20000000000 (40000000)
> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
> IVSHMEM: addr = 0 size = 1073741824
> kvm_register_phys_mem:605 memory: gpa: ffffffff00000000, size:
> 40000000, uaddr: 7f6dd7ffe000, slot: 7, flags: 0
> kvm_unregister_memory_area:666 Unregistering memory region
> ffffffff00000000 (40000000)
> kvm_destroy_phys_mem:649 slot 7 start ffffffff00000000 len 0 flags 0
> IVSHMEM: addr = 0 size = 1073741824
> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>
> (the IVSHMEM lines are my debug statements)
>
> address sizes   : 40 bits physical, 48 bits virtual  (guest)
> address sizes   : 38 bits physical, 48 bits virtual  (host)
>

Hi, I happened to run into this problem again when trying to use a
64-bit BAR.  I did a bit more digging and the test that is failing is
called from arch/x86/mm/ioremap.c in the guest and here it is.

static inline int phys_addr_valid(resource_size_t addr)
{
#ifdef CONFIG_PHYS_ADDR_T_64BIT
 	return !(addr >> boot_cpu_data.x86_phys_bits);
#else
	return 1;
#endif
}

the value of addr (in this case the 48-bit virtual address
c20000000000) is shifted to the right shift by
boot_cpu_data.x86_phys_bits (which is 40 bits, the physical address
size), so a non-zero value is returned which causes the test to fail
and generates the "invalid physical address" error in the guest.

Any help is appreciated as to whether this is a Qemu or guest kernel issue.

Thanks,
Cam

[Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Avi Kivity]

On 06/11/2010 08:31 PM, Cam Macdonell wrote:
> On Mon, Apr 19, 2010 at 10:41 AM, Cam Macdonell<cam@cs.ualberta.ca>  wrote:
>    
>> Hi,
>>
>> I'm trying to use a 64-bit BAR for my shared memory device.  In simply
>> changing the memory type in pci_register_bar() to
>> PCI_BASE_ADDRESS_MEM_TYPE_64 I get an unusual physical address for
>> that BAR (and my driver crashes in pci_ioremap).
>>
>> from lspci:
>>
>> 00:04.0 RAM memory: Qumranet, Inc. Device 1110
>>         Subsystem: Qumranet, Inc. Device 1100
>>         Flags: fast devsel, IRQ 10
>>         Memory at f1020000 (32-bit, non-prefetchable) [size=1K]
>>         Memory at f1021000 (32-bit, non-prefetchable) [size=4K]
>>         Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
>>         Capabilities:<access denied>
>> 00: f4 1a 10 11 03 00 10 00 00 00 00 05 00 00 00 00
>> 10: 00 00 02 f1 00 10 02 f1 04 00 00 00 00 c2 00 00
>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11
>> 30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00
>>
>> with DEBUG_MEMREG, I see
>>
>> kvm_unregister_memory_area:666 Unregistering memory region
>> c20000000000 (40000000)
>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
>> IVSHMEM: addr = 3221225472 size = 1073741824
>> kvm_register_phys_mem:605 memory: gpa: c200c0000000, size: 40000000,
>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>> kvm_unregister_memory_area:666 Unregistering memory region
>> c200c0000000 (40000000)
>> kvm_destroy_phys_mem:649 slot 7 start c200c0000000 len 0 flags 0
>> IVSHMEM: addr = 0 size = 1073741824
>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>> kvm_unregister_memory_area:666 Unregistering memory region
>> c20000000000 (40000000)
>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
>> IVSHMEM: addr = 0 size = 1073741824
>> kvm_register_phys_mem:605 memory: gpa: ffffffff00000000, size:
>> 40000000, uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>> kvm_unregister_memory_area:666 Unregistering memory region
>> ffffffff00000000 (40000000)
>> kvm_destroy_phys_mem:649 slot 7 start ffffffff00000000 len 0 flags 0
>> IVSHMEM: addr = 0 size = 1073741824
>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>
>> (the IVSHMEM lines are my debug statements)
>>
>> address sizes   : 40 bits physical, 48 bits virtual  (guest)
>> address sizes   : 38 bits physical, 48 bits virtual  (host)
>>
>>      
> Hi, I happened to run into this problem again when trying to use a
> 64-bit BAR.  I did a bit more digging and the test that is failing is
> called from arch/x86/mm/ioremap.c in the guest and here it is.
>
> static inline int phys_addr_valid(resource_size_t addr)
> {
> #ifdef CONFIG_PHYS_ADDR_T_64BIT
>   	return !(addr>>  boot_cpu_data.x86_phys_bits);
> #else
> 	return 1;
> #endif
> }
>
> the value of addr (in this case the 48-bit virtual address
> c20000000000) is shifted to the right shift by
> boot_cpu_data.x86_phys_bits (which is 40 bits, the physical address
> size), so a non-zero value is returned which causes the test to fail
> and generates the "invalid physical address" error in the guest.
>
> Any help is appreciated as to whether this is a Qemu or guest kernel issue.
>    

The guest kernel should never have generated an address that is bigger 
than cpu_phys_bits in the first place.  What's the value for 
cpu_phys_bits in the guest? (/proc/cpuinfo, 'address sizes :' line).

Is this really the address the guest programmed, or is qemu 
misinterpreting it?

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Tue, Jun 15, 2010 at 5:04 AM, Avi Kivity <avi@redhat.com> wrote:
> On 06/11/2010 08:31 PM, Cam Macdonell wrote:
>>
>> On Mon, Apr 19, 2010 at 10:41 AM, Cam Macdonell<cam@cs.ualberta.ca>
>>  wrote:
>>
>>>
>>> Hi,
>>>
>>> I'm trying to use a 64-bit BAR for my shared memory device.  In simply
>>> changing the memory type in pci_register_bar() to
>>> PCI_BASE_ADDRESS_MEM_TYPE_64 I get an unusual physical address for
>>> that BAR (and my driver crashes in pci_ioremap).
>>>
>>> from lspci:
>>>
>>> 00:04.0 RAM memory: Qumranet, Inc. Device 1110
>>>        Subsystem: Qumranet, Inc. Device 1100
>>>        Flags: fast devsel, IRQ 10
>>>        Memory at f1020000 (32-bit, non-prefetchable) [size=1K]
>>>        Memory at f1021000 (32-bit, non-prefetchable) [size=4K]
>>>        Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
>>>        Capabilities:<access denied>
>>> 00: f4 1a 10 11 03 00 10 00 00 00 00 05 00 00 00 00
>>> 10: 00 00 02 f1 00 10 02 f1 04 00 00 00 00 c2 00 00
>>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11
>>> 30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00
>>>
>>> with DEBUG_MEMREG, I see
>>>
>>> kvm_unregister_memory_area:666 Unregistering memory region
>>> c20000000000 (40000000)
>>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
>>> IVSHMEM: addr = 3221225472 size = 1073741824
>>> kvm_register_phys_mem:605 memory: gpa: c200c0000000, size: 40000000,
>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>> kvm_unregister_memory_area:666 Unregistering memory region
>>> c200c0000000 (40000000)
>>> kvm_destroy_phys_mem:649 slot 7 start c200c0000000 len 0 flags 0
>>> IVSHMEM: addr = 0 size = 1073741824
>>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>> kvm_unregister_memory_area:666 Unregistering memory region
>>> c20000000000 (40000000)
>>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
>>> IVSHMEM: addr = 0 size = 1073741824
>>> kvm_register_phys_mem:605 memory: gpa: ffffffff00000000, size:
>>> 40000000, uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>> kvm_unregister_memory_area:666 Unregistering memory region
>>> ffffffff00000000 (40000000)
>>> kvm_destroy_phys_mem:649 slot 7 start ffffffff00000000 len 0 flags 0
>>> IVSHMEM: addr = 0 size = 1073741824
>>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>
>>> (the IVSHMEM lines are my debug statements)
>>>
>>> address sizes   : 40 bits physical, 48 bits virtual  (guest)
>>> address sizes   : 38 bits physical, 48 bits virtual  (host)
>>>
>>>
>>
>> Hi, I happened to run into this problem again when trying to use a
>> 64-bit BAR.  I did a bit more digging and the test that is failing is
>> called from arch/x86/mm/ioremap.c in the guest and here it is.
>>
>> static inline int phys_addr_valid(resource_size_t addr)
>> {
>> #ifdef CONFIG_PHYS_ADDR_T_64BIT
>>        return !(addr>>  boot_cpu_data.x86_phys_bits);
>> #else
>>        return 1;
>> #endif
>> }
>>
>> the value of addr (in this case the 48-bit virtual address
>> c20000000000) is shifted to the right shift by
>> boot_cpu_data.x86_phys_bits (which is 40 bits, the physical address
>> size), so a non-zero value is returned which causes the test to fail
>> and generates the "invalid physical address" error in the guest.
>>
>> Any help is appreciated as to whether this is a Qemu or guest kernel
>> issue.
>>
>
> The guest kernel should never have generated an address that is bigger than
> cpu_phys_bits in the first place.  What's the value for cpu_phys_bits in the
> guest? (/proc/cpuinfo, 'address sizes :' line).

Sorry I missed your reply until now.  The guest address sizes are as follows:

address sizes   : 40 bits physical, 48 bits virtual

>
> Is this really the address the guest programmed, or is qemu misinterpreting
> it?
>
> --
> error compiling committee.c: too many arguments to function
>
>

[Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Avi Kivity]

On 06/25/2010 12:51 AM, Cam Macdonell wrote:
> On Tue, Jun 15, 2010 at 5:04 AM, Avi Kivity<avi@redhat.com>  wrote:
>    
>> On 06/11/2010 08:31 PM, Cam Macdonell wrote:
>>      
>>> On Mon, Apr 19, 2010 at 10:41 AM, Cam Macdonell<cam@cs.ualberta.ca>
>>>   wrote:
>>>
>>>        
>>>> Hi,
>>>>
>>>> I'm trying to use a 64-bit BAR for my shared memory device.  In simply
>>>> changing the memory type in pci_register_bar() to
>>>> PCI_BASE_ADDRESS_MEM_TYPE_64 I get an unusual physical address for
>>>> that BAR (and my driver crashes in pci_ioremap).
>>>>
>>>> from lspci:
>>>>
>>>> 00:04.0 RAM memory: Qumranet, Inc. Device 1110
>>>>         Subsystem: Qumranet, Inc. Device 1100
>>>>         Flags: fast devsel, IRQ 10
>>>>         Memory at f1020000 (32-bit, non-prefetchable) [size=1K]
>>>>         Memory at f1021000 (32-bit, non-prefetchable) [size=4K]
>>>>         Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
>>>>         Capabilities:<access denied>
>>>> 00: f4 1a 10 11 03 00 10 00 00 00 00 05 00 00 00 00
>>>> 10: 00 00 02 f1 00 10 02 f1 04 00 00 00 00 c2 00 00
>>>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11
>>>> 30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00
>>>>
>>>> with DEBUG_MEMREG, I see
>>>>
>>>> kvm_unregister_memory_area:666 Unregistering memory region
>>>> c20000000000 (40000000)
>>>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
>>>> IVSHMEM: addr = 3221225472 size = 1073741824
>>>> kvm_register_phys_mem:605 memory: gpa: c200c0000000, size: 40000000,
>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>> kvm_unregister_memory_area:666 Unregistering memory region
>>>> c200c0000000 (40000000)
>>>> kvm_destroy_phys_mem:649 slot 7 start c200c0000000 len 0 flags 0
>>>> IVSHMEM: addr = 0 size = 1073741824
>>>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>> kvm_unregister_memory_area:666 Unregistering memory region
>>>> c20000000000 (40000000)
>>>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
>>>> IVSHMEM: addr = 0 size = 1073741824
>>>> kvm_register_phys_mem:605 memory: gpa: ffffffff00000000, size:
>>>> 40000000, uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>> kvm_unregister_memory_area:666 Unregistering memory region
>>>> ffffffff00000000 (40000000)
>>>> kvm_destroy_phys_mem:649 slot 7 start ffffffff00000000 len 0 flags 0
>>>> IVSHMEM: addr = 0 size = 1073741824
>>>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>>
>>>> (the IVSHMEM lines are my debug statements)
>>>>
>>>> address sizes   : 40 bits physical, 48 bits virtual  (guest)
>>>> address sizes   : 38 bits physical, 48 bits virtual  (host)
>>>>
>>>>
>>>>          
>>> Hi, I happened to run into this problem again when trying to use a
>>> 64-bit BAR.  I did a bit more digging and the test that is failing is
>>> called from arch/x86/mm/ioremap.c in the guest and here it is.
>>>
>>> static inline int phys_addr_valid(resource_size_t addr)
>>> {
>>> #ifdef CONFIG_PHYS_ADDR_T_64BIT
>>>         return !(addr>>    boot_cpu_data.x86_phys_bits);
>>> #else
>>>         return 1;
>>> #endif
>>> }
>>>
>>> the value of addr (in this case the 48-bit virtual address
>>> c20000000000) is shifted to the right shift by
>>> boot_cpu_data.x86_phys_bits (which is 40 bits, the physical address
>>> size), so a non-zero value is returned which causes the test to fail
>>> and generates the "invalid physical address" error in the guest.
>>>
>>> Any help is appreciated as to whether this is a Qemu or guest kernel
>>> issue.
>>>
>>>        
>> The guest kernel should never have generated an address that is bigger than
>> cpu_phys_bits in the first place.  What's the value for cpu_phys_bits in the
>> guest? (/proc/cpuinfo, 'address sizes :' line).
>>      
> Sorry I missed your reply until now.  The guest address sizes are as follows:
>
> address sizes   : 40 bits physical, 48 bits virtual
>    

So the address c20000000000 is illegal.

>> Is this really the address the guest programmed, or is qemu misinterpreting
>> it?
>>      

Well, what's the answer?

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Sun, Jun 27, 2010 at 2:39 AM, Avi Kivity <avi@redhat.com> wrote:
> On 06/25/2010 12:51 AM, Cam Macdonell wrote:
>>
>> On Tue, Jun 15, 2010 at 5:04 AM, Avi Kivity<avi@redhat.com>  wrote:
>>
>>>
>>> On 06/11/2010 08:31 PM, Cam Macdonell wrote:
>>>
>>>>
>>>> On Mon, Apr 19, 2010 at 10:41 AM, Cam Macdonell<cam@cs.ualberta.ca>
>>>>  wrote:
>>>>
>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm trying to use a 64-bit BAR for my shared memory device.  In simply
>>>>> changing the memory type in pci_register_bar() to
>>>>> PCI_BASE_ADDRESS_MEM_TYPE_64 I get an unusual physical address for
>>>>> that BAR (and my driver crashes in pci_ioremap).
>>>>>
>>>>> from lspci:
>>>>>
>>>>> 00:04.0 RAM memory: Qumranet, Inc. Device 1110
>>>>>        Subsystem: Qumranet, Inc. Device 1100
>>>>>        Flags: fast devsel, IRQ 10
>>>>>        Memory at f1020000 (32-bit, non-prefetchable) [size=1K]
>>>>>        Memory at f1021000 (32-bit, non-prefetchable) [size=4K]
>>>>>        Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
>>>>>        Capabilities:<access denied>
>>>>> 00: f4 1a 10 11 03 00 10 00 00 00 00 05 00 00 00 00
>>>>> 10: 00 00 02 f1 00 10 02 f1 04 00 00 00 00 c2 00 00
>>>>> 20: 00 00 00 00 00 00 00 00 00 00 00 00 f4 1a 00 11
>>>>> 30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00
>>>>>
>>>>> with DEBUG_MEMREG, I see
>>>>>
>>>>> kvm_unregister_memory_area:666 Unregistering memory region
>>>>> c20000000000 (40000000)
>>>>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
>>>>> IVSHMEM: addr = 3221225472 size = 1073741824
>>>>> kvm_register_phys_mem:605 memory: gpa: c200c0000000, size: 40000000,
>>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>>> kvm_unregister_memory_area:666 Unregistering memory region
>>>>> c200c0000000 (40000000)
>>>>> kvm_destroy_phys_mem:649 slot 7 start c200c0000000 len 0 flags 0
>>>>> IVSHMEM: addr = 0 size = 1073741824
>>>>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
>>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>>> kvm_unregister_memory_area:666 Unregistering memory region
>>>>> c20000000000 (40000000)
>>>>> kvm_destroy_phys_mem:649 slot 7 start c20000000000 len 0 flags 0
>>>>> IVSHMEM: addr = 0 size = 1073741824
>>>>> kvm_register_phys_mem:605 memory: gpa: ffffffff00000000, size:
>>>>> 40000000, uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>>> kvm_unregister_memory_area:666 Unregistering memory region
>>>>> ffffffff00000000 (40000000)
>>>>> kvm_destroy_phys_mem:649 slot 7 start ffffffff00000000 len 0 flags 0
>>>>> IVSHMEM: addr = 0 size = 1073741824
>>>>> kvm_register_phys_mem:605 memory: gpa: c20000000000, size: 40000000,
>>>>> uaddr: 7f6dd7ffe000, slot: 7, flags: 0
>>>>>
>>>>> (the IVSHMEM lines are my debug statements)
>>>>>
>>>>> address sizes   : 40 bits physical, 48 bits virtual  (guest)
>>>>> address sizes   : 38 bits physical, 48 bits virtual  (host)
>>>>>
>>>>>
>>>>>
>>>>
>>>> Hi, I happened to run into this problem again when trying to use a
>>>> 64-bit BAR.  I did a bit more digging and the test that is failing is
>>>> called from arch/x86/mm/ioremap.c in the guest and here it is.
>>>>
>>>> static inline int phys_addr_valid(resource_size_t addr)
>>>> {
>>>> #ifdef CONFIG_PHYS_ADDR_T_64BIT
>>>>        return !(addr>>    boot_cpu_data.x86_phys_bits);
>>>> #else
>>>>        return 1;
>>>> #endif
>>>> }
>>>>
>>>> the value of addr (in this case the 48-bit virtual address
>>>> c20000000000) is shifted to the right shift by
>>>> boot_cpu_data.x86_phys_bits (which is 40 bits, the physical address
>>>> size), so a non-zero value is returned which causes the test to fail
>>>> and generates the "invalid physical address" error in the guest.
>>>>
>>>> Any help is appreciated as to whether this is a Qemu or guest kernel
>>>> issue.
>>>>
>>>>
>>>
>>> The guest kernel should never have generated an address that is bigger
>>> than
>>> cpu_phys_bits in the first place.  What's the value for cpu_phys_bits in
>>> the
>>> guest? (/proc/cpuinfo, 'address sizes :' line).
>>>
>>
>> Sorry I missed your reply until now.  The guest address sizes are as
>> follows:
>>
>> address sizes   : 40 bits physical, 48 bits virtual
>>
>
> So the address c20000000000 is illegal.
>
>>> Is this really the address the guest programmed, or is qemu
>>> misinterpreting
>>> it?
>>>
>
> Well, what's the answer?

You're going to have to give me a hint on how to determine that.

lspci in the guest shows the following

Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]

does that demonstrate a guest generated address?

Cam

>
> --
> error compiling committee.c: too many arguments to function
>
>

[Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Avi Kivity]

On 06/28/2010 11:38 PM, Cam Macdonell wrote:
>
>>
>>>> Is this really the address the guest programmed, or is qemu
>>>> misinterpreting
>>>> it?
>>>>
>>>>          
>> Well, what's the answer?
>>      
> You're going to have to give me a hint on how to determine that.
>
> lspci in the guest shows the following
>
> Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
>
> does that demonstrate a guest generated address?
>    

That's the result of a round trip: the guest programmed the address and 
then read it back.  It could have been screwed up in the first place, or 
perhaps qemu screwed it up.

Add a printf() to the config space handlers in qemu (likely in your own 
code) on writes and reads, and show the relevant writes (and reads) for 
this BAR.

That's the theory of deductive debugging; however browsing the code 
shows the guest is at fault:

         for (i = 0; i < PCI_NUM_REGIONS; i++) {
             int ofs;
             if (i == PCI_ROM_SLOT)
                 ofs = PCI_ROM_ADDRESS;
             else
                 ofs = PCI_BASE_ADDRESS_0 + i * 4;

             u32 old = pci_config_readl(bdf, ofs);
             u32 mask;
             if (i == PCI_ROM_SLOT) {
                 mask = PCI_ROM_ADDRESS_MASK;
                 pci_config_writel(bdf, ofs, mask);
             } else {
                 if (old & PCI_BASE_ADDRESS_SPACE_IO)
                     mask = PCI_BASE_ADDRESS_IO_MASK;
                 else
                     mask = PCI_BASE_ADDRESS_MEM_MASK;
                 pci_config_writel(bdf, ofs, ~0);
             }
             u32 val = pci_config_readl(bdf, ofs);
             pci_config_writel(bdf, ofs, old);

             if (val != 0) {
                 u32 size = (~(val & mask)) + 1;
                 if (val & PCI_BASE_ADDRESS_SPACE_IO)
                     paddr = &pci_bios_io_addr;
                 else
                     paddr = &pci_bios_mem_addr;
                 *paddr = ALIGN(*paddr, size);
                 pci_set_io_region_addr(bdf, i, *paddr);
                 *paddr += size;
             }
         }
         break;
     }

Seabios completely ignore the 64-bitness of the BAR.  Looks like it also 
thinks the second half of the BAR is an I/O region instead of memory 
(hence the c200, that's part of the pci portio region.

Do post those reads and writes, I think there's more than one thing 
wrong here.


-- 
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.

[Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Tue, Jun 29, 2010 at 12:50 AM, Avi Kivity <avi@redhat.com> wrote:
> On 06/28/2010 11:38 PM, Cam Macdonell wrote:
>>
>>>
>>>>> Is this really the address the guest programmed, or is qemu
>>>>> misinterpreting
>>>>> it?
>>>>>
>>>>>
>>>
>>> Well, what's the answer?
>>>
>>
>> You're going to have to give me a hint on how to determine that.
>>
>> lspci in the guest shows the following
>>
>> Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
>>
>> does that demonstrate a guest generated address?
>>
>
> That's the result of a round trip: the guest programmed the address and then
> read it back.  It could have been screwed up in the first place, or perhaps
> qemu screwed it up.
>
> Add a printf() to the config space handlers in qemu (likely in your own
> code) on writes and reads, and show the relevant writes (and reads) for this
> BAR.
>
> That's the theory of deductive debugging; however browsing the code shows
> the guest is at fault:
>
>        for (i = 0; i < PCI_NUM_REGIONS; i++) {
>            int ofs;
>            if (i == PCI_ROM_SLOT)
>                ofs = PCI_ROM_ADDRESS;
>            else
>                ofs = PCI_BASE_ADDRESS_0 + i * 4;
>
>            u32 old = pci_config_readl(bdf, ofs);
>            u32 mask;
>            if (i == PCI_ROM_SLOT) {
>                mask = PCI_ROM_ADDRESS_MASK;
>                pci_config_writel(bdf, ofs, mask);
>            } else {
>                if (old & PCI_BASE_ADDRESS_SPACE_IO)
>                    mask = PCI_BASE_ADDRESS_IO_MASK;
>                else
>                    mask = PCI_BASE_ADDRESS_MEM_MASK;
>                pci_config_writel(bdf, ofs, ~0);
>            }
>            u32 val = pci_config_readl(bdf, ofs);
>            pci_config_writel(bdf, ofs, old);
>
>            if (val != 0) {
>                u32 size = (~(val & mask)) + 1;
>                if (val & PCI_BASE_ADDRESS_SPACE_IO)
>                    paddr = &pci_bios_io_addr;
>                else
>                    paddr = &pci_bios_mem_addr;
>                *paddr = ALIGN(*paddr, size);
>                pci_set_io_region_addr(bdf, i, *paddr);
>                *paddr += size;
>            }
>        }
>        break;
>    }
>
> Seabios completely ignore the 64-bitness of the BAR.  Looks like it also
> thinks the second half of the BAR is an I/O region instead of memory (hence
> the c200, that's part of the pci portio region.
>
> Do post those reads and writes, I think there's more than one thing wrong
> here.

Here it is, I added the debug statements to pci_read_config and
pci_default_write_config.

here are the reads and writes to offsets 0x18 and 0x1c where a 64-bit
BAR2 config would be configured

pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0xc040 <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)

the complete read/write profile is below along with debug statements
from the map functions for the BARs (prefixed with IVSHMEM)

pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x1110 <- 0x2 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x500 <- 0xa (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x1110 <- 0x2 (addr)
pci_read_config: (val) 0x0 <- 0x10 (addr)
pci_write_config: (val) 0x0 -> 0x10 (addr)
pci_read_config: (val) 0xffffff00 <- 0x10 (addr)
pci_write_config: (val) 0x0 -> 0x10 (addr)
pci_read_config: (val) 0x0 <- 0x10 (addr)
pci_write_config: (val) 0x0 -> 0x10 (addr)
pci_read_config: (val) 0x0 <- 0x14 (addr)
pci_write_config: (val) 0x0 -> 0x14 (addr)
pci_read_config: (val) 0xfffff000 <- 0x14 (addr)
pci_write_config: (val) 0x0 -> 0x14 (addr)
pci_read_config: (val) 0x0 <- 0x14 (addr)
pci_write_config: (val) 0x0 -> 0x14 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x0 <- 0x20 (addr)
pci_write_config: (val) 0x0 -> 0x20 (addr)
pci_read_config: (val) 0x0 <- 0x20 (addr)
pci_write_config: (val) 0x0 -> 0x20 (addr)
pci_read_config: (val) 0x0 <- 0x24 (addr)
pci_write_config: (val) 0x0 -> 0x24 (addr)
pci_read_config: (val) 0x0 <- 0x24 (addr)
pci_write_config: (val) 0x0 -> 0x24 (addr)
pci_read_config: (val) 0x0 <- 0x30 (addr)
pci_write_config: (val) 0x0 -> 0x30 (addr)
pci_read_config: (val) 0x0 <- 0x30 (addr)
pci_write_config: (val) 0x0 -> 0x30 (addr)
pci_read_config: (val) 0x0 <- 0x4 (addr)
pci_write_config: (val) 0x0 -> 0x4 (addr)
IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
size = 40000000
pci_read_config: (val) 0x1 <- 0x3d (addr)
pci_write_config: (val) 0x0 -> 0x3c (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1 <- 0x3d (addr)
pci_read_config: (val) 0xb <- 0x3c (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x5000000 <- 0x8 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x500 <- 0xa (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x500 <- 0xa (addr)
pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x0 <- 0x30 (addr)
pci_write_config: (val) 0x0 -> 0x30 (addr)
pci_read_config: (val) 0x0 <- 0x30 (addr)
pci_write_config: (val) 0x0 -> 0x30 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x5000000 <- 0x8 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
pci_read_config: (val) 0x500 <- 0xa (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x1110 <- 0x2 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x5000000 <- 0x8 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x10 <- 0x6 (addr)
pci_read_config: (val) 0x40 <- 0x34 (addr)
pci_read_config: (val) 0x11 <- 0x40 (addr)
pci_read_config: (val) 0x0 <- 0x41 (addr)
pci_read_config: (val) 0x5000000 <- 0x8 (addr)
pci_read_config: (val) 0x10 <- 0x6 (addr)
pci_read_config: (val) 0x40 <- 0x34 (addr)
pci_read_config: (val) 0x11 <- 0x40 (addr)
pci_read_config: (val) 0x0 <- 0x41 (addr)
pci_read_config: (val) 0x1 <- 0x3d (addr)
pci_read_config: (val) 0xb <- 0x3c (addr)
pci_read_config: (val) 0xf2040000 <- 0x10 (addr)
pci_write_config: (val) 0x0 -> 0x10 (addr)
pci_read_config: (val) 0xffffff00 <- 0x10 (addr)
pci_write_config: (val) 0x0 -> 0x10 (addr)
pci_read_config: (val) 0xf2041000 <- 0x14 (addr)
pci_write_config: (val) 0x0 -> 0x14 (addr)
pci_read_config: (val) 0xfffff000 <- 0x14 (addr)
pci_write_config: (val) 0x0 -> 0x14 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
IVSHMEM: guest pci addr = c040c0000000, guest h/w addr = 1090912256,
size = 40000000
pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
size = 40000000
pci_read_config: (val) 0xc040 <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
1090912256, size = 40000000
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
size = 40000000
pci_read_config: (val) 0x0 <- 0x20 (addr)
pci_write_config: (val) 0x0 -> 0x20 (addr)
pci_read_config: (val) 0x0 <- 0x20 (addr)
pci_write_config: (val) 0x0 -> 0x20 (addr)
pci_read_config: (val) 0x0 <- 0x24 (addr)
pci_write_config: (val) 0x0 -> 0x24 (addr)
pci_read_config: (val) 0x0 <- 0x24 (addr)
pci_write_config: (val) 0x0 -> 0x24 (addr)
pci_read_config: (val) 0x0 <- 0x30 (addr)
pci_write_config: (val) 0x0 -> 0x30 (addr)
pci_read_config: (val) 0x0 <- 0x30 (addr)
pci_write_config: (val) 0x0 -> 0x30 (addr)
pci_read_config: (val) 0x1af4 <- 0x2c (addr)
pci_read_config: (val) 0x1100 <- 0x2e (addr)
pci_read_config: (val) 0x10 <- 0x6 (addr)
pci_read_config: (val) 0x40 <- 0x34 (addr)
pci_read_config: (val) 0x11 <- 0x40 (addr)
pci_read_config: (val) 0x0 <- 0x41 (addr)
pci_read_config: (val) 0x10 <- 0x6 (addr)
pci_read_config: (val) 0x40 <- 0x34 (addr)
pci_read_config: (val) 0x11 <- 0x40 (addr)
pci_read_config: (val) 0x0 <- 0x41 (addr)
pci_read_config: (val) 0x10 <- 0x6 (addr)
pci_read_config: (val) 0x40 <- 0x34 (addr)
pci_read_config: (val) 0x11 <- 0x40 (addr)
pci_read_config: (val) 0x0 <- 0x41 (addr)
pci_read_config: (val) 0x10 <- 0x6 (addr)
pci_read_config: (val) 0x40 <- 0x34 (addr)
pci_read_config: (val) 0x11 <- 0x40 (addr)
pci_read_config: (val) 0x0 <- 0x41 (addr)
pci_read_config: (val) 0x3 <- 0x4 (addr)
pci_read_config: (val) 0x3 <- 0x4 (addr)
pci_read_config: (val) 0x0 <- 0xc (addr)
pci_read_config: (val) 0x3 <- 0x4 (addr)
pci_read_config: (val) 0x3 <- 0x4 (addr)
pci_read_config: (val) 0x10 <- 0x6 (addr)
pci_read_config: (val) 0x40 <- 0x34 (addr)
pci_read_config: (val) 0x11 <- 0x40 (addr)
pci_read_config: (val) 0x0 <- 0x41 (addr)


>
>
> --
> I have a truly marvellous patch that fixes the bug which this
> signature is too narrow to contain.
>
>

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Isaku Yamahata]

On Tue, Jun 29, 2010 at 11:48:13AM -0600, Cam Macdonell wrote:
> On Tue, Jun 29, 2010 at 12:50 AM, Avi Kivity <avi@redhat.com> wrote:
> > On 06/28/2010 11:38 PM, Cam Macdonell wrote:
> >>
> >>>
> >>>>> Is this really the address the guest programmed, or is qemu
> >>>>> misinterpreting
> >>>>> it?
> >>>>>
> >>>>>
> >>>
> >>> Well, what's the answer?
> >>>
> >>
> >> You're going to have to give me a hint on how to determine that.
> >>
> >> lspci in the guest shows the following
> >>
> >> Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
> >>
> >> does that demonstrate a guest generated address?
> >>
> >
> > That's the result of a round trip: the guest programmed the address and then
> > read it back. ?It could have been screwed up in the first place, or perhaps
> > qemu screwed it up.
> >
> > Add a printf() to the config space handlers in qemu (likely in your own
> > code) on writes and reads, and show the relevant writes (and reads) for this
> > BAR.
> >
> > That's the theory of deductive debugging; however browsing the code shows
> > the guest is at fault:
> >
> > ? ? ? ?for (i = 0; i < PCI_NUM_REGIONS; i++) {
> > ? ? ? ? ? ?int ofs;
> > ? ? ? ? ? ?if (i == PCI_ROM_SLOT)
> > ? ? ? ? ? ? ? ?ofs = PCI_ROM_ADDRESS;
> > ? ? ? ? ? ?else
> > ? ? ? ? ? ? ? ?ofs = PCI_BASE_ADDRESS_0 + i * 4;
> >
> > ? ? ? ? ? ?u32 old = pci_config_readl(bdf, ofs);
> > ? ? ? ? ? ?u32 mask;
> > ? ? ? ? ? ?if (i == PCI_ROM_SLOT) {
> > ? ? ? ? ? ? ? ?mask = PCI_ROM_ADDRESS_MASK;
> > ? ? ? ? ? ? ? ?pci_config_writel(bdf, ofs, mask);
> > ? ? ? ? ? ?} else {
> > ? ? ? ? ? ? ? ?if (old & PCI_BASE_ADDRESS_SPACE_IO)
> > ? ? ? ? ? ? ? ? ? ?mask = PCI_BASE_ADDRESS_IO_MASK;
> > ? ? ? ? ? ? ? ?else
> > ? ? ? ? ? ? ? ? ? ?mask = PCI_BASE_ADDRESS_MEM_MASK;
> > ? ? ? ? ? ? ? ?pci_config_writel(bdf, ofs, ~0);
> > ? ? ? ? ? ?}
> > ? ? ? ? ? ?u32 val = pci_config_readl(bdf, ofs);
> > ? ? ? ? ? ?pci_config_writel(bdf, ofs, old);
> >
> > ? ? ? ? ? ?if (val != 0) {
> > ? ? ? ? ? ? ? ?u32 size = (~(val & mask)) + 1;
> > ? ? ? ? ? ? ? ?if (val & PCI_BASE_ADDRESS_SPACE_IO)
> > ? ? ? ? ? ? ? ? ? ?paddr = &pci_bios_io_addr;
> > ? ? ? ? ? ? ? ?else
> > ? ? ? ? ? ? ? ? ? ?paddr = &pci_bios_mem_addr;
> > ? ? ? ? ? ? ? ?*paddr = ALIGN(*paddr, size);
> > ? ? ? ? ? ? ? ?pci_set_io_region_addr(bdf, i, *paddr);
> > ? ? ? ? ? ? ? ?*paddr += size;
> > ? ? ? ? ? ?}
> > ? ? ? ?}
> > ? ? ? ?break;
> > ? ?}
> >
> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
> > thinks the second half of the BAR is an I/O region instead of memory (hence
> > the c200, that's part of the pci portio region.

I've sent the patches to address it. But they haven't been merged yet.
seabios doesn't map BARs beyond 4GB.
If bar is mapped beyond 4GB, guest BIOS does it.


To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
in config.h of seabios


> > Do post those reads and writes, I think there's more than one thing wrong
> > here.
> 
> Here it is, I added the debug statements to pci_read_config and
> pci_default_write_config.
> 
> here are the reads and writes to offsets 0x18 and 0x1c where a 64-bit
> BAR2 config would be configured

It seems that 0x1c is accessed as BAR3.
The write value in the log is always 0.

Some comment in the log.

> 
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0xc040 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> 
> the complete read/write profile is below along with debug statements
> from the map functions for the BARs (prefixed with IVSHMEM)
> 
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x1110 <- 0x2 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x500 <- 0xa (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x1110 <- 0x2 (addr)
> pci_read_config: (val) 0x0 <- 0x10 (addr)
> pci_write_config: (val) 0x0 -> 0x10 (addr)
> pci_read_config: (val) 0xffffff00 <- 0x10 (addr)
> pci_write_config: (val) 0x0 -> 0x10 (addr)
> pci_read_config: (val) 0x0 <- 0x10 (addr)
> pci_write_config: (val) 0x0 -> 0x10 (addr)
> pci_read_config: (val) 0x0 <- 0x14 (addr)
> pci_write_config: (val) 0x0 -> 0x14 (addr)
> pci_read_config: (val) 0xfffff000 <- 0x14 (addr)
> pci_write_config: (val) 0x0 -> 0x14 (addr)
> pci_read_config: (val) 0x0 <- 0x14 (addr)
> pci_write_config: (val) 0x0 -> 0x14 (addr)

> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)

seabios BAR2.

> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)

seabios BAR3. Not sure how it is mapped from this
message.



> pci_read_config: (val) 0x0 <- 0x20 (addr)
> pci_write_config: (val) 0x0 -> 0x20 (addr)
> pci_read_config: (val) 0x0 <- 0x20 (addr)
> pci_write_config: (val) 0x0 -> 0x20 (addr)
> pci_read_config: (val) 0x0 <- 0x24 (addr)
> pci_write_config: (val) 0x0 -> 0x24 (addr)
> pci_read_config: (val) 0x0 <- 0x24 (addr)
> pci_write_config: (val) 0x0 -> 0x24 (addr)
> pci_read_config: (val) 0x0 <- 0x30 (addr)
> pci_write_config: (val) 0x0 -> 0x30 (addr)
> pci_read_config: (val) 0x0 <- 0x30 (addr)
> pci_write_config: (val) 0x0 -> 0x30 (addr)
> pci_read_config: (val) 0x0 <- 0x4 (addr)
> pci_write_config: (val) 0x0 -> 0x4 (addr)
> IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
> size = 40000000
> pci_read_config: (val) 0x1 <- 0x3d (addr)
> pci_write_config: (val) 0x0 -> 0x3c (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1 <- 0x3d (addr)
> pci_read_config: (val) 0xb <- 0x3c (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x5000000 <- 0x8 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x500 <- 0xa (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x500 <- 0xa (addr)
> pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x0 <- 0x30 (addr)
> pci_write_config: (val) 0x0 -> 0x30 (addr)
> pci_read_config: (val) 0x0 <- 0x30 (addr)
> pci_write_config: (val) 0x0 -> 0x30 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x5000000 <- 0x8 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
> pci_read_config: (val) 0x500 <- 0xa (addr)
> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
> pci_read_config: (val) 0x1110 <- 0x2 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x5000000 <- 0x8 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
> pci_read_config: (val) 0x0 <- 0xe (addr)
> pci_read_config: (val) 0x10 <- 0x6 (addr)
> pci_read_config: (val) 0x40 <- 0x34 (addr)
> pci_read_config: (val) 0x11 <- 0x40 (addr)
> pci_read_config: (val) 0x0 <- 0x41 (addr)
> pci_read_config: (val) 0x5000000 <- 0x8 (addr)
> pci_read_config: (val) 0x10 <- 0x6 (addr)
> pci_read_config: (val) 0x40 <- 0x34 (addr)
> pci_read_config: (val) 0x11 <- 0x40 (addr)
> pci_read_config: (val) 0x0 <- 0x41 (addr)
> pci_read_config: (val) 0x1 <- 0x3d (addr)
> pci_read_config: (val) 0xb <- 0x3c (addr)
> pci_read_config: (val) 0xf2040000 <- 0x10 (addr)
> pci_write_config: (val) 0x0 -> 0x10 (addr)
> pci_read_config: (val) 0xffffff00 <- 0x10 (addr)
> pci_write_config: (val) 0x0 -> 0x10 (addr)
> pci_read_config: (val) 0xf2041000 <- 0x14 (addr)
> pci_write_config: (val) 0x0 -> 0x14 (addr)
> pci_read_config: (val) 0xfffff000 <- 0x14 (addr)
> pci_write_config: (val) 0x0 -> 0x14 (addr)


> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> IVSHMEM: guest pci addr = c040c0000000, guest h/w addr = 1090912256,
> size = 40000000
> pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
> size = 40000000
> pci_read_config: (val) 0xc040 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
> 1090912256, size = 40000000
> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
> size = 40000000

guest OS?
It seems that guset OS understands 64bit BAR
and remaps the BAR.


> pci_read_config: (val) 0x0 <- 0x20 (addr)
> pci_write_config: (val) 0x0 -> 0x20 (addr)
> pci_read_config: (val) 0x0 <- 0x20 (addr)
> pci_write_config: (val) 0x0 -> 0x20 (addr)
> pci_read_config: (val) 0x0 <- 0x24 (addr)
> pci_write_config: (val) 0x0 -> 0x24 (addr)
> pci_read_config: (val) 0x0 <- 0x24 (addr)
> pci_write_config: (val) 0x0 -> 0x24 (addr)
> pci_read_config: (val) 0x0 <- 0x30 (addr)
> pci_write_config: (val) 0x0 -> 0x30 (addr)
> pci_read_config: (val) 0x0 <- 0x30 (addr)
> pci_write_config: (val) 0x0 -> 0x30 (addr)
> pci_read_config: (val) 0x1af4 <- 0x2c (addr)
> pci_read_config: (val) 0x1100 <- 0x2e (addr)
> pci_read_config: (val) 0x10 <- 0x6 (addr)
> pci_read_config: (val) 0x40 <- 0x34 (addr)
> pci_read_config: (val) 0x11 <- 0x40 (addr)
> pci_read_config: (val) 0x0 <- 0x41 (addr)
> pci_read_config: (val) 0x10 <- 0x6 (addr)
> pci_read_config: (val) 0x40 <- 0x34 (addr)
> pci_read_config: (val) 0x11 <- 0x40 (addr)
> pci_read_config: (val) 0x0 <- 0x41 (addr)
> pci_read_config: (val) 0x10 <- 0x6 (addr)
> pci_read_config: (val) 0x40 <- 0x34 (addr)
> pci_read_config: (val) 0x11 <- 0x40 (addr)
> pci_read_config: (val) 0x0 <- 0x41 (addr)
> pci_read_config: (val) 0x10 <- 0x6 (addr)
> pci_read_config: (val) 0x40 <- 0x34 (addr)
> pci_read_config: (val) 0x11 <- 0x40 (addr)
> pci_read_config: (val) 0x0 <- 0x41 (addr)
> pci_read_config: (val) 0x3 <- 0x4 (addr)
> pci_read_config: (val) 0x3 <- 0x4 (addr)
> pci_read_config: (val) 0x0 <- 0xc (addr)
> pci_read_config: (val) 0x3 <- 0x4 (addr)
> pci_read_config: (val) 0x3 <- 0x4 (addr)
> pci_read_config: (val) 0x10 <- 0x6 (addr)
> pci_read_config: (val) 0x40 <- 0x34 (addr)
> pci_read_config: (val) 0x11 <- 0x40 (addr)
> pci_read_config: (val) 0x0 <- 0x41 (addr)
> 
> 
> >
> >
> > --
> > I have a truly marvellous patch that fixes the bug which this
> > signature is too narrow to contain.
> >
> >
> 

-- 
yamahata

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Tue, Jun 29, 2010 at 9:29 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> On Tue, Jun 29, 2010 at 11:48:13AM -0600, Cam Macdonell wrote:
>> On Tue, Jun 29, 2010 at 12:50 AM, Avi Kivity <avi@redhat.com> wrote:
>> > On 06/28/2010 11:38 PM, Cam Macdonell wrote:
>> >>
>> >>>
>> >>>>> Is this really the address the guest programmed, or is qemu
>> >>>>> misinterpreting
>> >>>>> it?
>> >>>>>
>> >>>>>
>> >>>
>> >>> Well, what's the answer?
>> >>>
>> >>
>> >> You're going to have to give me a hint on how to determine that.
>> >>
>> >> lspci in the guest shows the following
>> >>
>> >> Memory at c20000000000 (64-bit, non-prefetchable) [size=1024M]
>> >>
>> >> does that demonstrate a guest generated address?
>> >>
>> >
>> > That's the result of a round trip: the guest programmed the address and then
>> > read it back. ?It could have been screwed up in the first place, or perhaps
>> > qemu screwed it up.
>> >
>> > Add a printf() to the config space handlers in qemu (likely in your own
>> > code) on writes and reads, and show the relevant writes (and reads) for this
>> > BAR.
>> >
>> > That's the theory of deductive debugging; however browsing the code shows
>> > the guest is at fault:
>> >
>> > ? ? ? ?for (i = 0; i < PCI_NUM_REGIONS; i++) {
>> > ? ? ? ? ? ?int ofs;
>> > ? ? ? ? ? ?if (i == PCI_ROM_SLOT)
>> > ? ? ? ? ? ? ? ?ofs = PCI_ROM_ADDRESS;
>> > ? ? ? ? ? ?else
>> > ? ? ? ? ? ? ? ?ofs = PCI_BASE_ADDRESS_0 + i * 4;
>> >
>> > ? ? ? ? ? ?u32 old = pci_config_readl(bdf, ofs);
>> > ? ? ? ? ? ?u32 mask;
>> > ? ? ? ? ? ?if (i == PCI_ROM_SLOT) {
>> > ? ? ? ? ? ? ? ?mask = PCI_ROM_ADDRESS_MASK;
>> > ? ? ? ? ? ? ? ?pci_config_writel(bdf, ofs, mask);
>> > ? ? ? ? ? ?} else {
>> > ? ? ? ? ? ? ? ?if (old & PCI_BASE_ADDRESS_SPACE_IO)
>> > ? ? ? ? ? ? ? ? ? ?mask = PCI_BASE_ADDRESS_IO_MASK;
>> > ? ? ? ? ? ? ? ?else
>> > ? ? ? ? ? ? ? ? ? ?mask = PCI_BASE_ADDRESS_MEM_MASK;
>> > ? ? ? ? ? ? ? ?pci_config_writel(bdf, ofs, ~0);
>> > ? ? ? ? ? ?}
>> > ? ? ? ? ? ?u32 val = pci_config_readl(bdf, ofs);
>> > ? ? ? ? ? ?pci_config_writel(bdf, ofs, old);
>> >
>> > ? ? ? ? ? ?if (val != 0) {
>> > ? ? ? ? ? ? ? ?u32 size = (~(val & mask)) + 1;
>> > ? ? ? ? ? ? ? ?if (val & PCI_BASE_ADDRESS_SPACE_IO)
>> > ? ? ? ? ? ? ? ? ? ?paddr = &pci_bios_io_addr;
>> > ? ? ? ? ? ? ? ?else
>> > ? ? ? ? ? ? ? ? ? ?paddr = &pci_bios_mem_addr;
>> > ? ? ? ? ? ? ? ?*paddr = ALIGN(*paddr, size);
>> > ? ? ? ? ? ? ? ?pci_set_io_region_addr(bdf, i, *paddr);
>> > ? ? ? ? ? ? ? ?*paddr += size;
>> > ? ? ? ? ? ?}
>> > ? ? ? ?}
>> > ? ? ? ?break;
>> > ? ?}
>> >
>> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
>> > thinks the second half of the BAR is an I/O region instead of memory (hence
>> > the c200, that's part of the pci portio region.
>
> I've sent the patches to address it. But they haven't been merged yet.
> seabios doesn't map BARs beyond 4GB.
> If bar is mapped beyond 4GB, guest BIOS does it.

Have those patches been merged yet?

>
>
> To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> in config.h of seabios

Where does the output from seabios end up?  Inside dmesg?

>
>
>> > Do post those reads and writes, I think there's more than one thing wrong
>> > here.
>>
>> Here it is, I added the debug statements to pci_read_config and
>> pci_default_write_config.
>>
>> here are the reads and writes to offsets 0x18 and 0x1c where a 64-bit
>> BAR2 config would be configured
>
> It seems that 0x1c is accessed as BAR3.
> The write value in the log is always 0.
>
> Some comment in the log.
>
>>
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0xc040 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>>
>> the complete read/write profile is below along with debug statements
>> from the map functions for the BARs (prefixed with IVSHMEM)
>>
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x1110 <- 0x2 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x500 <- 0xa (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x1110 <- 0x2 (addr)
>> pci_read_config: (val) 0x0 <- 0x10 (addr)
>> pci_write_config: (val) 0x0 -> 0x10 (addr)
>> pci_read_config: (val) 0xffffff00 <- 0x10 (addr)
>> pci_write_config: (val) 0x0 -> 0x10 (addr)
>> pci_read_config: (val) 0x0 <- 0x10 (addr)
>> pci_write_config: (val) 0x0 -> 0x10 (addr)
>> pci_read_config: (val) 0x0 <- 0x14 (addr)
>> pci_write_config: (val) 0x0 -> 0x14 (addr)
>> pci_read_config: (val) 0xfffff000 <- 0x14 (addr)
>> pci_write_config: (val) 0x0 -> 0x14 (addr)
>> pci_read_config: (val) 0x0 <- 0x14 (addr)
>> pci_write_config: (val) 0x0 -> 0x14 (addr)
>
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>
> seabios BAR2.
>
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>
> seabios BAR3. Not sure how it is mapped from this
> message.

Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
BAR3 to store all 64-bits?

>
>
>
>> pci_read_config: (val) 0x0 <- 0x20 (addr)
>> pci_write_config: (val) 0x0 -> 0x20 (addr)
>> pci_read_config: (val) 0x0 <- 0x20 (addr)
>> pci_write_config: (val) 0x0 -> 0x20 (addr)
>> pci_read_config: (val) 0x0 <- 0x24 (addr)
>> pci_write_config: (val) 0x0 -> 0x24 (addr)
>> pci_read_config: (val) 0x0 <- 0x24 (addr)
>> pci_write_config: (val) 0x0 -> 0x24 (addr)
>> pci_read_config: (val) 0x0 <- 0x30 (addr)
>> pci_write_config: (val) 0x0 -> 0x30 (addr)
>> pci_read_config: (val) 0x0 <- 0x30 (addr)
>> pci_write_config: (val) 0x0 -> 0x30 (addr)
>> pci_read_config: (val) 0x0 <- 0x4 (addr)
>> pci_write_config: (val) 0x0 -> 0x4 (addr)
>> IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
>> size = 40000000
>> pci_read_config: (val) 0x1 <- 0x3d (addr)
>> pci_write_config: (val) 0x0 -> 0x3c (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1 <- 0x3d (addr)
>> pci_read_config: (val) 0xb <- 0x3c (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x5000000 <- 0x8 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x500 <- 0xa (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x500 <- 0xa (addr)
>> pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x0 <- 0x30 (addr)
>> pci_write_config: (val) 0x0 -> 0x30 (addr)
>> pci_read_config: (val) 0x0 <- 0x30 (addr)
>> pci_write_config: (val) 0x0 -> 0x30 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x5000000 <- 0x8 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x500 <- 0xa (addr)
>> pci_read_config: (val) 0x1af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x1110 <- 0x2 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x5000000 <- 0x8 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x11101af4 <- 0x0 (addr)
>> pci_read_config: (val) 0x0 <- 0xe (addr)
>> pci_read_config: (val) 0x10 <- 0x6 (addr)
>> pci_read_config: (val) 0x40 <- 0x34 (addr)
>> pci_read_config: (val) 0x11 <- 0x40 (addr)
>> pci_read_config: (val) 0x0 <- 0x41 (addr)
>> pci_read_config: (val) 0x5000000 <- 0x8 (addr)
>> pci_read_config: (val) 0x10 <- 0x6 (addr)
>> pci_read_config: (val) 0x40 <- 0x34 (addr)
>> pci_read_config: (val) 0x11 <- 0x40 (addr)
>> pci_read_config: (val) 0x0 <- 0x41 (addr)
>> pci_read_config: (val) 0x1 <- 0x3d (addr)
>> pci_read_config: (val) 0xb <- 0x3c (addr)
>> pci_read_config: (val) 0xf2040000 <- 0x10 (addr)
>> pci_write_config: (val) 0x0 -> 0x10 (addr)
>> pci_read_config: (val) 0xffffff00 <- 0x10 (addr)
>> pci_write_config: (val) 0x0 -> 0x10 (addr)
>> pci_read_config: (val) 0xf2041000 <- 0x14 (addr)
>> pci_write_config: (val) 0x0 -> 0x14 (addr)
>> pci_read_config: (val) 0xfffff000 <- 0x14 (addr)
>> pci_write_config: (val) 0x0 -> 0x14 (addr)
>
>
>> pci_read_config: (val) 0x4 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> IVSHMEM: guest pci addr = c040c0000000, guest h/w addr = 1090912256,
>> size = 40000000
>> pci_read_config: (val) 0xc0000004 <- 0x18 (addr)
>> pci_write_config: (val) 0x0 -> 0x18 (addr)
>> IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
>> size = 40000000
>> pci_read_config: (val) 0xc040 <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
>> 1090912256, size = 40000000
>> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 1090912256,
>> size = 40000000
>
> guest OS?
> It seems that guset OS understands 64bit BAR
> and remaps the BAR.

But c0400000000 is a 48-bit address and the guest only has 40-bit
physical addresses, so the remap fails in the guest.

Cam

>
>
>> pci_read_config: (val) 0x0 <- 0x20 (addr)
>> pci_write_config: (val) 0x0 -> 0x20 (addr)
>> pci_read_config: (val) 0x0 <- 0x20 (addr)
>> pci_write_config: (val) 0x0 -> 0x20 (addr)
>> pci_read_config: (val) 0x0 <- 0x24 (addr)
>> pci_write_config: (val) 0x0 -> 0x24 (addr)
>> pci_read_config: (val) 0x0 <- 0x24 (addr)
>> pci_write_config: (val) 0x0 -> 0x24 (addr)
>> pci_read_config: (val) 0x0 <- 0x30 (addr)
>> pci_write_config: (val) 0x0 -> 0x30 (addr)
>> pci_read_config: (val) 0x0 <- 0x30 (addr)
>> pci_write_config: (val) 0x0 -> 0x30 (addr)
>> pci_read_config: (val) 0x1af4 <- 0x2c (addr)
>> pci_read_config: (val) 0x1100 <- 0x2e (addr)
>> pci_read_config: (val) 0x10 <- 0x6 (addr)
>> pci_read_config: (val) 0x40 <- 0x34 (addr)
>> pci_read_config: (val) 0x11 <- 0x40 (addr)
>> pci_read_config: (val) 0x0 <- 0x41 (addr)
>> pci_read_config: (val) 0x10 <- 0x6 (addr)
>> pci_read_config: (val) 0x40 <- 0x34 (addr)
>> pci_read_config: (val) 0x11 <- 0x40 (addr)
>> pci_read_config: (val) 0x0 <- 0x41 (addr)
>> pci_read_config: (val) 0x10 <- 0x6 (addr)
>> pci_read_config: (val) 0x40 <- 0x34 (addr)
>> pci_read_config: (val) 0x11 <- 0x40 (addr)
>> pci_read_config: (val) 0x0 <- 0x41 (addr)
>> pci_read_config: (val) 0x10 <- 0x6 (addr)
>> pci_read_config: (val) 0x40 <- 0x34 (addr)
>> pci_read_config: (val) 0x11 <- 0x40 (addr)
>> pci_read_config: (val) 0x0 <- 0x41 (addr)
>> pci_read_config: (val) 0x3 <- 0x4 (addr)
>> pci_read_config: (val) 0x3 <- 0x4 (addr)
>> pci_read_config: (val) 0x0 <- 0xc (addr)
>> pci_read_config: (val) 0x3 <- 0x4 (addr)
>> pci_read_config: (val) 0x3 <- 0x4 (addr)
>> pci_read_config: (val) 0x10 <- 0x6 (addr)
>> pci_read_config: (val) 0x40 <- 0x34 (addr)
>> pci_read_config: (val) 0x11 <- 0x40 (addr)
>> pci_read_config: (val) 0x0 <- 0x41 (addr)
>>
>>
>> >
>> >
>> > --
>> > I have a truly marvellous patch that fixes the bug which this
>> > signature is too narrow to contain.
>> >
>> >
>>
>
> --
> yamahata
>
>

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Isaku Yamahata]

On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
> >> > thinks the second half of the BAR is an I/O region instead of memory (hence
> >> > the c200, that's part of the pci portio region.
> >
> > I've sent the patches to address it. But they haven't been merged yet.
> > seabios doesn't map BARs beyond 4GB.
> > If bar is mapped beyond 4GB, guest BIOS does it.
> 
> Have those patches been merged yet?

They have been merged into seabios upstream now.
qemu seabios fork hasn't pulled for a while, though.


> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> > in config.h of seabios
> 
> Where does the output from seabios end up?  Inside dmesg?

It outputs them to the serial console which qemu emulates.
seabios is out of kernel control, so dmesg doesn't show it.


> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >
> > seabios BAR3. Not sure how it is mapped from this
> > message.
> 
> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
> BAR3 to store all 64-bits?

Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
-- 
yamahata

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
>> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
>> >> > thinks the second half of the BAR is an I/O region instead of memory (hence
>> >> > the c200, that's part of the pci portio region.
>> >
>> > I've sent the patches to address it. But they haven't been merged yet.
>> > seabios doesn't map BARs beyond 4GB.
>> > If bar is mapped beyond 4GB, guest BIOS does it.
>>
>> Have those patches been merged yet?
>
> They have been merged into seabios upstream now.
> qemu seabios fork hasn't pulled for a while, though.
>
>
>> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
>> > in config.h of seabios
>>
>> Where does the output from seabios end up?  Inside dmesg?
>
> It outputs them to the serial console which qemu emulates.
> seabios is out of kernel control, so dmesg doesn't show it.
>
>
>> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >
>> > seabios BAR3. Not sure how it is mapped from this
>> > message.
>>
>> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
>> BAR3 to store all 64-bits?
>
> Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
> --
> yamahata
>
>

With the latest seabios git passed via -bios, I no longer see the
48-bit address, but instead a 32-bit address and then
ffffffff00000000.  This guest has 1gb of RAM so the address isn't be
mapped beyond 4g.

IVSHMEM: guest pci addr = e0000000, guest h/w addr = 1090912256, size = 20000000
IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
1090912256, size = 20000000

However, booting fails when I use a 64-bit BAR.  Booting is fine with
a 32-bit BAR.

HPET: 1 timers in total, 0 timers will be used for per-cpu timer
divide error: 0000 [#1] SMP last sysfs file:
CPU 0
Modules linked in:

Pid: 1, comm: swapper Not tainted 2.6.35-rc1 #280 /Bochs
RIP: 0010:[<ffffffff812a801b>]  [<ffffffff812a801b>] hpet_alloc+0x12c/0x35b
RSP: 0018:ffff88003e61fd80  EFLAGS: 00010246
RAX: 00038d7ea4c68000 RBX: ffff88003e6efd80 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff817b8520
RBP: ffff88003e61fdc0 R08: 00000000000080d0 R09: ffffc90000000000
R10: ffff88003f9ac600 R11: 0000000000000000 R12: ffff88003e61fdd0
R13: ffffc90000000000 R14: 0000000000000000 R15: ffffffff817a00a9
FS:  0000000000000000(0000) GS:ffff880002000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 0000000001a42000 CR4: 00000000000006f0DR0:
0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper (pid: 1, threadinfo ffff88003e61e000, task ffff88003e620000)
Stack:
 ffff88003f43ab90 ffff88003f43ab90 ffffffff81ca3174 ffffffff81b1d596
<0> 0000000000000000 0000000000000100 0000000000000100 0000000000000000
<0> ffff88003e61fe80 ffffffff8102945d 00000000fed00000 ffffc90000000000
Call Trace:
 [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
 [<ffffffff8102945d>] hpet_reserve_platform_timers+0x10b/0x115
 [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
 [<ffffffff81b1d601>] hpet_late_init+0x6b/0xea
 [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
 [<ffffffff81002069>] do_one_initcall+0x5e/0x159
 [<ffffffff81b0b71e>] kernel_init+0x18e/0x21c
 [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
 [<ffffffff81b0b590>] ? kernel_init+0x0/0x21c
 [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
Code: 89 1d 8a 92 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 a9 00 7a 81 48
8d 04 02 4c 89 f2 48 c7 c7 20 85 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
f7 f1 83 7b 30 01 48
c7 c1 cd fa 7c 81 49 0f 46 cf 48 89 43
RIP  [<ffffffff812a801b>] hpet_alloc+0x12c/0x35b
 RSP <ffff88003e61fd80>
---[ end trace a7919e7f17c0a725 ]---
Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: swapper Tainted: G      D     2.6.35-rc1 #280
Call Trace:
 [<ffffffff8145b870>] panic+0x8b/0x10b
 [<ffffffff81056a93>] ? exit_ptrace+0x38/0x121
 [<ffffffff8104f9e8>] do_exit+0x7a/0x722
 [<ffffffff8104c3bd>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8104cfd8>] ? kmsg_dump+0x12b/0x145
 [<ffffffff8145eaa9>] oops_end+0xbf/0xc7
 [<ffffffff8100d299>] die+0x5a/0x63
 [<ffffffff8145e4b3>] do_trap+0x121/0x130
 [<ffffffff8100b560>] do_divide_error+0x96/0x9f
 [<ffffffff812a801b>] ? hpet_alloc+0x12c/0x35b
 [<ffffffff8100a83b>] divide_error+0x1b/0x20
 [<ffffffff812a801b>] ? hpet_alloc+0x12c/0x35b
 [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
 [<ffffffff8102945d>] hpet_reserve_platform_timers+0x10b/0x115
 [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
 [<ffffffff81b1d601>] hpet_late_init+0x6b/0xea
 [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
 [<ffffffff81002069>] do_one_initcall+0x5e/0x159
 [<ffffffff81b0b71e>] kernel_init+0x18e/0x21c
 [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
 [<ffffffff81b0b590>] ? kernel_init+0x0/0x21c
 [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Isaku Yamahata]

On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
> >> >> > thinks the second half of the BAR is an I/O region instead of memory (hence
> >> >> > the c200, that's part of the pci portio region.
> >> >
> >> > I've sent the patches to address it. But they haven't been merged yet.
> >> > seabios doesn't map BARs beyond 4GB.
> >> > If bar is mapped beyond 4GB, guest BIOS does it.
> >>
> >> Have those patches been merged yet?
> >
> > They have been merged into seabios upstream now.
> > qemu seabios fork hasn't pulled for a while, though.
> >
> >
> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> >> > in config.h of seabios
> >>
> >> Where does the output from seabios end up? ?Inside dmesg?
> >
> > It outputs them to the serial console which qemu emulates.
> > seabios is out of kernel control, so dmesg doesn't show it.
> >
> >
> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >
> >> > seabios BAR3. Not sure how it is mapped from this
> >> > message.
> >>
> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
> >> BAR3 to store all 64-bits?
> >
> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
> > --
> > yamahata
> >
> >
> 
> With the latest seabios git passed via -bios, I no longer see the
> 48-bit address, but instead a 32-bit address and then
> ffffffff00000000.  This guest has 1gb of RAM so the address isn't be
> mapped beyond 4g.

Can I see the debug log like before?
(hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)

Do you know who sets the BAR to ffffffff00000000?
If it's seabios, does the following patch help?

diff --git a/src/pciinit.c b/src/pciinit.c
index b110531..ce07709 100644
--- a/src/pciinit.c
+++ b/src/pciinit.c
@@ -117,11 +117,7 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
     int is_64bit = !(val & PCI_BASE_ADDRESS_SPACE_IO) &&
         (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == PCI_BASE_ADDRESS_MEM_TYPE_64;
     if (is_64bit) {
-        if (size > 0) {
-            pci_config_writel(bdf, ofs + 4, 0);
-        } else {
-            pci_config_writel(bdf, ofs + 4, ~0);
-        }
+        pci_config_writel(bdf, ofs + 4, 0);
     }
     return is_64bit;
 }



> IVSHMEM: guest pci addr = e0000000, guest h/w addr = 1090912256, size = 20000000
> IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
> 1090912256, size = 20000000
> 
> However, booting fails when I use a 64-bit BAR.  Booting is fine with
> a 32-bit BAR.
> 
> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
> divide error: 0000 [#1] SMP last sysfs file:
> CPU 0
> Modules linked in:
> 
> Pid: 1, comm: swapper Not tainted 2.6.35-rc1 #280 /Bochs
> RIP: 0010:[<ffffffff812a801b>]  [<ffffffff812a801b>] hpet_alloc+0x12c/0x35b
> RSP: 0018:ffff88003e61fd80  EFLAGS: 00010246
> RAX: 00038d7ea4c68000 RBX: ffff88003e6efd80 RCX: 0000000000000000
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff817b8520
> RBP: ffff88003e61fdc0 R08: 00000000000080d0 R09: ffffc90000000000
> R10: ffff88003f9ac600 R11: 0000000000000000 R12: ffff88003e61fdd0
> R13: ffffc90000000000 R14: 0000000000000000 R15: ffffffff817a00a9
> FS:  0000000000000000(0000) GS:ffff880002000000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 0000000000000000 CR3: 0000000001a42000 CR4: 00000000000006f0DR0:
> 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process swapper (pid: 1, threadinfo ffff88003e61e000, task ffff88003e620000)
> Stack:
>  ffff88003f43ab90 ffff88003f43ab90 ffffffff81ca3174 ffffffff81b1d596
> <0> 0000000000000000 0000000000000100 0000000000000100 0000000000000000
> <0> ffff88003e61fe80 ffffffff8102945d 00000000fed00000 ffffc90000000000
> Call Trace:
>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>  [<ffffffff8102945d>] hpet_reserve_platform_timers+0x10b/0x115
>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>  [<ffffffff81b1d601>] hpet_late_init+0x6b/0xea
>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>  [<ffffffff81002069>] do_one_initcall+0x5e/0x159
>  [<ffffffff81b0b71e>] kernel_init+0x18e/0x21c
>  [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>  [<ffffffff81b0b590>] ? kernel_init+0x0/0x21c
>  [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
> Code: 89 1d 8a 92 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 a9 00 7a 81 48
> 8d 04 02 4c 89 f2 48 c7 c7 20 85 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
> f7 f1 83 7b 30 01 48
> c7 c1 cd fa 7c 81 49 0f 46 cf 48 89 43
> RIP  [<ffffffff812a801b>] hpet_alloc+0x12c/0x35b
>  RSP <ffff88003e61fd80>
> ---[ end trace a7919e7f17c0a725 ]---
> Kernel panic - not syncing: Attempted to kill init!
> Pid: 1, comm: swapper Tainted: G      D     2.6.35-rc1 #280
> Call Trace:
>  [<ffffffff8145b870>] panic+0x8b/0x10b
>  [<ffffffff81056a93>] ? exit_ptrace+0x38/0x121
>  [<ffffffff8104f9e8>] do_exit+0x7a/0x722
>  [<ffffffff8104c3bd>] ? spin_unlock_irqrestore+0xe/0x10
>  [<ffffffff8104cfd8>] ? kmsg_dump+0x12b/0x145
>  [<ffffffff8145eaa9>] oops_end+0xbf/0xc7
>  [<ffffffff8100d299>] die+0x5a/0x63
>  [<ffffffff8145e4b3>] do_trap+0x121/0x130
>  [<ffffffff8100b560>] do_divide_error+0x96/0x9f
>  [<ffffffff812a801b>] ? hpet_alloc+0x12c/0x35b
>  [<ffffffff8100a83b>] divide_error+0x1b/0x20
>  [<ffffffff812a801b>] ? hpet_alloc+0x12c/0x35b
>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>  [<ffffffff8102945d>] hpet_reserve_platform_timers+0x10b/0x115
>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>  [<ffffffff81b1d601>] hpet_late_init+0x6b/0xea
>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>  [<ffffffff81002069>] do_one_initcall+0x5e/0x159
>  [<ffffffff81b0b71e>] kernel_init+0x18e/0x21c
>  [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>  [<ffffffff81b0b590>] ? kernel_init+0x0/0x21c
>  [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
> 

-- 
yamahata

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
>> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
>> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
>> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
>> >> >> > thinks the second half of the BAR is an I/O region instead of memory (hence
>> >> >> > the c200, that's part of the pci portio region.
>> >> >
>> >> > I've sent the patches to address it. But they haven't been merged yet.
>> >> > seabios doesn't map BARs beyond 4GB.
>> >> > If bar is mapped beyond 4GB, guest BIOS does it.
>> >>
>> >> Have those patches been merged yet?
>> >
>> > They have been merged into seabios upstream now.
>> > qemu seabios fork hasn't pulled for a while, though.
>> >
>> >
>> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
>> >> > in config.h of seabios
>> >>
>> >> Where does the output from seabios end up? ?Inside dmesg?
>> >
>> > It outputs them to the serial console which qemu emulates.
>> > seabios is out of kernel control, so dmesg doesn't show it.
>> >
>> >
>> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >> >> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> >> >
>> >> > seabios BAR3. Not sure how it is mapped from this
>> >> > message.
>> >>
>> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
>> >> BAR3 to store all 64-bits?
>> >
>> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
>> > --
>> > yamahata
>> >
>> >
>>
>> With the latest seabios git passed via -bios, I no longer see the
>> 48-bit address, but instead a 32-bit address and then
>> ffffffff00000000.  This guest has 1gb of RAM so the address isn't be
>> mapped beyond 4g.
>
> Can I see the debug log like before?
> (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)

Here's the dump from SeaBIOS in the region related to the PCI devices.
 The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit.

PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8
region 0: 0xf0000000
region 1: 0xf2000000
region 6: 0xf2010000
PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000
region 0: 0x0000c020
region 1: 0xf2020000
region 6: 0xf2030000
PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
region 0: 0xf2040000
region 1: 0xf2041000
region 2: 0x00000000

>
> Do you know who sets the BAR to ffffffff00000000?

Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM'
lines are from the map function passed to pci_register_bar().  It
looks like SeaBIOS sets the address to 0 and then the potentially
useful e0000000 address gets mangled into ffffffff000000.

IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912

...snip...

pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
IVSHMEM: guest pci addr = e0000000, guest h/w addr = 1090912256, size = 20000000
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
1090912256, size = 20000000
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)

and with the 64-bit guest I get this error as well (recall the guest
fails to boot on 64-bit)

BUG: kvm_dirty_pages_log_change: invalid parameters
00000000f0000000-00000000f0ffffff

> If it's seabios, does the following patch help?

The patch doesn't make a difference.  Perhaps it's Qemu then?

>
> diff --git a/src/pciinit.c b/src/pciinit.c
> index b110531..ce07709 100644
> --- a/src/pciinit.c
> +++ b/src/pciinit.c
> @@ -117,11 +117,7 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
>     int is_64bit = !(val & PCI_BASE_ADDRESS_SPACE_IO) &&
>         (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == PCI_BASE_ADDRESS_MEM_TYPE_64;
>     if (is_64bit) {
> -        if (size > 0) {
> -            pci_config_writel(bdf, ofs + 4, 0);
> -        } else {
> -            pci_config_writel(bdf, ofs + 4, ~0);
> -        }
> +        pci_config_writel(bdf, ofs + 4, 0);
>     }
>     return is_64bit;
>  }
>
>
>
>> IVSHMEM: guest pci addr = e0000000, guest h/w addr = 1090912256, size = 20000000
>> IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
>> 1090912256, size = 20000000
>>
>> However, booting fails when I use a 64-bit BAR.  Booting is fine with
>> a 32-bit BAR.
>>
>> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
>> divide error: 0000 [#1] SMP last sysfs file:
>> CPU 0
>> Modules linked in:
>>
>> Pid: 1, comm: swapper Not tainted 2.6.35-rc1 #280 /Bochs
>> RIP: 0010:[<ffffffff812a801b>]  [<ffffffff812a801b>] hpet_alloc+0x12c/0x35b
>> RSP: 0018:ffff88003e61fd80  EFLAGS: 00010246
>> RAX: 00038d7ea4c68000 RBX: ffff88003e6efd80 RCX: 0000000000000000
>> RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff817b8520
>> RBP: ffff88003e61fdc0 R08: 00000000000080d0 R09: ffffc90000000000
>> R10: ffff88003f9ac600 R11: 0000000000000000 R12: ffff88003e61fdd0
>> R13: ffffc90000000000 R14: 0000000000000000 R15: ffffffff817a00a9
>> FS:  0000000000000000(0000) GS:ffff880002000000(0000) knlGS:0000000000000000
>> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> CR2: 0000000000000000 CR3: 0000000001a42000 CR4: 00000000000006f0DR0:
>> 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> Process swapper (pid: 1, threadinfo ffff88003e61e000, task ffff88003e620000)
>> Stack:
>>  ffff88003f43ab90 ffff88003f43ab90 ffffffff81ca3174 ffffffff81b1d596
>> <0> 0000000000000000 0000000000000100 0000000000000100 0000000000000000
>> <0> ffff88003e61fe80 ffffffff8102945d 00000000fed00000 ffffc90000000000
>> Call Trace:
>>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff8102945d>] hpet_reserve_platform_timers+0x10b/0x115
>>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff81b1d601>] hpet_late_init+0x6b/0xea
>>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff81002069>] do_one_initcall+0x5e/0x159
>>  [<ffffffff81b0b71e>] kernel_init+0x18e/0x21c
>>  [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>>  [<ffffffff81b0b590>] ? kernel_init+0x0/0x21c
>>  [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
>> Code: 89 1d 8a 92 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 a9 00 7a 81 48
>> 8d 04 02 4c 89 f2 48 c7 c7 20 85 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
>> f7 f1 83 7b 30 01 48
>> c7 c1 cd fa 7c 81 49 0f 46 cf 48 89 43
>> RIP  [<ffffffff812a801b>] hpet_alloc+0x12c/0x35b
>>  RSP <ffff88003e61fd80>
>> ---[ end trace a7919e7f17c0a725 ]---
>> Kernel panic - not syncing: Attempted to kill init!
>> Pid: 1, comm: swapper Tainted: G      D     2.6.35-rc1 #280
>> Call Trace:
>>  [<ffffffff8145b870>] panic+0x8b/0x10b
>>  [<ffffffff81056a93>] ? exit_ptrace+0x38/0x121
>>  [<ffffffff8104f9e8>] do_exit+0x7a/0x722
>>  [<ffffffff8104c3bd>] ? spin_unlock_irqrestore+0xe/0x10
>>  [<ffffffff8104cfd8>] ? kmsg_dump+0x12b/0x145
>>  [<ffffffff8145eaa9>] oops_end+0xbf/0xc7
>>  [<ffffffff8100d299>] die+0x5a/0x63
>>  [<ffffffff8145e4b3>] do_trap+0x121/0x130
>>  [<ffffffff8100b560>] do_divide_error+0x96/0x9f
>>  [<ffffffff812a801b>] ? hpet_alloc+0x12c/0x35b
>>  [<ffffffff8100a83b>] divide_error+0x1b/0x20
>>  [<ffffffff812a801b>] ? hpet_alloc+0x12c/0x35b
>>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff8102945d>] hpet_reserve_platform_timers+0x10b/0x115
>>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff81b1d601>] hpet_late_init+0x6b/0xea
>>  [<ffffffff81b1d596>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff81002069>] do_one_initcall+0x5e/0x159
>>  [<ffffffff81b0b71e>] kernel_init+0x18e/0x21c
>>  [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>>  [<ffffffff81b0b590>] ? kernel_init+0x0/0x21c
>>  [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
>>
>
> --
> yamahata
>
>

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Isaku Yamahata]

On Wed, Jul 14, 2010 at 09:10:28AM -0600, Cam Macdonell wrote:
> On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> > On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
> >> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> >> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
> >> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
> >> >> >> > thinks the second half of the BAR is an I/O region instead of memory (hence
> >> >> >> > the c200, that's part of the pci portio region.
> >> >> >
> >> >> > I've sent the patches to address it. But they haven't been merged yet.
> >> >> > seabios doesn't map BARs beyond 4GB.
> >> >> > If bar is mapped beyond 4GB, guest BIOS does it.
> >> >>
> >> >> Have those patches been merged yet?
> >> >
> >> > They have been merged into seabios upstream now.
> >> > qemu seabios fork hasn't pulled for a while, though.
> >> >
> >> >
> >> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> >> >> > in config.h of seabios
> >> >>
> >> >> Where does the output from seabios end up? ?Inside dmesg?
> >> >
> >> > It outputs them to the serial console which qemu emulates.
> >> > seabios is out of kernel control, so dmesg doesn't show it.
> >> >
> >> >
> >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> >> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> >> >> >
> >> >> > seabios BAR3. Not sure how it is mapped from this
> >> >> > message.
> >> >>
> >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
> >> >> BAR3 to store all 64-bits?
> >> >
> >> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
> >> > --
> >> > yamahata
> >> >
> >> >
> >>
> >> With the latest seabios git passed via -bios, I no longer see the
> >> 48-bit address, but instead a 32-bit address and then
> >> ffffffff00000000. ?This guest has 1gb of RAM so the address isn't be
> >> mapped beyond 4g.
> >
> > Can I see the debug log like before?
> > (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)
> 
> Here's the dump from SeaBIOS in the region related to the PCI devices.
>  The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit.
> 
> PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8
> region 0: 0xf0000000
> region 1: 0xf2000000
> region 6: 0xf2010000
> PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000
> region 0: 0x0000c020
> region 1: 0xf2020000
> region 6: 0xf2030000
> PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
> region 0: 0xf2040000
> region 1: 0xf2041000
> region 2: 0x00000000

Is this region (region 2 of devfn=0x20: vendor_id=0x1af4 device_id=0x1110)
the BAR in quistion?
The value 0 seems odd. Probably BAR address calculation overflowed.
Currently seabios doesn't check overflow. I attached the patch.


> > Do you know who sets the BAR to ffffffff00000000?
> 
> Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM'
> lines are from the map function passed to pci_register_bar().  It
> looks like SeaBIOS sets the address to 0 and then the potentially
> useful e0000000 address gets mangled into ffffffff000000.

There is something wrong with the debug message of write case, I suppose.
All written value are 0, but the resulted effect doesn't seems so.

> 
> IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912
> 
> ...snip...
> 
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x18 (addr)
> IVSHMEM: guest pci addr = e0000000, guest h/w addr = 1090912256, size = 20000000

If 0 is written to 0x18, the bar address should be 0, but it says e0000000.

> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)

The read value isn't 0. and so on...

> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
> 1090912256, size = 20000000
> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)
> 
> and with the 64-bit guest I get this error as well (recall the guest
> fails to boot on 64-bit)
> 
> BUG: kvm_dirty_pages_log_change: invalid parameters
> 00000000f0000000-00000000f0ffffff


diff --git a/src/pciinit.c b/src/pciinit.c
index b110531..6eca2ce 100644
--- a/src/pciinit.c
+++ b/src/pciinit.c
@@ -90,7 +90,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
                  /* If pci_bios_prefmem_addr == 0, keep old behaviour */
                  pci_bios_prefmem_addr != 0) {
             paddr = &pci_bios_prefmem_addr;
-            if (ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
+            if (ALIGN(*paddr, size) + size < *paddr ||
+                ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
                 dprintf(1,
                         "prefmem region of (bdf 0x%x bar %d) can't be mapped. "
                         "decrease BUILD_PCIMEM_SIZE and recompile. size %x\n",
@@ -99,7 +100,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
             }
         } else {
             paddr = &pci_bios_mem_addr;
-            if (ALIGN(*paddr, size) + size >= BUILD_PCIMEM_END) {
+            if (ALIGN(*paddr, size) + size < *paddr ||
+                ALIGN(*paddr, size) + size >= BUILD_PCIMEM_END) {
                 dprintf(1,
                         "mem region of (bdf 0x%x bar %d) can't be mapped. "
                         "increase BUILD_PCIMEM_SIZE and recompile. size %x\n",
@@ -116,12 +118,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
 
     int is_64bit = !(val & PCI_BASE_ADDRESS_SPACE_IO) &&
         (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == PCI_BASE_ADDRESS_MEM_TYPE_64;
-    if (is_64bit) {
-        if (size > 0) {
-            pci_config_writel(bdf, ofs + 4, 0);
-        } else {
-            pci_config_writel(bdf, ofs + 4, ~0);
-        }
+    if (is_64bit && size > 0) {
+        pci_config_writel(bdf, ofs + 4, 0);
     }
     return is_64bit;
 }

-- 
yamahata

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Michael S. Tsirkin]

On Tue, Jul 20, 2010 at 06:52:23PM +0900, Isaku Yamahata wrote:
> On Wed, Jul 14, 2010 at 09:10:28AM -0600, Cam Macdonell wrote:
> > On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> > > On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
> > >> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> > >> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
> > >> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
> > >> >> >> > thinks the second half of the BAR is an I/O region instead of memory (hence
> > >> >> >> > the c200, that's part of the pci portio region.
> > >> >> >
> > >> >> > I've sent the patches to address it. But they haven't been merged yet.
> > >> >> > seabios doesn't map BARs beyond 4GB.
> > >> >> > If bar is mapped beyond 4GB, guest BIOS does it.
> > >> >>
> > >> >> Have those patches been merged yet?
> > >> >
> > >> > They have been merged into seabios upstream now.
> > >> > qemu seabios fork hasn't pulled for a while, though.
> > >> >
> > >> >
> > >> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> > >> >> > in config.h of seabios
> > >> >>
> > >> >> Where does the output from seabios end up? ?Inside dmesg?
> > >> >
> > >> > It outputs them to the serial console which qemu emulates.
> > >> > seabios is out of kernel control, so dmesg doesn't show it.
> > >> >
> > >> >
> > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> > >> >> >> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> > >> >> >
> > >> >> > seabios BAR3. Not sure how it is mapped from this
> > >> >> > message.
> > >> >>
> > >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
> > >> >> BAR3 to store all 64-bits?
> > >> >
> > >> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
> > >> > --
> > >> > yamahata
> > >> >
> > >> >
> > >>
> > >> With the latest seabios git passed via -bios, I no longer see the
> > >> 48-bit address, but instead a 32-bit address and then
> > >> ffffffff00000000. ?This guest has 1gb of RAM so the address isn't be
> > >> mapped beyond 4g.
> > >
> > > Can I see the debug log like before?
> > > (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)
> > 
> > Here's the dump from SeaBIOS in the region related to the PCI devices.
> >  The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit.
> > 
> > PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8
> > region 0: 0xf0000000
> > region 1: 0xf2000000
> > region 6: 0xf2010000
> > PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000
> > region 0: 0x0000c020
> > region 1: 0xf2020000
> > region 6: 0xf2030000
> > PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
> > region 0: 0xf2040000
> > region 1: 0xf2041000
> > region 2: 0x00000000
> 
> Is this region (region 2 of devfn=0x20: vendor_id=0x1af4 device_id=0x1110)
> the BAR in quistion?
> The value 0 seems odd. Probably BAR address calculation overflowed.
> Currently seabios doesn't check overflow. I attached the patch.
> 
> 
> > > Do you know who sets the BAR to ffffffff00000000?
> > 
> > Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM'
> > lines are from the map function passed to pci_register_bar().  It
> > looks like SeaBIOS sets the address to 0 and then the potentially
> > useful e0000000 address gets mangled into ffffffff000000.
> 
> There is something wrong with the debug message of write case, I suppose.
> All written value are 0, but the resulted effect doesn't seems so.
> 
> > 
> > IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912
> > 
> > ...snip...
> > 
> > pci_read_config: (val) 0x4 <- 0x18 (addr)
> > pci_write_config: (val) 0x0 -> 0x18 (addr)
> > IVSHMEM: guest pci addr = e0000000, guest h/w addr = 1090912256, size = 20000000
> 
> If 0 is written to 0x18, the bar address should be 0, but it says e0000000.
> 
> > pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
> 
> The read value isn't 0. and so on...
> 
> > pci_write_config: (val) 0x0 -> 0x18 (addr)
> > pci_read_config: (val) 0x0 <- 0x1c (addr)
> > pci_write_config: (val) 0x0 -> 0x1c (addr)
> > IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
> > 1090912256, size = 20000000
> > pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> > pci_write_config: (val) 0x0 -> 0x1c (addr)
> > 
> > and with the 64-bit guest I get this error as well (recall the guest
> > fails to boot on 64-bit)
> > 
> > BUG: kvm_dirty_pages_log_change: invalid parameters
> > 00000000f0000000-00000000f0ffffff
> 
> 
> diff --git a/src/pciinit.c b/src/pciinit.c
> index b110531..6eca2ce 100644
> --- a/src/pciinit.c
> +++ b/src/pciinit.c
> @@ -90,7 +90,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
>                   /* If pci_bios_prefmem_addr == 0, keep old behaviour */
>                   pci_bios_prefmem_addr != 0) {
>              paddr = &pci_bios_prefmem_addr;
> -            if (ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
> +            if (ALIGN(*paddr, size) + size < *paddr ||
> +                ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
>                  dprintf(1,
>                          "prefmem region of (bdf 0x%x bar %d) can't be mapped. "
>                          "decrease BUILD_PCIMEM_SIZE and recompile. size %x\n",
> @@ -99,7 +100,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
>              }
>          } else {
>              paddr = &pci_bios_mem_addr;
> -            if (ALIGN(*paddr, size) + size >= BUILD_PCIMEM_END) {
> +            if (ALIGN(*paddr, size) + size < *paddr ||
> +                ALIGN(*paddr, size) + size >= BUILD_PCIMEM_END) {
>                  dprintf(1,
>                          "mem region of (bdf 0x%x bar %d) can't be mapped. "
>                          "increase BUILD_PCIMEM_SIZE and recompile. size %x\n",

Looking at the source, all of the values like pci_bios_prefmem_addr seem to be
32 bit. Since in the spec prefetcheable memory is up to 64 bit,
can't the math overflow, here and elsewhere?
Maybe we should switch to 64 bit values all over ...

> @@ -116,12 +118,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
>  
>      int is_64bit = !(val & PCI_BASE_ADDRESS_SPACE_IO) &&
>          (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == PCI_BASE_ADDRESS_MEM_TYPE_64;
> -    if (is_64bit) {
> -        if (size > 0) {
> -            pci_config_writel(bdf, ofs + 4, 0);
> -        } else {
> -            pci_config_writel(bdf, ofs + 4, ~0);
> -        }
> +    if (is_64bit && size > 0) {
> +        pci_config_writel(bdf, ofs + 4, 0);
>      }
>      return is_64bit;
>  }


Was there any reason we wrote all-ones there on size 0?
BAR sizing?

> -- 
> yamahata

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Isaku Yamahata]

Added Cc: seabios@seabios.org

On Wed, Jul 21, 2010 at 06:31:01AM +0300, Michael S. Tsirkin wrote:
> On Tue, Jul 20, 2010 at 06:52:23PM +0900, Isaku Yamahata wrote:
> > On Wed, Jul 14, 2010 at 09:10:28AM -0600, Cam Macdonell wrote:
> > > On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> > > > On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
> > > >> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> > > >> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
> > > >> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
> > > >> >> >> > thinks the second half of the BAR is an I/O region instead of memory (hence
> > > >> >> >> > the c200, that's part of the pci portio region.
> > > >> >> >
> > > >> >> > I've sent the patches to address it. But they haven't been merged yet.
> > > >> >> > seabios doesn't map BARs beyond 4GB.
> > > >> >> > If bar is mapped beyond 4GB, guest BIOS does it.
> > > >> >>
> > > >> >> Have those patches been merged yet?
> > > >> >
> > > >> > They have been merged into seabios upstream now.
> > > >> > qemu seabios fork hasn't pulled for a while, though.
> > > >> >
> > > >> >
> > > >> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
> > > >> >> > in config.h of seabios
> > > >> >>
> > > >> >> Where does the output from seabios end up? ?Inside dmesg?
> > > >> >
> > > >> > It outputs them to the serial console which qemu emulates.
> > > >> > seabios is out of kernel control, so dmesg doesn't show it.
> > > >> >
> > > >> >
> > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> > > >> >> >> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
> > > >> >> >
> > > >> >> > seabios BAR3. Not sure how it is mapped from this
> > > >> >> > message.
> > > >> >>
> > > >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
> > > >> >> BAR3 to store all 64-bits?
> > > >> >
> > > >> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
> > > >> > --
> > > >> > yamahata
> > > >> >
> > > >> >
> > > >>
> > > >> With the latest seabios git passed via -bios, I no longer see the
> > > >> 48-bit address, but instead a 32-bit address and then
> > > >> ffffffff00000000. ?This guest has 1gb of RAM so the address isn't be
> > > >> mapped beyond 4g.
> > > >
> > > > Can I see the debug log like before?
> > > > (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)
> > > 
> > > Here's the dump from SeaBIOS in the region related to the PCI devices.
> > >  The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit.
> > > 
> > > PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8
> > > region 0: 0xf0000000
> > > region 1: 0xf2000000
> > > region 6: 0xf2010000
> > > PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000
> > > region 0: 0x0000c020
> > > region 1: 0xf2020000
> > > region 6: 0xf2030000
> > > PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
> > > region 0: 0xf2040000
> > > region 1: 0xf2041000
> > > region 2: 0x00000000
> > 
> > Is this region (region 2 of devfn=0x20: vendor_id=0x1af4 device_id=0x1110)
> > the BAR in quistion?
> > The value 0 seems odd. Probably BAR address calculation overflowed.
> > Currently seabios doesn't check overflow. I attached the patch.
> > 
> > 
> > > > Do you know who sets the BAR to ffffffff00000000?
> > > 
> > > Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM'
> > > lines are from the map function passed to pci_register_bar().  It
> > > looks like SeaBIOS sets the address to 0 and then the potentially
> > > useful e0000000 address gets mangled into ffffffff000000.
> > 
> > There is something wrong with the debug message of write case, I suppose.
> > All written value are 0, but the resulted effect doesn't seems so.
> > 
> > > 
> > > IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912
> > > 
> > > ...snip...
> > > 
> > > pci_read_config: (val) 0x4 <- 0x18 (addr)
> > > pci_write_config: (val) 0x0 -> 0x18 (addr)
> > > IVSHMEM: guest pci addr = e0000000, guest h/w addr = 1090912256, size = 20000000
> > 
> > If 0 is written to 0x18, the bar address should be 0, but it says e0000000.
> > 
> > > pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
> > 
> > The read value isn't 0. and so on...
> > 
> > > pci_write_config: (val) 0x0 -> 0x18 (addr)
> > > pci_read_config: (val) 0x0 <- 0x1c (addr)
> > > pci_write_config: (val) 0x0 -> 0x1c (addr)
> > > IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
> > > 1090912256, size = 20000000
> > > pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> > > pci_write_config: (val) 0x0 -> 0x1c (addr)
> > > 
> > > and with the 64-bit guest I get this error as well (recall the guest
> > > fails to boot on 64-bit)
> > > 
> > > BUG: kvm_dirty_pages_log_change: invalid parameters
> > > 00000000f0000000-00000000f0ffffff
> > 
> > 
> > diff --git a/src/pciinit.c b/src/pciinit.c
> > index b110531..6eca2ce 100644
> > --- a/src/pciinit.c
> > +++ b/src/pciinit.c
> > @@ -90,7 +90,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
> >                   /* If pci_bios_prefmem_addr == 0, keep old behaviour */
> >                   pci_bios_prefmem_addr != 0) {
> >              paddr = &pci_bios_prefmem_addr;
> > -            if (ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
> > +            if (ALIGN(*paddr, size) + size < *paddr ||
> > +                ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
> >                  dprintf(1,
> >                          "prefmem region of (bdf 0x%x bar %d) can't be mapped. "
> >                          "decrease BUILD_PCIMEM_SIZE and recompile. size %x\n",
> > @@ -99,7 +100,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
> >              }
> >          } else {
> >              paddr = &pci_bios_mem_addr;
> > -            if (ALIGN(*paddr, size) + size >= BUILD_PCIMEM_END) {
> > +            if (ALIGN(*paddr, size) + size < *paddr ||
> > +                ALIGN(*paddr, size) + size >= BUILD_PCIMEM_END) {
> >                  dprintf(1,
> >                          "mem region of (bdf 0x%x bar %d) can't be mapped. "
> >                          "increase BUILD_PCIMEM_SIZE and recompile. size %x\n",
> 
> Looking at the source, all of the values like pci_bios_prefmem_addr seem to be
> 32 bit. Since in the spec prefetcheable memory is up to 64 bit,
> can't the math overflow, here and elsewhere?
> Maybe we should switch to 64 bit values all over ...

Make sense. I'll create a patch to convert them into u64.

> 
> > @@ -116,12 +118,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
> >  
> >      int is_64bit = !(val & PCI_BASE_ADDRESS_SPACE_IO) &&
> >          (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == PCI_BASE_ADDRESS_MEM_TYPE_64;
> > -    if (is_64bit) {
> > -        if (size > 0) {
> > -            pci_config_writel(bdf, ofs + 4, 0);
> > -        } else {
> > -            pci_config_writel(bdf, ofs + 4, ~0);
> > -        }
> > +    if (is_64bit && size > 0) {
> > +        pci_config_writel(bdf, ofs + 4, 0);
> >      }
> >      return is_64bit;
> >  }
> 
> 
> Was there any reason we wrote all-ones there on size 0?
> BAR sizing?

No reason. It's just left over from debugging.
So I'd like to remove it.

-- 
yamahata

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Tue, Jul 20, 2010 at 9:49 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> Added Cc: seabios@seabios.org
>
> On Wed, Jul 21, 2010 at 06:31:01AM +0300, Michael S. Tsirkin wrote:
>> On Tue, Jul 20, 2010 at 06:52:23PM +0900, Isaku Yamahata wrote:
>> > On Wed, Jul 14, 2010 at 09:10:28AM -0600, Cam Macdonell wrote:
>> > > On Tue, Jul 13, 2010 at 8:52 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
>> > > > On Tue, Jul 13, 2010 at 04:48:19PM -0600, Cam Macdonell wrote:
>> > > >> On Tue, Jul 13, 2010 at 2:41 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
>> > > >> > On Tue, Jul 13, 2010 at 02:05:51PM -0600, Cam Macdonell wrote:
>> > > >> >> >> > Seabios completely ignore the 64-bitness of the BAR. ?Looks like it also
>> > > >> >> >> > thinks the second half of the BAR is an I/O region instead of memory (hence
>> > > >> >> >> > the c200, that's part of the pci portio region.
>> > > >> >> >
>> > > >> >> > I've sent the patches to address it. But they haven't been merged yet.
>> > > >> >> > seabios doesn't map BARs beyond 4GB.
>> > > >> >> > If bar is mapped beyond 4GB, guest BIOS does it.
>> > > >> >>
>> > > >> >> Have those patches been merged yet?
>> > > >> >
>> > > >> > They have been merged into seabios upstream now.
>> > > >> > qemu seabios fork hasn't pulled for a while, though.
>> > > >> >
>> > > >> >
>> > > >> >> > To see how seabios works, it would help to increase CONFIG_DEBUG_LEVEL
>> > > >> >> > in config.h of seabios
>> > > >> >>
>> > > >> >> Where does the output from seabios end up? ?Inside dmesg?
>> > > >> >
>> > > >> > It outputs them to the serial console which qemu emulates.
>> > > >> > seabios is out of kernel control, so dmesg doesn't show it.
>> > > >> >
>> > > >> >
>> > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> > > >> >> >> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> > > >> >> >> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> > > >> >> >> pci_write_config: (val) 0x0 -> 0x1c (addr)
>> > > >> >> >
>> > > >> >> > seabios BAR3. Not sure how it is mapped from this
>> > > >> >> > message.
>> > > >> >>
>> > > >> >> Isn't the BAR3 from the fact that a 64-bit BAR would use both BAR2 and
>> > > >> >> BAR3 to store all 64-bits?
>> > > >> >
>> > > >> > Yes. Seabios misbehaves. 64bit bar is(was) a missing feature.
>> > > >> > --
>> > > >> > yamahata
>> > > >> >
>> > > >> >
>> > > >>
>> > > >> With the latest seabios git passed via -bios, I no longer see the
>> > > >> 48-bit address, but instead a 32-bit address and then
>> > > >> ffffffff00000000. ?This guest has 1gb of RAM so the address isn't be
>> > > >> mapped beyond 4g.
>> > > >
>> > > > Can I see the debug log like before?
>> > > > (hopefully seabios with CONFIG_DEBUG_LEVEL enabled.)
>> > >
>> > > Here's the dump from SeaBIOS in the region related to the PCI devices.
>> > >  The SeaBIOS output is identical whether the BAR is 32-bit or 64-bit.
>> > >
>> > > PCI: bus=0 devfn=0x10: vendor_id=0x1013 device_id=0x00b8
>> > > region 0: 0xf0000000
>> > > region 1: 0xf2000000
>> > > region 6: 0xf2010000
>> > > PCI: bus=0 devfn=0x18: vendor_id=0x1af4 device_id=0x1000
>> > > region 0: 0x0000c020
>> > > region 1: 0xf2020000
>> > > region 6: 0xf2030000
>> > > PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
>> > > region 0: 0xf2040000
>> > > region 1: 0xf2041000
>> > > region 2: 0x00000000
>> >
>> > Is this region (region 2 of devfn=0x20: vendor_id=0x1af4 device_id=0x1110)
>> > the BAR in quistion?
>> > The value 0 seems odd. Probably BAR address calculation overflowed.
>> > Currently seabios doesn't check overflow. I attached the patch.
>> >
>> >
>> > > > Do you know who sets the BAR to ffffffff00000000?
>> > >
>> > > Here are the config reads/writes related to the 0x18/1c, the 'IVSHMEM'
>> > > lines are from the map function passed to pci_register_bar().  It
>> > > looks like SeaBIOS sets the address to 0 and then the potentially
>> > > useful e0000000 address gets mangled into ffffffff000000.
>> >
>> > There is something wrong with the debug message of write case, I suppose.
>> > All written value are 0, but the resulted effect doesn't seems so.
>> >
>> > >
>> > > IVSHMEM: guest pci addr = 0, guest h/w addr = 1090912256, size = 536870912
>> > >
>> > > ...snip...
>> > >
>> > > pci_read_config: (val) 0x4 <- 0x18 (addr)
>> > > pci_write_config: (val) 0x0 -> 0x18 (addr)
>> > > IVSHMEM: guest pci addr = e0000000, guest h/w addr = 1090912256, size = 20000000
>> >
>> > If 0 is written to 0x18, the bar address should be 0, but it says e0000000.
>> >
>> > > pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
>> >
>> > The read value isn't 0. and so on...
>> >
>> > > pci_write_config: (val) 0x0 -> 0x18 (addr)
>> > > pci_read_config: (val) 0x0 <- 0x1c (addr)
>> > > pci_write_config: (val) 0x0 -> 0x1c (addr)
>> > > IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
>> > > 1090912256, size = 20000000
>> > > pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> > > pci_write_config: (val) 0x0 -> 0x1c (addr)
>> > >
>> > > and with the 64-bit guest I get this error as well (recall the guest
>> > > fails to boot on 64-bit)
>> > >
>> > > BUG: kvm_dirty_pages_log_change: invalid parameters
>> > > 00000000f0000000-00000000f0ffffff
>> >
>> >
>> > diff --git a/src/pciinit.c b/src/pciinit.c
>> > index b110531..6eca2ce 100644
>> > --- a/src/pciinit.c
>> > +++ b/src/pciinit.c
>> > @@ -90,7 +90,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
>> >                   /* If pci_bios_prefmem_addr == 0, keep old behaviour */
>> >                   pci_bios_prefmem_addr != 0) {
>> >              paddr = &pci_bios_prefmem_addr;
>> > -            if (ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
>> > +            if (ALIGN(*paddr, size) + size < *paddr ||
>> > +                ALIGN(*paddr, size) + size >= BUILD_PCIPREFMEM_END) {
>> >                  dprintf(1,
>> >                          "prefmem region of (bdf 0x%x bar %d) can't be mapped. "
>> >                          "decrease BUILD_PCIMEM_SIZE and recompile. size %x\n",
>> > @@ -99,7 +100,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
>> >              }
>> >          } else {
>> >              paddr = &pci_bios_mem_addr;
>> > -            if (ALIGN(*paddr, size) + size >= BUILD_PCIMEM_END) {
>> > +            if (ALIGN(*paddr, size) + size < *paddr ||
>> > +                ALIGN(*paddr, size) + size >= BUILD_PCIMEM_END) {
>> >                  dprintf(1,
>> >                          "mem region of (bdf 0x%x bar %d) can't be mapped. "
>> >                          "increase BUILD_PCIMEM_SIZE and recompile. size %x\n",
>>
>> Looking at the source, all of the values like pci_bios_prefmem_addr seem to be
>> 32 bit. Since in the spec prefetcheable memory is up to 64 bit,
>> can't the math overflow, here and elsewhere?
>> Maybe we should switch to 64 bit values all over ...
>
> Make sense. I'll create a patch to convert them into u64.
>
>>
>> > @@ -116,12 +118,8 @@ static int pci_bios_allocate_region(u16 bdf, int region_num)
>> >
>> >      int is_64bit = !(val & PCI_BASE_ADDRESS_SPACE_IO) &&
>> >          (val & PCI_BASE_ADDRESS_MEM_TYPE_MASK) == PCI_BASE_ADDRESS_MEM_TYPE_64;
>> > -    if (is_64bit) {
>> > -        if (size > 0) {
>> > -            pci_config_writel(bdf, ofs + 4, 0);
>> > -        } else {
>> > -            pci_config_writel(bdf, ofs + 4, ~0);
>> > -        }
>> > +    if (is_64bit && size > 0) {
>> > +        pci_config_writel(bdf, ofs + 4, 0);
>> >      }
>> >      return is_64bit;
>> >  }
>>
>>
>> Was there any reason we wrote all-ones there on size 0?
>> BAR sizing?
>
> No reason. It's just left over from debugging.
> So I'd like to remove it.
>
> --
> yamahata
>
>

Hi, 64-bit BARs still do not seem to be working.

When using the latest seabios the guest does not hit a "BUG:"
statement, but booting still fails

HPET: 1 timers in total, 0 timers will be used for per-cpu timer
divide error: 0000 [#1] SMP
last sysfs file:
CPU 0
Modules linked in:

Pid: 1, comm: swapper Not tainted 2.6.35+ #299 /Bochs
RIP: 0010:[<ffffffff812a9b5b>]  [<ffffffff812a9b5b>] hpet_alloc+0x12c/0x35b
RSP: 0018:ffff88007d7b3d80  EFLAGS: 00010246
RAX: 00038d7ea4c68000 RBX: ffff88007d062cc0 RCX: 0000000000000000
RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff817bb9b0
RBP: ffff88007d7b3dc0 R08: 00000000000080d0 R09: ffffc90000000000
R10: ffff88007d72b5a0 R11: 0000000000000000 R12: ffff88007d7b3dd0
R13: ffffc90000000000 R14: 0000000000000000 R15: ffffffff817a41c3
FS:  0000000000000000(0000) GS:ffff880002000000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 0000000000000000 CR3: 0000000001a42000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process swapper (pid: 1, threadinfo ffff88007d7b2000, task ffff88007d7b8000)
Stack:
 ffff88007f43ab90 ffff88007f43ab90 ffffffff81ca6174 ffffffff81b1f5e1
<0> 0000000000000000 0000000000000100 0000000000000100 0000000000000000
<0> ffff88007d7b3e80 ffffffff810294ea 00000000fed00000 ffffc90000000000
Call Trace:
 [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
 [<ffffffff810294ea>] hpet_reserve_platform_timers+0x10b/0x115
 [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
 [<ffffffff81b1f64c>] hpet_late_init+0x6b/0xea
 [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
 [<ffffffff81002069>] do_one_initcall+0x5e/0x159
 [<ffffffff81b0d72a>] kernel_init+0x19a/0x228
 [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
 [<ffffffff81b0d590>] ? kernel_init+0x0/0x228
 [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
Code: 89 1d ca b2 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 c3 41 7a 81 48
8d 04 02 4c 89 f2 48 c7 c7 b0 b9 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
f7 f1 83 7b 30 01 48 c7 c1 86 1c 7d 81 49 0f 46 cf 48 89 43
RIP  [<ffffffff812a9b5b>] hpet_alloc+0x12c/0x35b
 RSP <ffff88007d7b3d80>
---[ end trace a7919e7f17c0a725 ]---
Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: swapper Tainted: G      D     2.6.35+ #299
Call Trace:
 [<ffffffff81459a85>] panic+0x8b/0x10b
 [<ffffffff81056a83>] ? exit_ptrace+0x38/0x121
 [<ffffffff8104f9e8>] do_exit+0x7a/0x722
 [<ffffffff8104c3bd>] ? spin_unlock_irqrestore+0xe/0x10
 [<ffffffff8104cfd6>] ? kmsg_dump+0x12b/0x145
 [<ffffffff8145ccc8>] oops_end+0xbf/0xc7
 [<ffffffff8100d299>] die+0x5a/0x63
 [<ffffffff8145c6d2>] do_trap+0x121/0x130
 [<ffffffff8100b560>] do_divide_error+0x96/0x9f
 [<ffffffff812a9b5b>] ? hpet_alloc+0x12c/0x35b
 [<ffffffff8120cf80>] ? radix_tree_preload+0x34/0x88
 [<ffffffff8100a83b>] divide_error+0x1b/0x20
 [<ffffffff812a9b5b>] ? hpet_alloc+0x12c/0x35b
 [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
 [<ffffffff810294ea>] hpet_reserve_platform_timers+0x10b/0x115
 [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
 [<ffffffff81b1f64c>] hpet_late_init+0x6b/0xea
 [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
 [<ffffffff81002069>] do_one_initcall+0x5e/0x159
 [<ffffffff81b0d72a>] kernel_init+0x19a/0x228
 [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
 [<ffffffff81b0d590>] ? kernel_init+0x0/0x228
 [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10

seabios output for the device:

PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
region 0: 0xf1020000
region 2: 0x00000000
init smm

Running the latest seabios, the debug output only remaps the BAR
twice, once with a potentially correct address of e00000000

pci_read_config: (val) 0xe0000004 <- 0x18 (addr)

...snip...

pci_default_write_config: (val) 0x0 -> 0x18 (addr)
IVSHMEM: guest pci addr = e0000000, guest h/w addr = 2164588544, size = 20000000
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_default_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_default_write_config: (val) 0x0 -> 0x1c (addr)
IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
2164588544, size = 20000000
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_default_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x0 <- 0x20 (addr)

the pci writes are all still 0, I can't see how my debug statements
are incorrect though.  Below is my trivial pci config debugging patch.

diff --git a/hw/pci.c b/hw/pci.c
index 70dbace..01087b1 100644
--- a/hw/pci.c
+++ b/hw/pci.c
@@ -1159,6 +1159,8 @@ static uint32_t pci_read_config(PCIDevice *d,

     len = MIN(len, pci_config_size(d) - address);
     memcpy(&val, d->config + address, len);
+    if (strncmp(d->name, "ivshmem", 7) == 0)
+        printf("pci_read_config: (val) 0x%x <- 0x%x (addr)\n", val, address);
     return le32_to_cpu(val);
 }

@@ -1219,6 +1221,8 @@ void pci_default_write_config(PCIDevice *d,
uint32_t addr, uint32_t val, int l)
         d->config[addr + i] = (d->config[addr + i] & ~wmask) | (val & wmask);
     }

+    if (strncmp(d->name, "ivshmem", 7) == 0)
+        printf("pci_write_config: (val) 0x%x -> 0x%x (addr)\n", val, addr);
 #ifdef CONFIG_KVM_DEVICE_ASSIGNMENT
     if (kvm_enabled() && kvm_irqchip_in_kernel() &&
         addr >= PIIX_CONFIG_IRQ_ROUTE &&

Cam

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Isaku Yamahata]

On Tue, Aug 24, 2010 at 10:52:36AM -0600, Cam Macdonell wrote:
> Hi, 64-bit BARs still do not seem to be working.
> 
> When using the latest seabios the guest does not hit a "BUG:"
> statement, but booting still fails
> 
> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
> divide error: 0000 [#1] SMP
> last sysfs file:
> CPU 0
> Modules linked in:
> 
> Pid: 1, comm: swapper Not tainted 2.6.35+ #299 /Bochs
> RIP: 0010:[<ffffffff812a9b5b>]  [<ffffffff812a9b5b>] hpet_alloc+0x12c/0x35b
> RSP: 0018:ffff88007d7b3d80  EFLAGS: 00010246
> RAX: 00038d7ea4c68000 RBX: ffff88007d062cc0 RCX: 0000000000000000
> RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff817bb9b0
> RBP: ffff88007d7b3dc0 R08: 00000000000080d0 R09: ffffc90000000000
> R10: ffff88007d72b5a0 R11: 0000000000000000 R12: ffff88007d7b3dd0
> R13: ffffc90000000000 R14: 0000000000000000 R15: ffffffff817a41c3
> FS:  0000000000000000(0000) GS:ffff880002000000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 0000000000000000 CR3: 0000000001a42000 CR4: 00000000000006f0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process swapper (pid: 1, threadinfo ffff88007d7b2000, task ffff88007d7b8000)
> Stack:
>  ffff88007f43ab90 ffff88007f43ab90 ffffffff81ca6174 ffffffff81b1f5e1
> <0> 0000000000000000 0000000000000100 0000000000000100 0000000000000000
> <0> ffff88007d7b3e80 ffffffff810294ea 00000000fed00000 ffffc90000000000
> Call Trace:
>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>  [<ffffffff810294ea>] hpet_reserve_platform_timers+0x10b/0x115
>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>  [<ffffffff81b1f64c>] hpet_late_init+0x6b/0xea
>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>  [<ffffffff81002069>] do_one_initcall+0x5e/0x159
>  [<ffffffff81b0d72a>] kernel_init+0x19a/0x228
>  [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>  [<ffffffff81b0d590>] ? kernel_init+0x0/0x228
>  [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
> Code: 89 1d ca b2 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 c3 41 7a 81 48
> 8d 04 02 4c 89 f2 48 c7 c7 b0 b9 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
> f7 f1 83 7b 30 01 48 c7 c1 86 1c 7d 81 49 0f 46 cf 48 89 43
> RIP  [<ffffffff812a9b5b>] hpet_alloc+0x12c/0x35b
>  RSP <ffff88007d7b3d80>
> ---[ end trace a7919e7f17c0a725 ]---
> Kernel panic - not syncing: Attempted to kill init!
> Pid: 1, comm: swapper Tainted: G      D     2.6.35+ #299
> Call Trace:
>  [<ffffffff81459a85>] panic+0x8b/0x10b
>  [<ffffffff81056a83>] ? exit_ptrace+0x38/0x121
>  [<ffffffff8104f9e8>] do_exit+0x7a/0x722
>  [<ffffffff8104c3bd>] ? spin_unlock_irqrestore+0xe/0x10
>  [<ffffffff8104cfd6>] ? kmsg_dump+0x12b/0x145
>  [<ffffffff8145ccc8>] oops_end+0xbf/0xc7
>  [<ffffffff8100d299>] die+0x5a/0x63
>  [<ffffffff8145c6d2>] do_trap+0x121/0x130
>  [<ffffffff8100b560>] do_divide_error+0x96/0x9f
>  [<ffffffff812a9b5b>] ? hpet_alloc+0x12c/0x35b
>  [<ffffffff8120cf80>] ? radix_tree_preload+0x34/0x88
>  [<ffffffff8100a83b>] divide_error+0x1b/0x20
>  [<ffffffff812a9b5b>] ? hpet_alloc+0x12c/0x35b
>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>  [<ffffffff810294ea>] hpet_reserve_platform_timers+0x10b/0x115
>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>  [<ffffffff81b1f64c>] hpet_late_init+0x6b/0xea
>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>  [<ffffffff81002069>] do_one_initcall+0x5e/0x159
>  [<ffffffff81b0d72a>] kernel_init+0x19a/0x228
>  [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>  [<ffffffff81b0d590>] ? kernel_init+0x0/0x228
>  [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
> 
> seabios output for the device:
> 
> PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
> region 0: 0xf1020000
> region 2: 0x00000000
> init smm
> 
> Running the latest seabios, the debug output only remaps the BAR
> twice, once with a potentially correct address of e00000000
> 
> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)

The upstream seabios lacks overflow check at the moment.
I haven't found time to address PMM yet.


> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
> IVSHMEM: guest pci addr = e0000000, guest h/w addr = 2164588544, size = 20000000
> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
> IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
> 2164588544, size = 20000000
> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
> pci_read_config: (val) 0x0 <- 0x20 (addr)
> 
> the pci writes are all still 0, I can't see how my debug statements
> are incorrect though.  Below is my trivial pci config debugging patch.

The debug out put should be before the for-loop.

    for (i = 0; i < l && addr + i < config_size; val >>= 8, ++i) {
                                                 ^^^^^^^^^ 
                                                 Here val becomes 0
        uint8_t wmask = d->wmask[addr + i];
        d->config[addr + i] = (d->config[addr + i] & ~wmask) | (val & wmask);
    }

-- 
yamahata

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Cam Macdonell]

On Tue, Aug 24, 2010 at 8:21 PM, Isaku Yamahata <yamahata@valinux.co.jp> wrote:
> On Tue, Aug 24, 2010 at 10:52:36AM -0600, Cam Macdonell wrote:
>> Hi, 64-bit BARs still do not seem to be working.
>>
>> When using the latest seabios the guest does not hit a "BUG:"
>> statement, but booting still fails
>>
>> HPET: 1 timers in total, 0 timers will be used for per-cpu timer
>> divide error: 0000 [#1] SMP
>> last sysfs file:
>> CPU 0
>> Modules linked in:
>>
>> Pid: 1, comm: swapper Not tainted 2.6.35+ #299 /Bochs
>> RIP: 0010:[<ffffffff812a9b5b>]  [<ffffffff812a9b5b>] hpet_alloc+0x12c/0x35b
>> RSP: 0018:ffff88007d7b3d80  EFLAGS: 00010246
>> RAX: 00038d7ea4c68000 RBX: ffff88007d062cc0 RCX: 0000000000000000
>> RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffff817bb9b0
>> RBP: ffff88007d7b3dc0 R08: 00000000000080d0 R09: ffffc90000000000
>> R10: ffff88007d72b5a0 R11: 0000000000000000 R12: ffff88007d7b3dd0
>> R13: ffffc90000000000 R14: 0000000000000000 R15: ffffffff817a41c3
>> FS:  0000000000000000(0000) GS:ffff880002000000(0000) knlGS:0000000000000000
>> CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> CR2: 0000000000000000 CR3: 0000000001a42000 CR4: 00000000000006f0
>> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> Process swapper (pid: 1, threadinfo ffff88007d7b2000, task ffff88007d7b8000)
>> Stack:
>>  ffff88007f43ab90 ffff88007f43ab90 ffffffff81ca6174 ffffffff81b1f5e1
>> <0> 0000000000000000 0000000000000100 0000000000000100 0000000000000000
>> <0> ffff88007d7b3e80 ffffffff810294ea 00000000fed00000 ffffc90000000000
>> Call Trace:
>>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff810294ea>] hpet_reserve_platform_timers+0x10b/0x115
>>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff81b1f64c>] hpet_late_init+0x6b/0xea
>>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff81002069>] do_one_initcall+0x5e/0x159
>>  [<ffffffff81b0d72a>] kernel_init+0x19a/0x228
>>  [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>>  [<ffffffff81b0d590>] ? kernel_init+0x0/0x228
>>  [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
>> Code: 89 1d ca b2 b3 00 48 c1 ea 21 8b 73 34 49 c7 c7 c3 41 7a 81 48
>> 8d 04 02 4c 89 f2 48 c7 c7 b0 b9 7b 81 48 c1 ea 20 48 89 d1 31 d2 <48>
>> f7 f1 83 7b 30 01 48 c7 c1 86 1c 7d 81 49 0f 46 cf 48 89 43
>> RIP  [<ffffffff812a9b5b>] hpet_alloc+0x12c/0x35b
>>  RSP <ffff88007d7b3d80>
>> ---[ end trace a7919e7f17c0a725 ]---
>> Kernel panic - not syncing: Attempted to kill init!
>> Pid: 1, comm: swapper Tainted: G      D     2.6.35+ #299
>> Call Trace:
>>  [<ffffffff81459a85>] panic+0x8b/0x10b
>>  [<ffffffff81056a83>] ? exit_ptrace+0x38/0x121
>>  [<ffffffff8104f9e8>] do_exit+0x7a/0x722
>>  [<ffffffff8104c3bd>] ? spin_unlock_irqrestore+0xe/0x10
>>  [<ffffffff8104cfd6>] ? kmsg_dump+0x12b/0x145
>>  [<ffffffff8145ccc8>] oops_end+0xbf/0xc7
>>  [<ffffffff8100d299>] die+0x5a/0x63
>>  [<ffffffff8145c6d2>] do_trap+0x121/0x130
>>  [<ffffffff8100b560>] do_divide_error+0x96/0x9f
>>  [<ffffffff812a9b5b>] ? hpet_alloc+0x12c/0x35b
>>  [<ffffffff8120cf80>] ? radix_tree_preload+0x34/0x88
>>  [<ffffffff8100a83b>] divide_error+0x1b/0x20
>>  [<ffffffff812a9b5b>] ? hpet_alloc+0x12c/0x35b
>>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff810294ea>] hpet_reserve_platform_timers+0x10b/0x115
>>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff81b1f64c>] hpet_late_init+0x6b/0xea
>>  [<ffffffff81b1f5e1>] ? hpet_late_init+0x0/0xea
>>  [<ffffffff81002069>] do_one_initcall+0x5e/0x159
>>  [<ffffffff81b0d72a>] kernel_init+0x19a/0x228
>>  [<ffffffff8100aa24>] kernel_thread_helper+0x4/0x10
>>  [<ffffffff81b0d590>] ? kernel_init+0x0/0x228
>>  [<ffffffff8100aa20>] ? kernel_thread_helper+0x0/0x10
>>
>> seabios output for the device:
>>
>> PCI: bus=0 devfn=0x20: vendor_id=0x1af4 device_id=0x1110
>> region 0: 0xf1020000
>> region 2: 0x00000000
>> init smm
>>
>> Running the latest seabios, the debug output only remaps the BAR
>> twice, once with a potentially correct address of e00000000
>>
>> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
>
> The upstream seabios lacks overflow check at the moment.
> I haven't found time to address PMM yet.
>
>
>> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
>> IVSHMEM: guest pci addr = e0000000, guest h/w addr = 2164588544, size = 20000000
>> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
>> pci_default_write_config: (val) 0x0 -> 0x18 (addr)
>> pci_read_config: (val) 0x0 <- 0x1c (addr)
>> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
>> IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
>> 2164588544, size = 20000000
>> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
>> pci_default_write_config: (val) 0x0 -> 0x1c (addr)
>> pci_read_config: (val) 0x0 <- 0x20 (addr)
>>
>> the pci writes are all still 0, I can't see how my debug statements
>> are incorrect though.  Below is my trivial pci config debugging patch.
>
> The debug out put should be before the for-loop.
>
>    for (i = 0; i < l && addr + i < config_size; val >>= 8, ++i) {
>                                                 ^^^^^^^^^
>                                                 Here val becomes 0
>        uint8_t wmask = d->wmask[addr + i];
>        d->config[addr + i] = (d->config[addr + i] & ~wmask) | (val & wmask);
>    }
>
> --
> yamahata
>
>

Ah, thanks.  I moved the debug statement and now the writes are the
proper values.

In seabios-kvm, it seems the guest is writing the address of c040 to
0x1c which results to the 48-bit address c04000000000.

pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x1110 <- 0x2 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x0 <- 0xe (addr)
pci_read_config: (val) 0x500 <- 0xa (addr)
pci_read_config: (val) 0x1af4 <- 0x0 (addr)
pci_read_config: (val) 0x1110 <- 0x2 (addr)
pci_read_config: (val) 0x0 <- 0x10 (addr)
pci_write_config: (val) 0xffffffff -> 0x10 (addr)
pci_read_config: (val) 0xffffff00 <- 0x10 (addr)
pci_write_config: (val) 0x0 -> 0x10 (addr)
pci_read_config: (val) 0x0 <- 0x10 (addr)
pci_write_config: (val) 0xf1020000 -> 0x10 (addr)
pci_read_config: (val) 0x0 <- 0x14 (addr)
pci_write_config: (val) 0xffffffff -> 0x14 (addr)
pci_read_config: (val) 0x0 <- 0x14 (addr)
pci_write_config: (val) 0x0 -> 0x14 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0xffffffff -> 0x18 (addr)
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x4 -> 0x18 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0xffffffff -> 0x1c (addr)
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0xc040 -> 0x1c (addr)

<snip>

pci_read_config: (val) 0x0 <- 0x4 (addr)
pci_write_config: (val) 0x3 -> 0x4 (addr)
IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 2164588544,
size = 20000000
pci_read_config: (val) 0x1 <- 0x3d (addr)

<snip>

pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0xffffffff -> 0x18 (addr)
IVSHMEM: guest pci addr = c040e0000000, guest h/w addr = 2164588544,
size = 20000000
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x4 -> 0x18 (addr)
IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 2164588544,
size = 20000000
pci_read_config: (val) 0xc040 <- 0x1c (addr)
pci_write_config: (val) 0xffffffff -> 0x1c (addr)
IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
2164588544, size = 20000000
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0xc040 -> 0x1c (addr)
IVSHMEM: guest pci addr = c04000000000, guest h/w addr = 2164588544,
size = 20000000
pci_read_config: (val) 0x0 <- 0x20 (addr)
pci_write_config: (val) 0xffffffff -> 0x20 (addr)
pci_read_config: (val) 0x0 <- 0x20 (addr)
pci_write_config: (val) 0x0 -> 0x20 (addr)

In upstream seabios.git, the c040 is not written, but the device
returns ffffffff from 0x1c (only reads and writes to 0x18 and 0x1c are
shown below)

pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0xffffffff -> 0x18 (addr)
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x4 -> 0x18 (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x18 (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)
pci_read_config: (val) 0x4 <- 0x18 (addr)
pci_write_config: (val) 0xffffffff -> 0x18 (addr)
IVSHMEM: guest pci addr = e0000000, guest h/w addr = 2164588544, size = 20000000
pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
pci_write_config: (val) 0x4 -> 0x18 (addr)
pci_read_config: (val) 0x0 <- 0x1c (addr)
pci_write_config: (val) 0xffffffff -> 0x1c (addr)
IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
2164588544, size = 20000000
pci_read_config: (val) 0xffffffff <- 0x1c (addr)
pci_write_config: (val) 0x0 -> 0x1c (addr)

Thanks,
Cam

Re: [Qemu-devel] Re: Unusual physical address when using 64-bit BAR

[Isaku Yamahata]

On Fri, Aug 27, 2010 at 01:35:23PM -0600, Cam Macdonell wrote:
> In upstream seabios.git, the c040 is not written, but the device
> returns ffffffff from 0x1c (only reads and writes to 0x18 and 0x1c are
> shown below)
> 
> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0xffffffff -> 0x18 (addr)
> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
> pci_write_config: (val) 0x4 -> 0x18 (addr)
> pci_read_config: (val) 0x4 <- 0x18 (addr)
	this read is useless. I sent out the patch.

> pci_write_config: (val) 0x0 -> 0x18 (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)

It looks like that overflow occurred. You need overflow patch.
As the size is huge, seabios can't map it.
Anyway even with overflow patch, the expected result is
to leave the bar unmodified(initial value).


> pci_read_config: (val) 0x4 <- 0x18 (addr)
> pci_write_config: (val) 0xffffffff -> 0x18 (addr)
> IVSHMEM: guest pci addr = e0000000, guest h/w addr = 2164588544, size = 20000000
> pci_read_config: (val) 0xe0000004 <- 0x18 (addr)
> pci_write_config: (val) 0x4 -> 0x18 (addr)
> pci_read_config: (val) 0x0 <- 0x1c (addr)
> pci_write_config: (val) 0xffffffff -> 0x1c (addr)
> IVSHMEM: guest pci addr = ffffffff00000000, guest h/w addr =
> 2164588544, size = 20000000
> pci_read_config: (val) 0xffffffff <- 0x1c (addr)
> pci_write_config: (val) 0x0 -> 0x1c (addr)

Is the above done by guest OS?
The guest OS doesn't seem to know 64bit bar.

-- 
yamahata