Re: [Qemu-devel] BUG: Re: [PATCH v3 11/14] ioport: Switch dispatching to memory core layer

[Alexey Kardashevskiy <aik@ozlabs.ru>]

On 07/20/2013 01:48 AM, Alexey Kardashevskiy wrote:
> Ok. So.
> 
> What broke is...
> I could try explaining but backtraces are lot better :)
> 
> Shortly - virtio_pci_config_ops.endianness was ignored before (was bad but
> we had a workaround in spapr_io_ops), now it works so double swap happens
> and everything gets broken.
> 
> If we talk about VGA (in powerpc, it is all about powerpc), I guess
> memory_region_iorange_write() will go through mr->ops->old_portio branch
> and won't do any byte swapping (so spapr_io_ops will do the job), so we are
> fine here. I do not understand yet why it works on mac99 though, too late
> here :)


I understood. VGA does not work for mac99 either with this command line:
./qemu-system-ppc64 -m "1024" -M "mac99" -vga "std"
So it works for pseries only because of parity bug in spapr_io_ops.

So the right fix is to get rid of spapr_io_ops and every other hack like
that and to add byte swapping to every "if (mr->ops->old_portio)" branch
(should fix VGA and all other old_portio users). Current byte swapping in
memory regions seems to be right.

I would try fixing it but since all my patches were terrible shit so far, I
won't risk :)



> h_logical_store is a hypercall for system firmware to do cache inhibited
> read/write.
> 
> 
> This is with the patch applied (git checkout  b40acf9):
> 
> 
> #0  virtqueue_init (vq=0x11014ac0) at
> /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio.c:90
> #1  0x0000000010371f28 in virtio_queue_set_addr (vdev=0x11019dd0, n=0x0,
> addr=0xd0fb0000000)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio.c:662
> #2  0x00000000102027f0 in virtio_ioport_write (opaque=0x11019580, addr=0x8,
> val=0xd0fb0000)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio-pci.c:278
> #3  0x0000000010202f08 in virtio_pci_config_write (opaque=0x11019580,
> addr=0x8, val=0xd0fb0000, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio-pci.c:416
> #4  0x000000001037e220 in memory_region_write_accessor (opaque=0x11019c78,
> addr=0x8, value=0x1fffff0edc00,
>     size=0x4, shift=0x0, mask=0xffffffff) at
> /home/alexey/pcipassthru/qemu-impreza/memory.c:364
> #5  0x000000001037e36c in access_with_adjusted_size (addr=0x8,
> value=0x1fffff0edc00, size=0x4,
>     access_size_min=0x1, access_size_max=0x4, access=
>     @0x1069df40: 0x1037e164 <memory_region_write_accessor>, opaque=0x11019c78)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:396
> #6  0x0000000010380b5c in memory_region_dispatch_write (mr=0x11019c78,
> addr=0x8, data=0xd0fb0000, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:905
> #7  0x0000000010383fa4 in io_mem_write (mr=0x11019c78, addr=0x8,
> val=0xfbd0, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:1608
> #8  0x00000000102e2fdc in address_space_rw (as=0x10ef4350
> <address_space_io>, addr=0x48,
>     buf=0x1fffff0edde0 "", len=0x4, is_write=0x1) at
> /home/alexey/pcipassthru/qemu-impreza/exec.c:1918
> #9  0x00000000102e33c8 in address_space_write (as=0x10ef4350
> <address_space_io>, addr=0x48,
>     buf=0x1fffff0edde0 "", len=0x4) at
> /home/alexey/pcipassthru/qemu-impreza/exec.c:1969
> #10 0x0000000010375754 in cpu_outl (addr=0x48, val=0xfbd0)
>     at /home/alexey/pcipassthru/qemu-impreza/ioport.c:309
> #11 0x0000000010358240 in spapr_io_write (opaque=0x11016a00, addr=0x48,
> data=0xfbd0, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_pci.c:468
> #12 0x000000001037e220 in memory_region_write_accessor (opaque=0x110191f8,
> addr=0x48, value=0x1fffff0ee060,
>     size=0x4, shift=0x0, mask=0xffffffff) at
> /home/alexey/pcipassthru/qemu-impreza/memory.c:364
> #13 0x000000001037e36c in access_with_adjusted_size (addr=0x48,
> value=0x1fffff0ee060, size=0x4,
>     access_size_min=0x1, access_size_max=0x4, access=
>     @0x1069df40: 0x1037e164 <memory_region_write_accessor>, opaque=0x110191f8)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:396
> #14 0x0000000010380b5c in memory_region_dispatch_write (mr=0x110191f8,
> addr=0x48, data=0xfbd0, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:905
> #15 0x0000000010383fa4 in io_mem_write (mr=0x110191f8, addr=0x48,
> val=0xd0fb0000, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:1608
> #16 0x00000000102e47ac in stl_phys_internal (addr=0x10080000048,
> val=0xd0fb0000, endian=
>     DEVICE_NATIVE_ENDIAN) at /home/alexey/pcipassthru/qemu-impreza/exec.c:2420
> #17 0x00000000102e48a8 in stl_phys (addr=0x10080000048, val=0xd0fb0000)
>     at /home/alexey/pcipassthru/qemu-impreza/exec.c:2442
> #18 0x0000000010354f1c in h_logical_store (cpu=0x1fffff0f0010,
> spapr=0x10fe9510, opcode=0x40,
>     args=0x1ffffffd0030) at
> /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_hcall.c:570
> 
> 
> 
> This is without this patch (i.e. git checkout  b40acf9^ ):
> 
> #0  virtqueue_init (vq=0x11014ac0) at
> /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio.c:90
> #1  0x00000000103720e4 in virtio_queue_set_addr (vdev=0x11019dd0, n=0x0,
> addr=0xffe2000)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio.c:662
> #2  0x00000000102027f0 in virtio_ioport_write (opaque=0x11019580, addr=0x8,
> val=0xffe2)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio-pci.c:278
> #3  0x0000000010202f08 in virtio_pci_config_write (opaque=0x11019580,
> addr=0x8, val=0xffe2, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/virtio/virtio-pci.c:416
> #4  0x000000001037dca8 in memory_region_write_accessor (opaque=0x11019c78,
> addr=0x8, value=0x1fffff0edca8,
>     size=0x4, shift=0x0, mask=0xffffffff) at
> /home/alexey/pcipassthru/qemu-impreza/memory.c:364
> #5  0x000000001037ddf4 in access_with_adjusted_size (addr=0x8,
> value=0x1fffff0edca8, size=0x4,
>     access_size_min=0x1, access_size_max=0x4, access=
>     @0x1069def8: 0x1037dbec <memory_region_write_accessor>, opaque=0x11019c78)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:396
> #6  0x000000001037e474 in memory_region_iorange_write
> (iorange=0x1ffff0005430, offset=0x8, width=0x4,
>     data=0xffe2) at /home/alexey/pcipassthru/qemu-impreza/memory.c:475
> #7  0x00000000103750d4 in ioport_writel_thunk (opaque=0x1ffff0005430,
> addr=0x48, data=0xffe2)
>     at /home/alexey/pcipassthru/qemu-impreza/ioport.c:226
> #8  0x0000000010374728 in ioport_write (index=0x2, address=0x48, data=0xffe2)
>     at /home/alexey/pcipassthru/qemu-impreza/ioport.c:83
> #9  0x0000000010375688 in cpu_outl (addr=0x48, val=0xffe2)
>     at /home/alexey/pcipassthru/qemu-impreza/ioport.c:296
> #10 0x00000000103583fc in spapr_io_write (opaque=0x11016a00, addr=0x48,
> data=0xffe2, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_pci.c:468
> #11 0x000000001037dca8 in memory_region_write_accessor (opaque=0x110191f8,
> addr=0x48, value=0x1fffff0ee060,
>     size=0x4, shift=0x0, mask=0xffffffff) at
> /home/alexey/pcipassthru/qemu-impreza/memory.c:364
> #12 0x000000001037ddf4 in access_with_adjusted_size (addr=0x48,
> value=0x1fffff0ee060, size=0x4,
>     access_size_min=0x1, access_size_max=0x4, access=
>     @0x1069def8: 0x1037dbec <memory_region_write_accessor>, opaque=0x110191f8)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:396
> #13 0x0000000010380c90 in memory_region_dispatch_write (mr=0x110191f8,
> addr=0x48, data=0xffe2, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:993
> #14 0x00000000103840d8 in io_mem_write (mr=0x110191f8, addr=0x48,
> val=0xe2ff0000, size=0x4)
>     at /home/alexey/pcipassthru/qemu-impreza/memory.c:1696
> #15 0x00000000102e4968 in stl_phys_internal (addr=0x10080000048,
> val=0xe2ff0000, endian=
>     DEVICE_NATIVE_ENDIAN) at /home/alexey/pcipassthru/qemu-impreza/exec.c:2447
> #16 0x00000000102e4a64 in stl_phys (addr=0x10080000048, val=0xe2ff0000)
>     at /home/alexey/pcipassthru/qemu-impreza/exec.c:2469
> #17 0x00000000103550d8 in h_logical_store (cpu=0x1fffff0f0010,
> spapr=0x10fe9510, opcode=0x40,
>     args=0x1ffffffd0030) at
> /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_hcall.c:570
> #18 0x0000000010355698 in spapr_hypercall (cpu=0x1fffff0f0010, opcode=0x40,
> args=0x1ffffffd0030)
>     at /home/alexey/pcipassthru/qemu-impreza/hw/ppc/spapr_hcall.c:689
> 
> 
> 
> 
> 
> 
> On 07/19/2013 10:49 PM, Paolo Bonzini wrote:
>> Il 19/07/2013 13:09, Alexey Kardashevskiy ha scritto:
>>> Hi!
>>>
>>> This patch also breaks virtio on powerpc. I thought it was fixed
>>> (reverted?) in the master branch from qemu.org but it is still there. What
>>> did I miss?
>>
>> It was not reverted, only the "DEVICE_LITTLE_ENDIAN" marking was.
>>
>> Let me check if I can reproduce this, it looks like a endianness
>> problems reading virtio-blk config space.
>>
>> Paolo
>>
>>> Trying to load:  from: disk ... virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> virtioblk_read: Access beyond end of device!
>>> [many of those]
>>>
>>>
>>>
>>> On 07/11/2013 10:29 PM, Alexander Graf wrote:
>>>>
>>>> On 24.06.2013, at 08:07, Jan Kiszka wrote:
>>>>
>>>>> On 2013-06-23 22:50, Hervé Poussineau wrote:
>>>>>> Jan Kiszka a écrit :
>>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>
>>>>>>> The current ioport dispatcher is a complex beast, mostly due to the
>>>>>>> need to deal with old portio interface users. But we can overcome it
>>>>>>> without converting all portio users by embedding the required base
>>>>>>> address of a MemoryRegionPortio access into that data structure. That
>>>>>>> removes the need to have the additional MemoryRegionIORange structure
>>>>>>> in the loop on every access.
>>>>>>>
>>>>>>> To handle old portio memory ops, we simply install dispatching handlers
>>>>>>> for portio memory regions when registering them with the memory core.
>>>>>>> This removes the need for the old_portio field.
>>>>>>>
>>>>>>> We can drop the additional aliasing of ioport regions and also the
>>>>>>> special address space listener. cpu_in and cpu_out now simply call
>>>>>>> address_space_read/write. And we can concentrate portio handling in a
>>>>>>> single source file.
>>>>>>>
>>>>>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>> ---
>>>>>>
>>>>>> ...
>>>>>>
>>>>>>> +
>>>>>>> +static void portio_write(void *opaque, hwaddr addr, uint64_t data,
>>>>>>> +                         unsigned size)
>>>>>>> +{
>>>>>>> +    MemoryRegionPortioList *mrpio = opaque;
>>>>>>> +    const MemoryRegionPortio *mrp = find_portio(mrpio, addr, size,
>>>>>>> true);
>>>>>>> +
>>>>>>> +    if (mrp) {
>>>>>>> +        mrp->write(mrpio->portio_opaque, mrp->base + addr, data);
>>>>>>> +    } else if (size == 2) {
>>>>>>> +        mrp = find_portio(mrpio, addr, 1, true);
>>>>>>> +        assert(mrp);
>>>>>>> +        mrp->write(mrpio->portio_opaque, mrp->base + addr, data & 0xff);
>>>>>>> +        mrp->write(mrpio->portio_opaque, mrp->base + addr + 1, data
>>>>>>>>> 8);
>>>>>>> +    }
>>>>>>> +}
>>>>>>> +
>>>>>>> +static const MemoryRegionOps portio_ops = {
>>>>>>> +    .read = portio_read,
>>>>>>> +    .write = portio_write,
>>>>>>> +    .valid.unaligned = true,
>>>>>>> +    .impl.unaligned = true,
>>>>>>> +};
>>>>>>> +
>>>>>>
>>>>>> You need to mark these operations as DEVICE_LITTLE_ENDIAN.
>>>>>> In portio_write above, you clearly assume that data is in LE format.
>>>>>
>>>>> Anything behind PIO is little endian, of course. Will add this.
>>>>
>>>> This patch breaks VGA on PPC as it is in master today.
>>>>
>>>>
>>>> Alex
>>>>
>>>>>
>>>>>>
>>>>>> This fixes PPC PReP emulation, which would otherwise be broken with this
>>>>>> patchset.
>>>>>
>>>>> Thanks,
>>>>> Jan
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
> 
> 


-- 
Alexey