Re: [Qemu-devel] [RFC] Memory API

[Anthony Liguori <anthony@codemonkey.ws>]

On 05/18/2011 11:41 AM, Avi Kivity wrote:
> On 05/18/2011 07:33 PM, Anthony Liguori wrote:
>> On 05/18/2011 10:23 AM, Avi Kivity wrote:
>>>> The tricky part is wiring this up efficiently for TCG, ie. in QEMU's
>>>> softmmu. I played with passing the issuing CPUState (or NULL for
>>>> devices) down the MMIO handler chain. Not totally beautiful as
>>>> decentralized dispatching was still required, but at least only
>>>> moderately invasive. Maybe your API allows for cleaning up the
>>>> management and dispatching part, need to rethink...
>>>
>>> My suggestion is opposite - have a different MemoryRegion for each (e.g.
>>> CPUState::memory). Then the TLBs will resolve to a different ram_addr_t
>>> for the same physical address, for the local APIC range.
>>
>> I don't understand the different ram_addr_t part.
>>
>
> The TLBs map a virtual address to a ram_addr_t.

It actually maps virtual address to host virtual addresses.  Virtual 
addresses that map to I/O memory never get stored in the TLB.

You don't need separate I/O registration addresses in order to do 
per-CPU dispatch provided that you route the dispatch routines through 
the CPUs first.

>> Overlapping regions can be handled differently at each level. For
>> instance, if a PCI device registers an IO region to the same location
>> as the APIC, the APIC always wins because the PCI bus will never see
>> the access.
>>
>
> That's inefficient, since you always have to traverse the hierarchy.

Is efficiency really a problem here?  Besides, I don't think that's 
really correct.  You're adding at most 2-3 extra function pointer 
invocations.  I don't think you can really call that inefficient.

>> You cannot do this properly with a single dispatch table because the
>> behavior depends on where in the hierarchy the I/O is being handled.
>
> You can. When you have a TLB miss, you walk the memory hierarchy (which
> is per-cpu) and end up with a ram_addr_t which is stowed in the TLB
> entry.

I think we're overloading the term TLB.  Are you referring to 
l1_phys_map as the TLB because I thought Jan was referring to the actual 
emulated TLB that TCG uses?

> Further accesses dispatch via this ram_addr_t, without taking the
> cpu into consideration (the TLB is, after all, already per-cpu).
>
> Since each APIC will have its own ram_addr_t, we don't need per-cpu
> dispatch.

You need to have per-CPU l1_phys_maps which would result in quite a lot 
of additional memory overhead.

Regards,

Anthony Liguori