Re: R/W HG memory mappings with kvm?

[Stephen Donnelly <sfdonnelly@gmail.com>]

On Thu, Aug 27, 2009 at 4:08 PM, Avi Kivity<avi@redhat.com> wrote:
> On 08/27/2009 05:34 AM, Stephen Donnelly wrote:
>>
>> On Mon, Aug 24, 2009 at 4:55 PM, Avi Kivity<avi@redhat.com>  wrote:
>>
>>>
>>> On 08/24/2009 12:59 AM, Stephen Donnelly wrote:
>>>
>>>>
>>>> On Thu, Aug 20, 2009 at 12:14 AM, Avi Kivity<avi@redhat.com>    wrote:
>>>>
>>>>>
>>>>> On 08/13/2009 07:07 AM, Stephen Donnelly wrote:
>>>>>
>>>>>>
>>>>>> npages = get_user_pages_fast(addr, 1, 1, page); returns -EFAULT,
>>>>>> presumably because (vma->vm_flags&      (VM_IO | VM_PFNMAP)).
>>>>>>
>>>>>> It takes then unlikely branch, and checks the vma, but I don't
>>>>>> understand what it is doing here: pfn = ((addr - vma->vm_start)>>
>>>>>> PAGE_SHIFT) + vma->vm_pgoff;
>>>>>>
>>>>>
>>>>> It's calculating the pfn according to pfnmap rules.
>>>>>
>>>>
>>>>  From what I understand this will only work when remapping 'main
>>>> memory', e.g. where the pgoff is equal to the physical page offset?
>>>> VMAs that remap IO memory will usually set pgoff to 0 for the start of
>>>> the mapping.
>>>>
>>>
>>> If so, how do they calculate the pfn when mapping pages?  kvm needs to be
>>> able to do the same thing.
>>>
>>
>> If the vma->vm_file is /dev/mem, then the pg_off will map to physical
>> addresses directly (at least on x86), and the calculation works. If
>> the vma is remapping io memory from a driver, then vma->vm_file will
>> point to the device node for that driver. Perhaps we can do a check
>> for this at least?
>>
>
> We can't duplicate mm/ in kvm.  However, mm/memory.c says:
>
>
>  * The way we recognize COWed pages within VM_PFNMAP mappings is through the
>  * rules set up by "remap_pfn_range()": the vma will have the VM_PFNMAP bit
>  * set, and the vm_pgoff will point to the first PFN mapped: thus every
> special
>  * mapping will always honor the rule
>  *
>  *      pfn_of_page == vma->vm_pgoff + ((addr - vma->vm_start) >>
> PAGE_SHIFT)
>  *
>  * And for normal mappings this is false.
>
> So it seems the kvm calculation is right and you should set vm_pgoff in your
> driver.

That may be true for COW pages, which are main memory, but I don't
think it is true for device drivers.

In a device driver the mmap function receives the vma from the OS. The
vm_pgoff field contains the offset area in the file. For drivers this
is used to determine where to start the map compared to the io base
address.

If the driver is mapping io memory to user space it calls
io_remap_pfn_range with the pfn for the io memory. The remap_pfn_range
call sets the VM_IO and VM_PFNMAP bits in vm_flags. It does not alter
the vm_pgoff value.

A simple example is hpet_mmap() in drivers/char/hpet.c, or
mbcs_gscr_mmap() in drivers/char/mbcs.c.

>>>> I'm still not sure how genuine IO memory (mapped from a driver to
>>>> userspace with remap_pfn_range or io_remap_page_range) could be mapped
>>>> into kvm though.
>>>>
>>>
>>> If it can be mapped to userspace, it can be mapped to kvm.  We just need
>>> to
>>> synchronize the rules.
>>>
>>
>> We can definitely map it into userspace. The problem seems to be how
>> the kvm kernel module translates the guest pfn back to a host physical
>> address.
>>
>> Is there a kernel equivalent of mmap?
>
> do_mmap(), but don't use it.  Use mmap() from userspace like everyone else.

Of course you are right, gfn_to_pfn is in user space. There is already
a mapping of the memory to the process (from qemu_ram_mmap), the
question is how to look it up.

Regards,
Stephen.