Re: R/W HG memory mappings with kvm?

[Stephen Donnelly <sfdonnelly@gmail.com>]

On Mon, Aug 31, 2009 at 8:44 PM, Avi Kivity<avi@redhat.com> wrote:
> On 08/31/2009 01:33 AM, Stephen Donnelly wrote:
>>
>>> We can't duplicate mm/ in kvm.  However, mm/memory.c says:
>>>
>>>  * The way we recognize COWed pages within VM_PFNMAP mappings is through
>>> the
>>>  * rules set up by "remap_pfn_range()": the vma will have the VM_PFNMAP
>>> bit
>>>  * set, and the vm_pgoff will point to the first PFN mapped: thus every
>>> special
>>>  * mapping will always honor the rule
>>>  *
>>>  *      pfn_of_page == vma->vm_pgoff + ((addr - vma->vm_start)>>
>>> PAGE_SHIFT)
>>>  *
>>>  * And for normal mappings this is false.
>>>
>>> So it seems the kvm calculation is right and you should set vm_pgoff in
>>> your
>>> driver.
>>>
>>
>> That may be true for COW pages, which are main memory, but I don't
>> think it is true for device drivers.
>>
>
> No, COW pages have no linear pfn mapping.  It's only true for
> remap_pfn_range).
>
>> In a device driver the mmap function receives the vma from the OS. The
>> vm_pgoff field contains the offset area in the file. For drivers this
>> is used to determine where to start the map compared to the io base
>> address.
>>
>> If the driver is mapping io memory to user space it calls
>> io_remap_pfn_range with the pfn for the io memory. The remap_pfn_range
>> call sets the VM_IO and VM_PFNMAP bits in vm_flags. It does not alter
>> the vm_pgoff value.
>>
>> A simple example is hpet_mmap() in drivers/char/hpet.c, or
>> mbcs_gscr_mmap() in drivers/char/mbcs.c.
>>
>
> io_remap_pfn_range() is remap_pfn_range(), which has this:
>
>        if (addr == vma->vm_start && end == vma->vm_end) {
>                vma->vm_pgoff = pfn;
>                vma->vm_flags |= VM_PFN_AT_MMAP;
>        }
>
> So remap_pfn_range() will alter the pgoff.

Aha! We are looking at different kernels. I should have mentioned I
was looking at 2.6.28. In mm/memory.c remap_pfn_range() this has:

	 * There's a horrible special case to handle copy-on-write
	 * behaviour that some programs depend on. We mark the "original"
	 * un-COW'ed pages by matching them up with "vma->vm_pgoff".
	 */
	if (is_cow_mapping(vma->vm_flags)) {
		if (addr != vma->vm_start || end != vma->vm_end)
			return -EINVAL;
		vma->vm_pgoff = pfn;
	}

The macro is:

static inline int is_cow_mapping(unsigned int flags)
{
	return (flags & (VM_SHARED | VM_MAYWRITE)) == VM_MAYWRITE;
}

Because my vma is marked shared, this clause does not operate and
vm_pgoff is not modified (it is still 0).

> I'm totally confused now.

Sorry about that. The issue is the BUG in gfn_to_pgn where the pfn is
not calculated correctly after looking up the vma.

I still don't see how to get the physical address from the vma, since
vm_pgoff is zero, and the vm_ops are not filled. The vma does not seem
to store the physical base address.

Regards,
Stephen.