gnttab_end_foreign_access/unmap fails for GNTMAP_device_map?

gnttab_end_foreign_access/unmap fails for GNTMAP_device_map?

[Oleksandr Andrushchenko]

Hi, all!

I am trying to map and then unmap grant refs in Dom0 from DomU
which are used for DMA by a real device in Dom0 (x86_64).
Mapping is done with GNTMAP_host_map | GNTMAP_device_map flags
and device can successfully access the pages and do DMA.
The problem is I cannot unmap the buffer back: Dom0 says
unmapping was done w/o errors, but when DomU tries to
gnttab_end_foreign_access I see lots of
[   55.052979] xen:grant_table: WARNING: g.e. 0xdce still in use!
[   55.052984] deferring g.e. 0xdce (pfn 0xffffffffffffffff)
messages in DomU and those grefs are never freed and system gets
exhausted at some point.

The same code, but with only GNTMAP_host_map flag is able to release
references as expected. Even if I don't give the buffer to any
HW device in Dom0 the use of GNTMAP_device_map flag makes troubles.
I also tried to add GNTMAP_readonly, but it didn't help.
In DomU right before doing gnttab_end_foreign_access I am checking
flags with gnttab_query_foreign_access: both GTF_reading
and GTF_writing are set. With GNTMAP_readonly only GTF_reading
(which I believe is expected).
I tried to look at the Xen sources and thought that it could
probably be related to pin/unpin?
Can anyone please shed some light if anything additional needs to
be done in case GNTMAP_device_map is in use or point me to
what needs to checked so I can find the root cause?

Thank you,
Oleksandr


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: gnttab_end_foreign_access/unmap fails for GNTMAP_device_map?

[Jan Beulich]

>>> On 14.12.16 at 15:46, <andr2000@gmail.com> wrote:
> Hi, all!
> 
> I am trying to map and then unmap grant refs in Dom0 from DomU
> which are used for DMA by a real device in Dom0 (x86_64).
> Mapping is done with GNTMAP_host_map | GNTMAP_device_map flags
> and device can successfully access the pages and do DMA.
> The problem is I cannot unmap the buffer back: Dom0 says
> unmapping was done w/o errors, but when DomU tries to
> gnttab_end_foreign_access I see lots of
> [   55.052979] xen:grant_table: WARNING: g.e. 0xdce still in use!
> [   55.052984] deferring g.e. 0xdce (pfn 0xffffffffffffffff)
> messages in DomU and those grefs are never freed and system gets
> exhausted at some point.
> 
> The same code, but with only GNTMAP_host_map flag is able to release
> references as expected. Even if I don't give the buffer to any
> HW device in Dom0 the use of GNTMAP_device_map flag makes troubles.
> I also tried to add GNTMAP_readonly, but it didn't help.
> In DomU right before doing gnttab_end_foreign_access I am checking
> flags with gnttab_query_foreign_access: both GTF_reading
> and GTF_writing are set. With GNTMAP_readonly only GTF_reading
> (which I believe is expected).
> I tried to look at the Xen sources and thought that it could
> probably be related to pin/unpin?

Possibly - looking at __gnttab_unmap_common(), are you passing
GNTMAP_device_map also to the unmap operation (together with
the frame number, or else the flag would be ignored)?

Jan

> Can anyone please shed some light if anything additional needs to
> be done in case GNTMAP_device_map is in use or point me to
> what needs to checked so I can find the root cause?
> 
> Thank you,
> Oleksandr



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: gnttab_end_foreign_access/unmap fails for GNTMAP_device_map?

[Oleksandr Andrushchenko]

Hi, Jan!
On 12/14/2016 04:59 PM, Jan Beulich wrote:
>>>> On 14.12.16 at 15:46, <andr2000@gmail.com> wrote:
>> Hi, all!
>>
>> I am trying to map and then unmap grant refs in Dom0 from DomU
>> which are used for DMA by a real device in Dom0 (x86_64).
>> Mapping is done with GNTMAP_host_map | GNTMAP_device_map flags
>> and device can successfully access the pages and do DMA.
>> The problem is I cannot unmap the buffer back: Dom0 says
>> unmapping was done w/o errors, but when DomU tries to
>> gnttab_end_foreign_access I see lots of
>> [   55.052979] xen:grant_table: WARNING: g.e. 0xdce still in use!
>> [   55.052984] deferring g.e. 0xdce (pfn 0xffffffffffffffff)
>> messages in DomU and those grefs are never freed and system gets
>> exhausted at some point.
>>
>> The same code, but with only GNTMAP_host_map flag is able to release
>> references as expected. Even if I don't give the buffer to any
>> HW device in Dom0 the use of GNTMAP_device_map flag makes troubles.
>> I also tried to add GNTMAP_readonly, but it didn't help.
>> In DomU right before doing gnttab_end_foreign_access I am checking
>> flags with gnttab_query_foreign_access: both GTF_reading
>> and GTF_writing are set. With GNTMAP_readonly only GTF_reading
>> (which I believe is expected).
>> I tried to look at the Xen sources and thought that it could
>> probably be related to pin/unpin?
> Possibly - looking at __gnttab_unmap_common(), are you passing
> GNTMAP_device_map also to the unmap operation (together with
> the frame number, or else the flag would be ignored)?
>
> Jan
Indeed, it helped:
gnttab_set_unmap_op(&unmap_ops[i], addr,
             GNTMAP_host_map | GNTMAP_device_map,
             xen_obj->map_handles[i]);
*unmap_ops[i].dev_bus_addr = xen_obj->dev_bus_addr[i];*
I was confused by gnttab_set_unmap_op which
unmap->dev_bus_addr = 0;
>> Can anyone please shed some light if anything additional needs to
>> be done in case GNTMAP_device_map is in use or point me to
>> what needs to checked so I can find the root cause?
>>
>> Thank you,
>> Oleksandr
>
Thank you,
Oleksandr

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel