get_user_pages and pinning

get_user_pages and pinning

[Sadanand Warrier]

Folks
   Since Linux uses lazy evaluation with respect to allocating memory
what happens if a user process allocates memory using malloc in excess
of say 8 or nine pages and calls a device ioctl and passes the user
space pointer to it to be used to pin pages using get_user_pages().
This buffer is to be used for a dma read from a device.
What happens if the memory isn't actually allocated (no initialization
with memset or calloc etc. and all PTEs are pointing to some zero
filled page) and there is no IOMMU?
Thanks

S

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: get_user_pages and pinning

[Cengiz Can]

Hello Sadanand

On January 2, 2021 3:39:08 PM GMT+03:00, Sadanand Warrier <sadanandwarrier@gmail.com> wrote:
>Folks
>   Since Linux uses lazy evaluation with respect to allocating memory
>what happens if a user process allocates memory using malloc in excess
>of say 8 or nine pages and calls a device ioctl and passes the user
>space pointer to it to be used to pin pages using get_user_pages().

I don't have an answer to your question because your sentences are too long and I couldn't understand the thing you are asking. 

>This buffer is to be used for a dma read from a device.
>What happens if the memory isn't actually allocated (no initialization
>with memset or calloc etc. and all PTEs are pointing to some zero
>filled page) and there is no IOMMU?
>Thanks
>
>S

 Please use punctuation marks as termination points and keep your sentences shorter for better responses from any community. 

Thank you 

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: get_user_pages and pinning

[Sadanand Warrier]

Cengiz

> I don't have an answer to your question because your sentences are too long and I couldn't understand the thing you are asking.
 A clearer picture below.
1) There is a user program that allocates memory using malloc. It does
not initialize the memory.
2) It uses an ioctl to pass a pointer to different areas of this
allocated memory to a driver controlling a device.
3) The device is supposed to fill the buffer that this pointer points
to with requested data.
4) The driver pins the pages that the pointer points to in memory
using get_user_pages.
5) The driver gets the physical (bus) address of this buffer using
kernel apis such as dma_map_sg and queues up the DMA.
6) Inspecting the bus address returned by kernel apis shows that the
same address is returned for different locations in the buffer.
7) So the DMA fails (rather the device sends the data but it is over
writing the same location)
8) It looks like the memory isn't actually allocated by malloc. Not a surprise.
9) It will ideally allocate memory when a write to the memory location occurs.
10) But the write is happening not from user space but by DMA from a device.

So the question is what should happen if the memory isn't actually
allocated. Should get_user_pages check?
It doesn't seem to be doing so.  Can it?

S

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: get_user_pages and pinning

[Rik van Riel]

[-- Attachment #1.1: Type: text/plain, Size: 366 bytes --]

On Sun, 2021-01-03 at 12:54 -0500, Sadanand Warrier wrote:
> 
> So the question is what should happen if the memory isn't actually
> allocated. Should get_user_pages check?
> It doesn't seem to be doing so.  Can it?

If the driver needs the memory to be allocated, why not
use the FOLL_POPULATE flag when calling get_user_pages?

-- 
All Rights Reversed.

[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: get_user_pages and pinning

[Sadanand Warrier]

On Sun, 3 Jan 2021 at 13:26, Rik van Riel <riel@surriel.com> wrote:
>
>
> If the driver needs the memory to be allocated, why not
> use the FOLL_POPULATE flag when calling get_user_pages?

Thanks that ought to fix it.

S
>
> --
> All Rights Reversed.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: get_user_pages and pinning

[Greg KH]

On Sat, Jan 02, 2021 at 07:39:08AM -0500, Sadanand Warrier wrote:
> Folks
>    Since Linux uses lazy evaluation with respect to allocating memory
> what happens if a user process allocates memory using malloc in excess
> of say 8 or nine pages and calls a device ioctl and passes the user
> space pointer to it to be used to pin pages using get_user_pages().
> This buffer is to be used for a dma read from a device.
> What happens if the memory isn't actually allocated (no initialization
> with memset or calloc etc. and all PTEs are pointing to some zero
> filled page) and there is no IOMMU?

You should get an error of -EFAULT when you try to do this when the
kernel tries to access memory you don't have.

But the best way to be sure is to actually try it yourself and see!
There's nothing preventing you from doing that, right?  :)

good luck!

greg k-h

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: get_user_pages and pinning

[Sadanand Warrier]

Thanks Greg

> You should get an error of -EFAULT when you try to do this when the
> kernel tries to access memory you don't have.
>
> But the best way to be sure is to actually try it yourself and see!
> There's nothing preventing you from doing that, right?  :)

Ah yes that is of course true.:-)  I did try it on Centos 4.18 and
sadly I do not get an -EFAULT.
Even access_ok blithely passes it off to the depths in the driver. But
I suppose unless the kernel actually tries to access the memory in the
midst of get_user_pages I would not see an error.
Since I do not see such an error I am supposing it doesn't (actually
touch the pages I mean), I haven't yet delved into the bowels of the
code yet. I thought I would ask here.
Now after get_user_pages and  after suitable calls to sg_dma_map etc
to get the bus address associated with the buffer, this area is passed
to a device.

If the buffer has been allocated and initialized in user space all is
good. This has been tried several times, by using memset, calloc,
actually initializing every element of the allocated buffer etc.

If not then I notice for various offsets in the buffer at page
boundaries the physical(bus) address points to the same page.
The device does the DMA and we end up with segmentation faults in
unrelated areas . systemd seems to be a frequent victim, I guess our
device has a bone to pick.

I know 5,0 onwards has the pin_user_pages family which may have some
solutions but for the moment I am stuck with 4.18.

Perhaps some other adventurous soul has tried this?  Again an IOMMU is
not present in bit of hardware though that may be neither here nor
there.

S

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: get_user_pages and pinning

[Sadanand Warrier]

From: daniela daniela <daniela.daniela.daniela.daniela@gmail.com>
To: Kernelnewbies@kernelnewbies.org
Subject: Re: get_user_pages and pinning

>Any details about the machine?

Intel i7 core x58-ICH10 chassis.

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies