userspace address to bus address

userspace address to bus address

[Florian Bauer]

Hi,
I'm currently writing my first kernel module, but i got stuck with address
translations. I want my pcie device to access a buffer which was allocated
in userspace (page aligned, currently smaller than one page to ignore
scattering). I tried to translate the userspace address to a bus address
in my ioctrl handler like this:

void __user *data = arg;
down_read(&current->mm->mmap_sem);
get_user_pages(current, current->mm, (unsigned long)data, len/*1*/, 1, 0,
&pages, NULL);
up_read(&current->mm->mmap_sem);
void* kvaddr = kmap(pages[0]);
unsigned long baddr = virt_to_bus(kaddr);
kunmap(pages[0]);
page_cache_release(pages[0]);
//...

data is page-aligned and was locked in userspace with mlock.
There is some address in baddr, but the device does not write to the
correct location and sometimes causes kernel freezes. Can someone tell me,
what is wrong with this code?TheI device works with a address, that was
received in different way (closed source driver).
I use 32 bit Ubuntu with an 2.6.32 kernel.

userspace address to bus address

[Peter Teoh]

while reading this:

http://linux.die.net/man/2/mlock

notice that u have to allocate + write to the memory first, before
mlocking it - did u do that?

(this is to prevent copy-on-write page fault, which means the memory
is allocated.....but is not really resident until written to it....so
allocated memory does not mean the physical memory exist.)

also, check this function out:   drivers/dma/iovlock.c (this function
prepare the memory for IO scatter-gather operation, ie, physical
address must be used, and no segfault is allowed):

struct dma_pinned_list *dma_pin_iovec_pages(struct iovec *iov, size_t len)
{

                if (!access_ok(VERIFY_WRITE, iov[i].iov_base, iov[i].iov_len))
                        goto unpin;

                page_list->nr_pages = num_pages_spanned(&iov[i]);
                page_list->base_address = iov[i].iov_base;

                page_list->pages = pages;
                pages += page_list->nr_pages;

                /* pin pages down */
                down_read(&current->mm->mmap_sem);
                ret = get_user_pages(
                        current,
                        current->mm,
                        (unsigned long) iov[i].iov_base,
                        page_list->nr_pages,
                        1,      /* write */
                        0,      /* force */
                        page_list->pages,
                        NULL);
                up_read(&current->mm->mmap_sem);
=========================
which thus showed the same pattern here, write to it to ensure the TLB
(translation lookaside) exists for the physical address, so there is
no pagefault occurring why hardware read/write the memory bus, which
always goes by physical address (vs everything internal to CPU is
logical/virtual address).

On Tue, Sep 13, 2011 at 1:45 AM, Florian Bauer
<florian.bauer@informatik.stud.uni-erlangen.de> wrote:
> Hi,
> I'm currently writing my first kernel module, but i got stuck with address
> translations. I want my pcie device to access a buffer which was allocated
> in userspace (page aligned, currently smaller than one page to ignore
> scattering). I tried to translate the userspace address to a bus address
> in my ioctrl handler like this:
>
> void __user *data = arg;
> down_read(&current->mm->mmap_sem);
> get_user_pages(current, current->mm, (unsigned long)data, len/*1*/, 1, 0,
> &pages, NULL);
> up_read(&current->mm->mmap_sem);
> void* kvaddr = kmap(pages[0]);
> unsigned long baddr = virt_to_bus(kaddr);
> kunmap(pages[0]);
> page_cache_release(pages[0]);
> //...
>
> data is page-aligned and was locked in userspace with mlock.
> There is some address in baddr, but the device does not write to the
> correct location and sometimes causes kernel freezes. Can someone tell me,
> what is wrong with this code?TheI device works with a address, that was
> received in different way (closed source driver).
> I use 32 bit Ubuntu with an 2.6.32 kernel.
>
>
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>



-- 
Regards,
Peter Teoh

userspace address to bus address

[Peter Teoh]

comments about "consistent memory" from here:

http://kerneltrap.org/node/7194

might be useful for you too.....

On Mon, Oct 10, 2011 at 1:13 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
> while reading this:
>
> http://linux.die.net/man/2/mlock
>
> notice that u have to allocate + write to the memory first, before
> mlocking it - did u do that?
>
> (this is to prevent copy-on-write page fault, which means the memory
> is allocated.....but is not really resident until written to it....so
> allocated memory does not mean the physical memory exist.)
>
> also, check this function out: ? drivers/dma/iovlock.c (this function
> prepare the memory for IO scatter-gather operation, ie, physical
> address must be used, and no segfault is allowed):
>
> struct dma_pinned_list *dma_pin_iovec_pages(struct iovec *iov, size_t len)
> {
>
> ? ? ? ? ? ? ? ?if (!access_ok(VERIFY_WRITE, iov[i].iov_base, iov[i].iov_len))
> ? ? ? ? ? ? ? ? ? ? ? ?goto unpin;
>
> ? ? ? ? ? ? ? ?page_list->nr_pages = num_pages_spanned(&iov[i]);
> ? ? ? ? ? ? ? ?page_list->base_address = iov[i].iov_base;
>
> ? ? ? ? ? ? ? ?page_list->pages = pages;
> ? ? ? ? ? ? ? ?pages += page_list->nr_pages;
>
> ? ? ? ? ? ? ? ?/* pin pages down */
> ? ? ? ? ? ? ? ?down_read(&current->mm->mmap_sem);
> ? ? ? ? ? ? ? ?ret = get_user_pages(
> ? ? ? ? ? ? ? ? ? ? ? ?current,
> ? ? ? ? ? ? ? ? ? ? ? ?current->mm,
> ? ? ? ? ? ? ? ? ? ? ? ?(unsigned long) iov[i].iov_base,
> ? ? ? ? ? ? ? ? ? ? ? ?page_list->nr_pages,
> ? ? ? ? ? ? ? ? ? ? ? ?1, ? ? ?/* write */
> ? ? ? ? ? ? ? ? ? ? ? ?0, ? ? ?/* force */
> ? ? ? ? ? ? ? ? ? ? ? ?page_list->pages,
> ? ? ? ? ? ? ? ? ? ? ? ?NULL);
> ? ? ? ? ? ? ? ?up_read(&current->mm->mmap_sem);
> =========================
> which thus showed the same pattern here, write to it to ensure the TLB
> (translation lookaside) exists for the physical address, so there is
> no pagefault occurring why hardware read/write the memory bus, which
> always goes by physical address (vs everything internal to CPU is
> logical/virtual address).
>
> On Tue, Sep 13, 2011 at 1:45 AM, Florian Bauer
> <florian.bauer@informatik.stud.uni-erlangen.de> wrote:
>> Hi,
>> I'm currently writing my first kernel module, but i got stuck with address
>> translations. I want my pcie device to access a buffer which was allocated
>> in userspace (page aligned, currently smaller than one page to ignore
>> scattering). I tried to translate the userspace address to a bus address
>> in my ioctrl handler like this:
>>
>> void __user *data = arg;
>> down_read(&current->mm->mmap_sem);
>> get_user_pages(current, current->mm, (unsigned long)data, len/*1*/, 1, 0,
>> &pages, NULL);
>> up_read(&current->mm->mmap_sem);
>> void* kvaddr = kmap(pages[0]);
>> unsigned long baddr = virt_to_bus(kaddr);
>> kunmap(pages[0]);
>> page_cache_release(pages[0]);
>> //...
>>
>> data is page-aligned and was locked in userspace with mlock.
>> There is some address in baddr, but the device does not write to the
>> correct location and sometimes causes kernel freezes. Can someone tell me,
>> what is wrong with this code?TheI device works with a address, that was
>> received in different way (closed source driver).
>> I use 32 bit Ubuntu with an 2.6.32 kernel.
>>
>>
>> _______________________________________________
>> Kernelnewbies mailing list
>> Kernelnewbies at kernelnewbies.org
>> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
>>
>
>
>
> --
> Regards,
> Peter Teoh
>



-- 
Regards,
Peter Teoh