All of lore.kernel.org
 help / color / mirror / Atom feed
From: Sherry Sun <sherry.sun@nxp.com>
To: David Laight <David.Laight@ACULAB.COM>,
	'Christoph Hellwig' <hch@infradead.org>
Cc: "sudeep.dutt@intel.com" <sudeep.dutt@intel.com>,
	"ashutosh.dixit@intel.com" <ashutosh.dixit@intel.com>,
	"arnd@arndb.de" <arnd@arndb.de>,
	"gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
	"kishon@ti.com" <kishon@ti.com>,
	"lorenzo.pieralisi@arm.com" <lorenzo.pieralisi@arm.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	dl-linux-imx <linux-imx@nxp.com>, Andy Duan <fugang.duan@nxp.com>
Subject: RE: [PATCH V2 4/4] misc: vop: mapping kernel memory to user space as noncached
Date: Tue, 13 Oct 2020 02:02:32 +0000	[thread overview]
Message-ID: <VI1PR04MB49601B933254E881097575CC92040@VI1PR04MB4960.eurprd04.prod.outlook.com> (raw)
In-Reply-To: <38d86b352ed6452ea225aa45e81af390@AcuMS.aculab.com>

Hi David, thanks for your information.
Hi Christoph, please see my comments below.

> Subject: RE: [PATCH V2 4/4] misc: vop: mapping kernel memory to user space
> as noncached
> 
> From: Christoph Hellwig
> > Sent: 29 September 2020 11:29
> ...
> > You can't call remap_pfn_range on memory returned from
> > dma_alloc_coherent (which btw is not marked uncached on many
> platforms).
> >
> > You need to use the dma_mmap_coherent helper instead.
> 

I tried to use dma_mmap_coherent helper here, but I met the same problem as David said.
Since the user space calls mmap() to map all the device page and vring size at one time.

     va = mmap(NULL, MIC_DEVICE_PAGE_END + vr_size * num_vq, PROT_READ, MAP_SHARED, fd, 0);

But the physical addresses of device page and multiple vrings are not consecutive, so we called
multiple remap_pfn_range before. When changing to use dma_mmap_coherent, it will return error
because vma_pages(the size user space want to map) are bigger than the actual size we do multiple
map(one non-continuous memory size at a time).

David believes that we could modify the vm_start address before call the multiple dma_mmap_coherent to
avoid the vma_pages check error and map multiple discontinuous memory.
Do you have any suggestions?

Best regards
Sherry

> Hmmmm. I've a driver that does that.
> Fortunately it only has to work on x86 where it doesn't matter.
> However I can't easily convert it.
> The 'problem' is that the mmap() request can cover multiple dma buffers and
> need not start at the beginning of one.
> 
> Basically we have a PCIe card that has an inbuilt iommu to convert internal
> 32bit addresses to 64bit PCIe ones.
> This has 512 16kB pages.
> So we do a number of dma_alloc_coherent() for 16k blocks.
> The user process then does an mmap() for part of the buffer.
> This request is 4k aligned so we do multiple remap_pfn_range() calls to map
> the disjoint physical (and kernel virtual) buffers into contiguous user memory.
> 
> So both ends see contiguous addresses even though the physical addresses
> are non-contiguous.
> 
> I guess I could modify the vm_start address and then restore it at the end.
> 
> I found this big discussion:
> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.k
> ernel.org%2Fpatchwork%2Fpatch%2F351245%2F&amp;data=02%7C01%7Csh
> erry.sun%40nxp.com%7C876724689688440581a708d8648dceb3%7C686ea1d
> 3bc2b4c6fa92cd99c5c301635%7C0%7C0%7C637369907516376294&amp;sdat
> a=amSClQfVGhI0%2F3bZfo8HF7UmCktkPluArWW22YlQzMQ%3D&amp;reser
> ved=0
> about these functions.
> 
> The bit about VIPT caches is problematic.
> I don't think you can change the kernel address during mmap.
> What you need to do is defer allocating the user address until you know the
> kernel address.
> Otherwise you get into problems is you try to mmap the same memory into
> two processes.
> This is a general problem even for mmap() of files.
> ISTR SYSV on some sparc systems having to use uncached maps.
> 
> If you might want to mmap two kernel buffers (dma or not) into adjacent
> user addresses then you need some way of allocating the second buffer to
> follow the first one in the VIVT cache.
> 
> 	David
> 
> -
> Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes,
> MK1 1PT, UK Registration No: 1397386 (Wales)


  reply	other threads:[~2020-10-13  2:42 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-09-29  8:44 [PATCH V2 0/4] Change vring space from nomal memory to dma coherent memory Sherry Sun
2020-09-29  8:44 ` [PATCH V2 1/4] misc: vop: change the way of allocating vring Sherry Sun
2020-09-29 10:23   ` Christoph Hellwig
2020-09-29 13:02     ` Sherry Sun
2020-09-29  8:44 ` [PATCH V2 2/4] misc: vop: do not allocate and reassign the used ring Sherry Sun
2020-09-29 10:24   ` Christoph Hellwig
2020-09-29 13:02     ` Sherry Sun
2020-09-29 14:28       ` David Laight
2020-09-29  8:44 ` [PATCH V2 3/4] misc: vop: simply return the saved dma address instead of virt_to_phys Sherry Sun
2020-09-29 10:26   ` Christoph Hellwig
2020-09-29 13:10     ` Sherry Sun
2020-09-29 18:11       ` Christoph Hellwig
2020-09-30  7:30         ` Sherry Sun
2020-10-05 13:42           ` Christoph Hellwig
2020-09-29  8:44 ` [PATCH V2 4/4] misc: vop: mapping kernel memory to user space as noncached Sherry Sun
2020-09-29 10:28   ` Christoph Hellwig
2020-09-29 13:12     ` Sherry Sun
2020-09-29 15:39     ` David Laight
2020-10-13  2:02       ` Sherry Sun [this message]
2020-10-20  1:33         ` Sherry Sun

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=VI1PR04MB49601B933254E881097575CC92040@VI1PR04MB4960.eurprd04.prod.outlook.com \
    --to=sherry.sun@nxp.com \
    --cc=David.Laight@ACULAB.COM \
    --cc=arnd@arndb.de \
    --cc=ashutosh.dixit@intel.com \
    --cc=fugang.duan@nxp.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=hch@infradead.org \
    --cc=kishon@ti.com \
    --cc=linux-imx@nxp.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=lorenzo.pieralisi@arm.com \
    --cc=sudeep.dutt@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.