Re: [PATCH RFC v3 13/14] dmaengine: idxd: add char driver to expose submission portal to userland

From: Vinod Koul <vkoul@kernel.org>
To: Dave Jiang <dave.jiang@intel.com>
Cc: "dmaengine@vger.kernel.org" <dmaengine@vger.kernel.org>,
	"Williams, Dan J" <dan.j.williams@intel.com>,
	"Luck, Tony" <tony.luck@intel.com>,
	"Lin, Jing" <jing.lin@intel.com>,
	"Raj, Ashok" <ashok.raj@intel.com>,
	"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>
Subject: Re: [PATCH RFC v3 13/14] dmaengine: idxd: add char driver to expose submission portal to userland
Date: Fri, 10 Jan 2020 13:29:41 +0530	[thread overview]
Message-ID: <20200110075941.GF2818@vkoul-mobl> (raw)
In-Reply-To: <c7ba628c-4ffe-fec9-6832-8139bd7865db@intel.com>

On 07-01-20, 13:18, Dave Jiang wrote:
> Dropped everyone except dmaengine and related parties.
> 
> On 1/7/20 11:17 AM, Dave Jiang wrote:
> > 
> > 
> > On 1/7/20 10:45 AM, Dave Jiang wrote:
> > > 
> > > 
> > > On 12/26/19 10:58 PM, Vinod Koul wrote:
> > > > On 17-12-19, 16:34, Dave Jiang wrote:
> > > > > Create a char device region that will allow acquisition of
> > > > > user portals in
> > > > > order to allow applications to submit DMA operations. A char
> > > > > device will be
> > > > > created per work queue that gets exposed. The workqueue type "user"
> > > > > is used to mark a work queue for user char device. For example if the
> > > > > workqueue 0 of DSA device 0 is marked for char device, then
> > > > > a device node
> > > > > of /dev/dsa/wq0.0 will be created.
> > > > 
> > > > do we really want to create a device specific interface..? why not move
> > > > it to dmaengine core and create a dmaengine device for userland to
> > > > submit dma operations?
> > > 
> > > I'm keeping an eye on the uacce framework [1] progress. If that goes
> > > upstream then there's probably not any reason for dmaengine to export
> > > that. Otherwise then yes that would be reasonable. Are you thinking that
> > > the uacce guys should consider doing that for dmaengine instead?
> > > 
> > > The char device export in idxd driver is a bridge solution until we
> > > bottom out on whichever interface to provide the generic framework.
> > 
> > Oops forgot to provide URL
> > 
> > [1]: https://lkml.org/lkml/2019/12/15/332
> 
> So after thinking a bit more and talking to Dan, I think there's some
> confusion that needs explanation on how the idxd user portal works vs
> traditional DMA engines. For idxd, the char device allows the user app to
> mmap() a 4k region called a portal. With that portal address the user can
> use either MOVDIR64B or ENQCMD CPU instruction to directly submit a DMA
> descriptor to the device without touching the kernel. I think this part is
> device specific and probably can only be provided by the driver for now.
> 
> I think you on the other hand are looking for a kernel assisted DMA
> operation submission from userspace through the dmaengine framework correct?
> We would expose a char device for a channel that allows a user app to submit
> DMA operations via ioctls? That is something the idxd driver needs as well
> when the operation hits a point where the user portal is saturated and we
> need kernel assistance to move forward. But I would like to mark this as
> follow on work for me to add to dmaengine as a common framework.

Yeah agreed that is something we would need eventually...

-- 
~Vinod