LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Kenneth Lee <liguozhu@hisilicon.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Kenneth Lee <nek.in.cn@gmail.com>,
	Jonathan Corbet <corbet@lwn.net>,
	Herbert Xu <herbert@gondor.apana.org.au>,
	"David S . Miller" <davem@davemloft.net>,
	Joerg Roedel <joro@8bytes.org>,
	Alex Williamson <alex.williamson@redhat.com>,
	Hao Fang <fanghao11@huawei.com>,
	Zhou Wang <wangzhou1@hisilicon.com>,
	Zaibo Xu <xuzaibo@huawei.com>,
	Philippe Ombredanne <pombredanne@nexb.com>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"linux-accelerators@lists.ozlabs.org" 
	<linux-accelerators@lists.ozlabs.org>,
	Lu Baolu <baolu.lu@linux.intel.com>,
	"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
	"linuxarm@huawei.com" <linuxarm@huawei.com>
Subject: Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive
Date: Thu, 2 Aug 2018 11:40:06 +0800
Message-ID: <20180802034006.GJ160746@Turing-Arch-b> (raw)
In-Reply-To: <AADFC41AFE54684AB9EE6CBC0274A5D191290EB3@SHSMSX101.ccr.corp.intel.com>

On Thu, Aug 02, 2018 at 02:59:33AM +0000, Tian, Kevin wrote:
> Date: Thu, 2 Aug 2018 02:59:33 +0000
> From: "Tian, Kevin" <kevin.tian@intel.com>
> To: Kenneth Lee <nek.in.cn@gmail.com>, Jonathan Corbet <corbet@lwn.net>,
>  Herbert Xu <herbert@gondor.apana.org.au>, "David S . Miller"
>  <davem@davemloft.net>, Joerg Roedel <joro@8bytes.org>, Alex Williamson
>  <alex.williamson@redhat.com>, Kenneth Lee <liguozhu@hisilicon.com>, Hao
>  Fang <fanghao11@huawei.com>, Zhou Wang <wangzhou1@hisilicon.com>, Zaibo Xu
>  <xuzaibo@huawei.com>, Philippe Ombredanne <pombredanne@nexb.com>, Greg
>  Kroah-Hartman <gregkh@linuxfoundation.org>, Thomas Gleixner
>  <tglx@linutronix.de>, "linux-doc@vger.kernel.org"
>  <linux-doc@vger.kernel.org>, "linux-kernel@vger.kernel.org"
>  <linux-kernel@vger.kernel.org>, "linux-crypto@vger.kernel.org"
>  <linux-crypto@vger.kernel.org>, "iommu@lists.linux-foundation.org"
>  <iommu@lists.linux-foundation.org>, "kvm@vger.kernel.org"
>  <kvm@vger.kernel.org>, "linux-accelerators@lists.ozlabs.org"
>  <linux-accelerators@lists.ozlabs.org>, Lu Baolu
>  <baolu.lu@linux.intel.com>, "Kumar, Sanjay K" <sanjay.k.kumar@intel.com>
> CC: "linuxarm@huawei.com" <linuxarm@huawei.com>
> Subject: RE: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive
> Message-ID: <AADFC41AFE54684AB9EE6CBC0274A5D191290EB3@SHSMSX101.ccr.corp.intel.com>
> 
> > From: Kenneth Lee
> > Sent: Wednesday, August 1, 2018 6:22 PM
> > 
> > From: Kenneth Lee <liguozhu@hisilicon.com>
> > 
> > WarpDrive is an accelerator framework to expose the hardware capabilities
> > directly to the user space. It makes use of the exist vfio and vfio-mdev
> > facilities. So the user application can send request and DMA to the
> > hardware without interaction with the kernel. This remove the latency
> > of syscall and context switch.
> > 
> > The patchset contains documents for the detail. Please refer to it for more
> > information.
> > 
> > This patchset is intended to be used with Jean Philippe Brucker's SVA
> > patch [1] (Which is also in RFC stage). But it is not mandatory. This
> > patchset is tested in the latest mainline kernel without the SVA patches.
> > So it support only one process for each accelerator.
> 
> If no sharing, then why not just assigning the whole parent device to
> the process? IMO if SVA usage is the clear goal of your series, it
> might be made clearly so then Jean's series is mandatory dependency...
> 

We don't know how SVA will be finally. But the feature, "make use of
per-PASID/substream ID IOMMU page table", should be able to be enabled in the
kernel. So we don't want to enforce it here. After we have this serial ready, it
can be hooked to any implementation.

Further more, even without "per-PASID IOMMU page table", this series has its
value. It is not simply dedicate the whole device to the process. It "shares"
the device with the kernel driver. So you can support crypto and a user
application at the same time.

> > 
> > With SVA support, WarpDrive can support multi-process in the same
> > accelerator device.  We tested it in our SoC integrated Accelerator (board
> > ID: D06, Chip ID: HIP08). A reference work tree can be found here: [2].
> > 
> > We have noticed the IOMMU aware mdev RFC announced recently [3].
> > 
> > The IOMMU aware mdev has similar idea but different intention comparing
> > to
> > WarpDrive. It intends to dedicate part of the hardware resource to a VM.
> 
> Not just to VM, though I/O Virtualization is in the name. You can assign
> such mdev to either VMs, containers, or bare metal processes. It's just
> a fully-isolated device from user space p.o.v.

Oh, yes. Thank you for clarification.

> 
> > And the design is supposed to be used with Scalable I/O Virtualization.
> > While spimdev is intended to share the hardware resource with a big
> > amount
> > of processes.  It just requires the hardware supporting address
> > translation per process (PCIE's PASID or ARM SMMU's substream ID).
> > 
> > But we don't see serious confliction on both design. We believe they can be
> > normalized as one.
> 
> yes there are something which can be shared, e.g. regarding to
> the interface to IOMMU.
> 
> Conceptually I see them different mindset on device resource sharing:
> 
> WarpDrive more aims to provide a generic framework to enable SVA
> usages on various accelerators, which lack of a well-abstracted user
> API like OpenCL. SVA is a hardware capability - sort of exposing resources
> composing ONE capability to user space through mdev framework. It is
> not like a VF which naturally carries most capabilities as PF.
> 

Yes. But we believe the user abstraction layer will be enabled soon when the
channel is opened. WarpDrive gives the hardware the chance to serve the
application directly. For example, an AI engine can be called by many processes
for inference. The resource need not to be dedicated to one particular process.

> Intel Scalable I/O virtualization is a thorough design to partition the
> device into minimal sharable copies (queue, queue pair, context), 
> while each copy carries most PF capabilities (including SVA) similar to
> VF. Also with IOMMU scalable mode support, the copy can be 
> independently assigned to any client (process, container, VM, etc.)
> 
Yes, we can see this intension.
> Thanks
> Kevin

Thank you.

-- 
			-Kenneth(Hisilicon)

  reply index

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-01 10:22 Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 1/7] vfio/spimdev: Add documents for WarpDrive framework Kenneth Lee
2018-08-02  3:14   ` Tian, Kevin
2018-08-02  4:22     ` Kenneth Lee
2018-08-02  4:41       ` Tian, Kevin
2018-08-06 12:27   ` Pavel Machek
2018-08-08  1:43     ` Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 2/7] iommu: Add share domain interface in iommu for spimdev Kenneth Lee
2018-08-02  3:17   ` Tian, Kevin
2018-08-02  4:15     ` Kenneth Lee
2018-08-02  4:39       ` Tian, Kevin
2018-08-08  9:13   ` Joerg Roedel
2018-08-09  1:09     ` Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 3/7] vfio: add spimdev support Kenneth Lee
2018-08-01 16:23   ` Randy Dunlap
2018-08-02  3:07     ` Kenneth Lee
2018-08-02  3:21   ` Tian, Kevin
2018-08-02  3:47     ` Kenneth Lee
2018-08-02  4:24       ` Tian, Kevin
2018-08-02  7:34         ` Kenneth Lee
     [not found]           ` <20180802103528.0b863030.cohuck@redhat.com>
     [not found]             ` <20180802124327.403b10ab@t450s.home>
2018-08-06  1:40               ` Kenneth Lee
2018-08-06 15:49                 ` Alex Williamson
2018-08-06 16:34                   ` Raj, Ashok
2018-08-06 17:05                     ` Alex Williamson
2018-08-08  1:32                       ` Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 4/7] crypto: add hisilicon Queue Manager driver Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 5/7] crypto: Add Hisilicon Zip driver Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 6/7] crypto: add spimdev support to Hisilicon QM Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 7/7] vfio/spimdev: add user sample for spimdev Kenneth Lee
2018-08-01 16:56 ` [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive Jerome Glisse
2018-08-02  2:33   ` Tian, Kevin
2018-08-02  4:05     ` Kenneth Lee
2018-08-02 14:22       ` Jerome Glisse
2018-08-03  3:47         ` Kenneth Lee
2018-08-03 14:39           ` Jerome Glisse
2018-08-06  3:12             ` Kenneth Lee
2018-08-06 15:32               ` Jerome Glisse
2018-08-08  1:08                 ` Kenneth Lee
2018-08-08 15:18                   ` Jerome Glisse
2018-08-09  8:03                     ` Kenneth Lee
2018-08-09  8:31                       ` Tian, Kevin
2018-08-10  1:37                         ` Kenneth Lee
2018-08-09 14:46                       ` Jerome Glisse
2018-08-10  3:39                         ` Kenneth Lee
2018-08-10 13:12                           ` Jean-Philippe Brucker
2018-08-11 15:26                             ` Kenneth Lee
2018-08-13  9:29                               ` Kenneth Lee
2018-08-13 19:23                                 ` Jerome Glisse
2018-08-14  3:46                                   ` Kenneth Lee
2018-08-10 14:32                           ` Jerome Glisse
2018-08-11 14:44                             ` Kenneth Lee
2018-08-02 10:10     ` Alan Cox
2018-08-02 12:24       ` Xu Zaibo
2018-08-02 14:46       ` Jerome Glisse
2018-08-03 14:20         ` Alan Cox
2018-08-03 14:55           ` Jerome Glisse
2018-08-06  1:26           ` Kenneth Lee
2018-08-02  2:59 ` Tian, Kevin
2018-08-02  3:40   ` Kenneth Lee [this message]
2018-08-02  4:36     ` Tian, Kevin
2018-08-02  5:35       ` Kenneth Lee

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180802034006.GJ160746@Turing-Arch-b \
    --to=liguozhu@hisilicon.com \
    --cc=alex.williamson@redhat.com \
    --cc=baolu.lu@linux.intel.com \
    --cc=corbet@lwn.net \
    --cc=davem@davemloft.net \
    --cc=fanghao11@huawei.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=herbert@gondor.apana.org.au \
    --cc=iommu@lists.linux-foundation.org \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=linux-accelerators@lists.ozlabs.org \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxarm@huawei.com \
    --cc=nek.in.cn@gmail.com \
    --cc=pombredanne@nexb.com \
    --cc=sanjay.k.kumar@intel.com \
    --cc=tglx@linutronix.de \
    --cc=wangzhou1@hisilicon.com \
    --cc=xuzaibo@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git