LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Kenneth Lee <nek.in.cn@gmail.com>
To: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>,
	Kenneth Lee <liguozhu@hisilicon.com>,
	Jerome Glisse <jglisse@redhat.com>
Cc: Herbert Xu <herbert@gondor.apana.org.au>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Zaibo Xu <xuzaibo@huawei.com>,
	"linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
	"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linuxarm@huawei.com" <linuxarm@huawei.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	"linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
	Philippe Ombredanne <pombredanne@nexb.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Hao Fang <fanghao11@huawei.com>,
	"David S . Miller" <davem@davemloft.net>,
	"linux-accelerators@lists.ozlabs.org" 
	<linux-accelerators@lists.ozlabs.org>
Subject: Re: [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive
Date: Sat, 11 Aug 2018 23:26:48 +0800
Message-ID: <6ea4dcfd-d539-93e4-acf1-d09ea35f0ddc@gmail.com> (raw)
In-Reply-To: <0f6bac9b-8381-1874-9367-46b5f4cef56e@arm.com>



在 2018年08月10日 星期五 09:12 下午, Jean-Philippe Brucker 写道:
> Hi Kenneth,
>
> On 10/08/18 04:39, Kenneth Lee wrote:
>>> You can achieve everything you want to achieve with existing upstream
>>> solution. Re-inventing a whole new driver infrastructure should really
>>> be motivated with strong and obvious reasons.
>> I want to understand better of your idea. If I create some unified helper
>> APIs in drivers/iommu/, say:
>>
>> 	wd_create_dev(parent_dev, wd_dev)
>> 	wd_release_dev(wd_dev)
>>
>> The API create chrdev to take request from user space for open(resource
>> allocation), iomap, epoll (irq), and dma_map(with pasid automatically).
>>
>> Do you think it is acceptable?
> Maybe not drivers/iommu/ :) That subsystem only contains tools for
> dealing with DMA, I don't think epoll, resource enumeration or iomap fit
> in there.
Yes. I should consider where to put it carefully.
>
> Creating new helpers seems to be precisely what we're trying to avoid in
> this thread, and vfio-mdev does provide the components that you
> describe, so I wouldn't discard it right away. When the GPU, net, block
> or another subsystem doesn't fit your needs, either because your
> accelerator provides some specialized function, or because for
> performance reasons your client wants direct MMIO access, you can at
> least build your driver and library on top of those existing VFIO
> components:
>
> * open allocates a partition of an accelerator.
> * vfio_device_info, vfio_region_info and vfio_irq_info enumerates
> available resources.
> * vfio_irq_set deals with epoll.
> * mmap gives you a private MMIO doorbell.
> * vfio_iommu_type1 provides the DMA operations.
>
> Currently missing:
>
> * Sharing the parent IOMMU between mdev, which is also what the "IOMMU
> aware mediated device" series tackles, and seems like a logical addition
> to VFIO. I'd argue that the existing IOMMU ops (or ones implemented by
> the SVA series) can be used to deal with this
>
> * The interface to discover an accelerator near your memory node, or one
> that you can chain with other devices. If I understood correctly the
> conclusion was that the API (a topology description in sysfs?) should be
> common to various subsystems, in which case vfio-mdev (or the mediating
> driver) could also use it.
>
> * The queue abstraction discussed on patch 3/7. Perhaps the current vfio
> resource description of MMIO and IRQ is sufficient here as well, since
> vendors tend to each implement their own queue schemes. If you need
> additional features, read/write fops give the mediating driver a lot of
> freedom. To support features that are too specific for drivers/vfio/ you
> can implement a config space with capabilities and registers of your
> choice. If you're versioning the capabilities, the code to handle them
> could even be shared between different accelerator drivers and libraries.
Thank you, Jean,

The major reason that I want to remove dependency to VFIO is: I accepted 
that the whole logic of VFIO was built on the idea of creating virtual 
device.

Let's consider it in this way: We have hardware with IOMMU support. So 
we create a default_domain to the particular IOMMU (unit) in the group 
for the kernel driver to use it. Now the device is going to be used by a 
VM or a Container. So we unbind it from the original driver, and put the 
default_domain away,  create a new domain for this particular use case.  
So now the device shows up as a platform or pci device to the user 
space. This is what VFIO try to provide. Mdev extends the scenario but 
dose not change the intention. And I think that is why Alex emphasis 
pre-allocating resource to the mdev.

But what WarpDrive need is to get service from the hardware itself and 
set mapping to its current domain, aka defaut_domain. If we do it in 
VFIO-mdev, it looks like the VFIO framework takes all the effort to put 
the default_domain away and create a new one and be ready for user space 
to use. But I tell him stop using the new domain and try the original one...

It is not reasonable, isn't it:)

So why don't I just take the request and set it into the default_domain 
directly? The true requirement of WarpDrive is to let process set the 
page table for particular pasid or substream id, so it can accept 
command with address in the process space. It needs no device.

 From this perspective, it seems there is no reason to keep it in VFIO.

Thanks
Kenneth
>
> Thanks,
> Jean
>


  reply index

Thread overview: 61+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-01 10:22 Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 1/7] vfio/spimdev: Add documents for WarpDrive framework Kenneth Lee
2018-08-02  3:14   ` Tian, Kevin
2018-08-02  4:22     ` Kenneth Lee
2018-08-02  4:41       ` Tian, Kevin
2018-08-06 12:27   ` Pavel Machek
2018-08-08  1:43     ` Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 2/7] iommu: Add share domain interface in iommu for spimdev Kenneth Lee
2018-08-02  3:17   ` Tian, Kevin
2018-08-02  4:15     ` Kenneth Lee
2018-08-02  4:39       ` Tian, Kevin
2018-08-08  9:13   ` Joerg Roedel
2018-08-09  1:09     ` Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 3/7] vfio: add spimdev support Kenneth Lee
2018-08-01 16:23   ` Randy Dunlap
2018-08-02  3:07     ` Kenneth Lee
2018-08-02  3:21   ` Tian, Kevin
2018-08-02  3:47     ` Kenneth Lee
2018-08-02  4:24       ` Tian, Kevin
2018-08-02  7:34         ` Kenneth Lee
     [not found]           ` <20180802103528.0b863030.cohuck@redhat.com>
     [not found]             ` <20180802124327.403b10ab@t450s.home>
2018-08-06  1:40               ` Kenneth Lee
2018-08-06 15:49                 ` Alex Williamson
2018-08-06 16:34                   ` Raj, Ashok
2018-08-06 17:05                     ` Alex Williamson
2018-08-08  1:32                       ` Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 4/7] crypto: add hisilicon Queue Manager driver Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 5/7] crypto: Add Hisilicon Zip driver Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 6/7] crypto: add spimdev support to Hisilicon QM Kenneth Lee
2018-08-01 10:22 ` [RFC PATCH 7/7] vfio/spimdev: add user sample for spimdev Kenneth Lee
2018-08-01 16:56 ` [RFC PATCH 0/7] A General Accelerator Framework, WarpDrive Jerome Glisse
2018-08-02  2:33   ` Tian, Kevin
2018-08-02  4:05     ` Kenneth Lee
2018-08-02 14:22       ` Jerome Glisse
2018-08-03  3:47         ` Kenneth Lee
2018-08-03 14:39           ` Jerome Glisse
2018-08-06  3:12             ` Kenneth Lee
2018-08-06 15:32               ` Jerome Glisse
2018-08-08  1:08                 ` Kenneth Lee
2018-08-08 15:18                   ` Jerome Glisse
2018-08-09  8:03                     ` Kenneth Lee
2018-08-09  8:31                       ` Tian, Kevin
2018-08-10  1:37                         ` Kenneth Lee
2018-08-09 14:46                       ` Jerome Glisse
2018-08-10  3:39                         ` Kenneth Lee
2018-08-10 13:12                           ` Jean-Philippe Brucker
2018-08-11 15:26                             ` Kenneth Lee [this message]
2018-08-13  9:29                               ` Kenneth Lee
2018-08-13 19:23                                 ` Jerome Glisse
2018-08-14  3:46                                   ` Kenneth Lee
2018-08-10 14:32                           ` Jerome Glisse
2018-08-11 14:44                             ` Kenneth Lee
2018-08-02 10:10     ` Alan Cox
2018-08-02 12:24       ` Xu Zaibo
2018-08-02 14:46       ` Jerome Glisse
2018-08-03 14:20         ` Alan Cox
2018-08-03 14:55           ` Jerome Glisse
2018-08-06  1:26           ` Kenneth Lee
2018-08-02  2:59 ` Tian, Kevin
2018-08-02  3:40   ` Kenneth Lee
2018-08-02  4:36     ` Tian, Kevin
2018-08-02  5:35       ` Kenneth Lee

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6ea4dcfd-d539-93e4-acf1-d09ea35f0ddc@gmail.com \
    --to=nek.in.cn@gmail.com \
    --cc=alex.williamson@redhat.com \
    --cc=corbet@lwn.net \
    --cc=davem@davemloft.net \
    --cc=fanghao11@huawei.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=herbert@gondor.apana.org.au \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jean-philippe.brucker@arm.com \
    --cc=jglisse@redhat.com \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=liguozhu@hisilicon.com \
    --cc=linux-accelerators@lists.ozlabs.org \
    --cc=linux-crypto@vger.kernel.org \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxarm@huawei.com \
    --cc=pombredanne@nexb.com \
    --cc=sanjay.k.kumar@intel.com \
    --cc=tglx@linutronix.de \
    --cc=xuzaibo@huawei.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git