From mboxrd@z Thu Jan  1 00:00:00 1970
From: "Xu, Rosen" <rosen.xu@intel.com>
Subject: Re: [PATCH V2 3/5] Add Intel FPGA BUS Lib Code
Date: Wed, 21 Mar 2018 14:06:33 +0000
Message-ID: <0E78D399C70DA940A335608C6ED296D739F216E2@SHSMSX104.ccr.corp.intel.com>
References: <1521618694-140757-1-git-send-email-rosen.xu@intel.com>
 <1521618694-140757-4-git-send-email-rosen.xu@intel.com>
 <20180321102025.v2zeijxrsauuvkss@bidouze.vm.6wind.com>
 <20180321133509.GA15364@bricha3-MOBL.ger.corp.intel.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Cc: "dev@dpdk.org" <dev@dpdk.org>, "Doherty, Declan"
 <declan.doherty@intel.com>, "shreyansh.jain@nxp.com"
 <shreyansh.jain@nxp.com>, "Zhang, Tianfei" <tianfei.zhang@intel.com>, "Wu,
 Hao" <hao.wu@intel.com>
To: "Richardson, Bruce" <bruce.richardson@intel.com>, Ga?tan Rivet
 <gaetan.rivet@6wind.com>
Return-path: <dev-bounces@dpdk.org>
Received: from mga12.intel.com (mga12.intel.com [192.55.52.136])
 by dpdk.org (Postfix) with ESMTP id 856F72C54
 for <dev@dpdk.org>; Wed, 21 Mar 2018 15:06:38 +0100 (CET)
In-Reply-To: <20180321133509.GA15364@bricha3-MOBL.ger.corp.intel.com>
Content-Language: en-US
List-Id: DPDK patches and discussions <dev.dpdk.org>
List-Unsubscribe: <https://dpdk.org/ml/options/dev>,
 <mailto:dev-request@dpdk.org?subject=unsubscribe>
List-Archive: <http://dpdk.org/ml/archives/dev/>
List-Post: <mailto:dev@dpdk.org>
List-Help: <mailto:dev-request@dpdk.org?subject=help>
List-Subscribe: <https://dpdk.org/ml/listinfo/dev>,
 <mailto:dev-request@dpdk.org?subject=subscribe>
Errors-To: dev-bounces@dpdk.org
Sender: "dev" <dev-bounces@dpdk.org>



-----Original Message-----
From: Richardson, Bruce=20
Sent: Wednesday, March 21, 2018 21:35
To: Ga=EBtan Rivet <gaetan.rivet@6wind.com>
Cc: Xu, Rosen <rosen.xu@intel.com>; dev@dpdk.org; Doherty, Declan <declan.d=
oherty@intel.com>; shreyansh.jain@nxp.com; Zhang, Tianfei <tianfei.zhang@in=
tel.com>; Wu, Hao <hao.wu@intel.com>
Subject: Re: [PATCH V2 3/5] Add Intel FPGA BUS Lib Code

On Wed, Mar 21, 2018 at 11:20:25AM +0100, Ga=EBtan Rivet wrote:
> > Hi,
> >=20
> > I have had issues compiling a few things here, have you checked build=20
> > status before submitting?
> >=20
> > On Wed, Mar 21, 2018 at 03:51:32PM +0800, Rosen Xu wrote:
> > > Signed-off-by: Rosen Xu <rosen.xu@intel.com>
> > > ---
<snip>
> > +
> > > +/*
> > > + * Scan the content of the FPGA bus, and the devices in the devices
> > > + * list
> > > + */
> >=20
> > So you seem to scan your bus by reading parameters given to the=20
> > --ifpga EAL option.
> >=20
> > Can you justify why you cannot use the PCI bus, have your FPGA be=20
> > probed by a PCI driver, that would take those parameters as driver=20
> > parameters, and spawn raw devices (one per bitstream) as needed as a re=
sult?
> >=20
> > I see no reason this is not feasible. Unless you duly justify this=20
> > approach, it seems unacceptable to me. You are subverting generic EAL=20
> > code to bend things to your approach, without clear rationale.
> >=20
> While I agree with the comments in other emails about avoiding special-ca=
ses in the code=20
>  that makes things not-scalable, I would take the view that using a bus-t=
ype is the correct=20
> choice for this. While you could have a single device that creates other =
devices, that is also
> true for all other buses as well.  [Furthermore, I think it is incorrect =
assume that all devices=20
> on the FPGA bus would be raw devices, it's entirely possible to have cryp=
todevs, bbdevs or compress devs implemented in the AFUs].
>
> Consider what a bus driver provides: it's a generic mechanism for scannin=
g for=20
> devices - which all use a common connection method - for DPDK use, and=20
> mapping device drivers to those devices. For an FPGA device which present=
s=20
> multiple AFUs, this seems to be exactly what is required - a device drive=
r to=20
> scan for devices and present them to DPDK. The FPGA bus driver will have=
=20
> to check each AFU and match it against the set of registered AFU device=20
> drivers to ensure that the crypto AFU gets the cryptodev driver, etc.
>
> Logically, therefore, it is a bus - which just happens to be a sub-bus of=
=20
> PCI, i.e. presented as a PCI device. Consider also that it may be possibl=
e=20
> or even desirable, to use blacklisting and whitelisting for those AFU=20
> devices so that some AFUs could be used by one app, while others by=20
> another. If we just have a single PCI device, I think we'll find ourselve=
s=20
> duplicating a lot of bus-related functionality inside the driver in that =
case.

In our FPGA Usage Framework, each FPAG Bitstream is divided 2 parts,=20
one part is only one Blue AFU, another part consist of many Green AFUs.

Blue AFU includes PCIe Interface and FPGA PR Unit, Blue AFU is fixed after
OS Initialization, because if we change Blue AFU, the PCIe Interface needed=
 to be=20
rescanned and the OS need to reboot.

Green AFUs can be dynamically PR by different Users.

The benefit of this FPGA Architecture is that we can dynamically change Gre=
en AFUs,
but OS don't need to rescan the FPGA PCIe Interface. For Cloud Scenario the=
 FPGA=20
Device can be viewed a common resource pools such as DDR Memory, and it can=
 be easily
assigned to different users at some time. For TelCom/NFV Scenario, we can e=
asily
upgrade the Acceleration AFU but the server don't need to reboot.

For Software Usage, there many FPGA Device in one system, some have the sam=
e
Green AFUs and some have different Green AFUs, same Green AFUs means same A=
cceleration,=20
and it will use same driver. So we want to involve a new bus,
which can easily bind the same Green AFUs with its driver.


Regards,
/Bruce