From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ahmed Mansour Subject: Re: [PATCH] compressdev: implement API Date: Mon, 26 Feb 2018 19:15:44 +0000 Message-ID: References: <1517595924-25963-1-git-send-email-fiona.trahe@intel.com> <12544144.czVLKRyaz4@xps> <348A99DA5F5B7549AA880327E580B43589325187@IRSMSX101.ger.corp.intel.com> <348A99DA5F5B7549AA880327E580B435893279DB@IRSMSX101.ger.corp.intel.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Cc: "De Lara Guarch, Pablo" , "Athreya, Narayana Prasad" , "Gupta, Ashish" , "Sahu, Sunila" , "Challa, Mahipal" , "Jain, Deepak K" , Hemant Agrawal , Roy Pledge , Youri Querry To: "Trahe, Fiona" , "dev@dpdk.org" , "Shally.Verma@cavium.com" Return-path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0086.outbound.protection.outlook.com [104.47.0.86]) by dpdk.org (Postfix) with ESMTP id D720D126B for ; Mon, 26 Feb 2018 20:15:47 +0100 (CET) Content-Language: en-US List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" >> Hi Fiona,=0A= >>=0A= >> Thanks for starting this discussion. In the current API the user must=0A= >> make 12 API calls just to get information to compress. Maybe there is a= =0A= >> way to simplify. At least for some use cases (stateless). I think a call= =0A= >> sometime next week would be good to help clarify coalesce some of the=0A= >> complexity.=0A= > [Fiona] Would 10:30 GMT on Wednesday 28th Feb suit?=0A= [Ahmed] I am in Ottawa Canada so Wednesday 12:00 or 1:00 GMT would be=0A= better. Does that work?=0A= >> I added specific comments inline.=0A= >>=0A= >> Thanks,=0A= >>=0A= >> Ahmed=0A= >>=0A= >> On 2/21/2018 2:12 PM, Trahe, Fiona wrote:=0A= >>> We've been struggling with the idea of session in compressdev.=0A= >>>=0A= >>> Is it really a session?=0A= >>> - It's not in the same sense as cryptodev where it's used to hold a ke= y, and maps to a Security=0A= >> Association.=0A= >>> - It's a set of immutable data that is needed with the op and stream t= o perform the operation.=0A= >>> - It inherited from cryptodev the ability to be set up for multiple dr= iver types and used across any=0A= >>> devices of those types. For stateful ops this facility can't be use= d.=0A= >>> For stateless we don't think it's important, and think it's unlikel= y to be used.=0A= >>> - Drivers use it to prepare private data, set up resources, do pre-wor= k, so there's=0A= >>> less work to be done on the data path. Initially we didn't have a s= tream, we do now,=0A= >>> this may be a better alternative place for that work.=0A= >>> So we've been toying with the idea of getting rid of the session.=0A= >> [Ahmed] In our proprietary API the stream and session are one. A session= =0A= >> holds many properties like the op-type, instead of having this=0A= >> information in the op itself. This way we lower the per op setup cost.= =0A= >> This also allows rapid reuse of stateful infrastructure, once a stream= =0A= >> is closed on a stateful session, the next op (stream) on this session=0A= >> reuses the stateful storage. Obviously if a stream is in "pause mode" on= =0A= >> a session, all following ops that may be unrelated to this=0A= >> stream/session must also wait until this current stream is closed or=0A= >> aborted before the infrastructure can be reused.=0A= >>> We also struggle with the idea of setting up a stream for stateless ops= .=0A= >>> - Well, really I just think the name is misleading, i.e. there's no p= roblem with setting=0A= >>> up some private PMD data to use with stateless operations, just cal= ling it a=0A= >>> stream doesn't seem right.=0A= >> [Ahmed] I agree. The op has all the necessary information to process it= =0A= >> in the current API? Both the stream and the op are one time use. We=0A= >> can't attach multiple similar ops to a single stream/session and rely on= =0A= >> their properties to simplify op setup, so why the hassle.=0A= >>> So putting above thoughts together I want to propose:=0A= >>> - Removal of the session and all associated APIs.=0A= >>> - Passing in one of three data types in the rte_comp_op=0A= >>>=0A= >>> union {=0A= >>> struct rte_comp_xform *xform;=0A= >>> /**< Immutable compress/decompress params */=0A= >>> void *pmd_stateless_data;=0A= >>> /**< Stateless private PMD data derived from an rte_comp_xform= =0A= >>> * rte_comp_stateless_data_init() must be called on a device=0A= >>> * before sending any STATELESS operations. If the PMD returns = a non-NULL=0A= >>> * value the handle must be attached to subsequent STATELESS op= erations.=0A= >>> * If a PMD returns NULL, then the xform should be passed direc= tly to each op=0A= >>> */=0A= >>> void *stream;=0A= >>> /* Private PMD data derived initially from an rte_comp_xform, w= hich holds state=0A= >>> * and history data and evolves as operations are processed.=0A= >>> * rte_comp_stream_create() must be called on a device for all = STATEFUL=0A= >>> * data streams and the resulting stream attached=0A= >>> * to the one or more operations associated with the data strea= m.=0A= >>> * All operations in a stream must be sent to the same device.= =0A= >>> */=0A= >>> }=0A= >> [Ahmed] I like this setup, but I am not sure in what cases the xform=0A= >> immutable would be used. I understand the other two.=0A= > [Fiona] The xform is there because I don't know yet what limitations may = apply to the=0A= > pmd_stateless_data. If it has no limitation and once set up once on a dev= ice =0A= > can be attached simultaneously to any op sent to any qp on that device=0A= > then we don't need the xform. But I understood from Shally's earlier requ= est for=0A= > setting up a stream on a stateless request that some resources are=0A= > allocated, so we may need to document some limitations.=0A= > In this case the xform may be a better path for PMDs which don't have the= same=0A= > limitations. =0A= >=0A= >=0A= >>> Notes:=0A= >>> 1. Internally if a PMD wants to use the exact same data structure for b= oth it can do,=0A= >>> just on the API I think it's better if they're named differently w= ith=0A= >>> different comments.=0A= >>> 2. I'm not clear of the constraints if any, which attach to the pmd_sta= teless_data=0A= >>> For our PMD it would only hold immutable data as the session did, = and so=0A= >>> could be attached to many ops in parallel.=0A= >>> Is this true for all PMDs or are there constraints which should be= called out?=0A= >>> Is it limited to a specific device, qp, or to be used on one op at= a time?=0A= >>> 3. Am open to other naming suggestions, just trying to capture the esse= nce=0A= >>> of these data structs better than our current API does.=0A= >>>=0A= >>> We would put some more helper fns and structure around the above code i= f people=0A= >>> are in agreement, just want to see if the concept flies before going fu= rther?=0A= >>>=0A= >>> Fiona=0A= >>>=0A= >>>=0A= >>>=0A= >=0A= =0A=