From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Dumitrescu, Cristian" Subject: Re: [PATCH v2 1/2] librte_net: add crc init and compute APIs Date: Wed, 15 Mar 2017 21:11:55 +0000 Message-ID: <3EB4FA525960D640B5BDFFD6A3D8912652760A86@IRSMSX108.ger.corp.intel.com> References: <1487969657-172541-2-git-send-email-jasvinder.singh@intel.com> <6450918.dWRLMxV3EZ@xps13> <3EB4FA525960D640B5BDFFD6A3D89126527609A4@IRSMSX108.ger.corp.intel.com> <9340287.qWvXv5QBAa@xps13> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Cc: "De Lara Guarch, Pablo" , "Singh, Jasvinder" , "dev@dpdk.org" , "Doherty, Declan" To: Thomas Monjalon Return-path: Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by dpdk.org (Postfix) with ESMTP id 4E902133F for ; Wed, 15 Mar 2017 22:11:58 +0100 (CET) In-Reply-To: <9340287.qWvXv5QBAa@xps13> Content-Language: en-US List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" > -----Original Message----- > From: Thomas Monjalon [mailto:thomas.monjalon@6wind.com] > Sent: Wednesday, March 15, 2017 8:16 PM > To: Dumitrescu, Cristian > Cc: De Lara Guarch, Pablo ; Singh, > Jasvinder ; dev@dpdk.org; Doherty, Declan > > Subject: Re: [dpdk-dev] [PATCH v2 1/2] librte_net: add crc init and compu= te > APIs >=20 > 2017-03-15 19:03, Dumitrescu, Cristian: > > ... > > > > > > > > > I think it should be in librte_hash. > > > > > > > > > > > > > > Please check lib/librte_hash/rte_hash_crc.h > > > > > > > > > > > > Is it good to include payload crc calculation in hash library a= s I see all > > > hash > > > > > related functionality there? > > > > > > > > > > I think yes. Pablo? > > > > > > > > I think this doesn't belong in the hash library. These new function= s > calculate > > > CRC, but not as a hash function. > > > > > > Can't we say that a CRC is a hash? What is a hash? > > > A function generating the same output bytes from given input bytes. > > > > > > I think you must separate hash functions and hash table management. > > > > > > > The fact that CRC32 instruction is opportunistically used to compute a = hash > digest/signature for load balancing (affinity-based) or hash tables (flow > tables, ARP cache, etc) does not mean that all the code that uses CRC32 > instruction should be placed in librte_hash. > > > > The purpose of the hash functions in librte_hash is to compute a > digest/signature for a given array of bytes (the key) as fast as possible= . Any > hash function that produces a hash signature with good uniform distributi= on > in a small amount of cycles belongs here, including those opportunistical= ly > using specialized CPU instructions such as CRC32 (or XOR, AESNI, etc). > > > > The code proposed in this patch is used to compute packet fields for > various protocols that have a CRC field, such as FCS of Ethernet frames, = etc. > according to the relevant standard (IEEE 802, others). It is an utility t= o be used > for implementing protocol-level functionality for various protocols from = the > network stack, similar to e.g. IP or UDP checksum. If we were to add an I= P or > UDCP checksum calculator, would you put it in librte_hash? > > > > The code from this patch is never going to be used to compute a fast > signature/digest. Typically this CRC is computed over the entire frame/pa= cket > rather than just selected fields from the packet representing the applica= tion- > specific flow key. Also note that the signature produced by CRC32 hash > function from librte_hash is actually not the correct Cyclic Redundancy C= heck > of that array of bytes (or, for math guys, of the associated polynomial),= it is > just a partial/intermediate value. > > > > Therefore, I suggest placing this code into: librte_ether (given that i= t can be > used to compute Ethernet FCS), or librte_net, or librte_crc. Definitely n= ot in > librte_hash. >=20 > I agree with you Cristian that the protocol layer must be in librte_net. > But I think most of this patch is not protocol level. Nope, this is the true CRC computed over entire protocol header and/or payl= oad. Similar to to IPv4/UDP/TCP checksum. The only reason for computing it = is because the protocol specs require it for data integrity checks, nothing= to do with our signature for load balancing/hash tables. More details on covered protocols from a reliable source :) [1]: CRC-32 (polynomial =3D 0x04C11DB7): used for HDLC, ANSI X3.66, ITU-T V.42,= Ethernet, Serial ATA, MPEG-2, PKZIP, Gzip, Bzip2, PNG, many others CRC-16-CCITT (polynomial =3D 0x1021): used for X.25, V.41, HDLC FCS, XMODE= M, Bluetooth, PACTOR, SD, DigRF, many others [1] Wikipedia: https://en.wikipedia.org/wiki/Cyclic_redundancy_check#Standa= rds_and_common_use > I think you agree with me that the code computing a > "digest/signature for a given array of bytes" must go to librte_hash? Yes, but this is the true protocol-level CRC, not a digest/signature. Non-cryptographic hash digest/signature: -computed over selected packet fields (flow key) for load balancing (affini= ty scheme) or hash table; key size typical range: 8 .. 192 bytes -required by application requirements (such as flow packet ordering preserv= ation), not by protocol standards -has uniform distribution -requires small amount of cycles to compute -used as meta-data, not written in the packet -can be opportunistically generated using specialized CPU instructions, suc= h as CRC32 (or XEOR, or AESNI); in this case, it is a partial/intermediate = value, not the correct CRC of the array of bytes Protocol CRC: -computed over entire packet header and/or payload -protocol overhead (required by standards) -computational cost is typically big and proportional with the packet lengt= h; packet length typical range: 64 .. 1514 .. 9K -written in the packet (by the application SW or by the HW)