From: Ori Kam <orika@mellanox.com>
To: Wang Xiang <xiang.w.wang@intel.com>
Cc: "jerinj@marvell.com" <jerinj@marvell.com>,
"dev@dpdk.org" <dev@dpdk.org>,
"pbhagavatula@marvell.com" <pbhagavatula@marvell.com>,
Shahaf Shuler <shahafs@mellanox.com>,
"hemant.agrawal@nxp.com" <hemant.agrawal@nxp.com>,
Opher Reviv <opher@mellanox.com>,
Alex Rosenbaum <alexr@mellanox.com>,
"dovrat@marvell.com" <dovrat@marvell.com>,
"pkapoor@marvell.com" <pkapoor@marvell.com>,
"nipun.gupta@nxp.com" <nipun.gupta@nxp.com>,
"bruce.richardson@intel.com" <bruce.richardson@intel.com>,
"yang.a.hong@intel.com" <yang.a.hong@intel.com>,
"harry.chang@intel.com" <harry.chang@intel.com>,
"gu.jian1@zte.com.cn" <gu.jian1@zte.com.cn>,
"shanjiangh@chinatelecom.cn" <shanjiangh@chinatelecom.cn>,
"zhangy.yun@chinatelecom.cn" <zhangy.yun@chinatelecom.cn>,
"lixingfu@huachentel.com" <lixingfu@huachentel.com>,
"wushuai@inspur.com" <wushuai@inspur.com>,
"yuyingxia@yxlink.com" <yuyingxia@yxlink.com>,
"fanchenggang@sunyainfo.com" <fanchenggang@sunyainfo.com>,
"davidfgao@tencent.com" <davidfgao@tencent.com>,
"liuzhong1@chinaunicom.cn" <liuzhong1@chinaunicom.cn>,
"zhaoyong11@huawei.com" <zhaoyong11@huawei.com>,
"oc@yunify.com" <oc@yunify.com>,
"jim@netgate.com" <jim@netgate.com>,
"hongjun.ni@intel.com" <hongjun.ni@intel.com>,
"j.bromhead@titan-ic.com" <j.bromhead@titan-ic.com>,
"deri@ntop.org" <deri@ntop.org>,
"fc@napatech.com" <fc@napatech.com>,
"arthur.su@lionic.com" <arthur.su@lionic.com>,
Thomas Monjalon <thomas@monjalon.net>
Subject: Re: [dpdk-dev] [RFC v5] regexdev: introduce regexdev subsystem
Date: Tue, 3 Mar 2020 07:44:18 +0000 [thread overview]
Message-ID: <AM6PR05MB51764668F2C742EB1D99D5F9DBE40@AM6PR05MB5176.eurprd05.prod.outlook.com> (raw)
In-Reply-To: <20200302070500.GA31870@hyperscan>
Hi Xiang,
May I ask when do you plan to add the Hyperscan code to the DPDK?
> -----Original Message-----
> From: dev <dev-bounces@dpdk.org> On Behalf Of Wang Xiang
> Sent: Monday, March 2, 2020 9:05 AM
> To: Ori Kam <orika@mellanox.com>
> Cc: jerinj@marvell.com; dev@dpdk.org; pbhagavatula@marvell.com; Shahaf
> Shuler <shahafs@mellanox.com>; hemant.agrawal@nxp.com; Opher Reviv
> <opher@mellanox.com>; Alex Rosenbaum <alexr@mellanox.com>;
> dovrat@marvell.com; pkapoor@marvell.com; nipun.gupta@nxp.com;
> bruce.richardson@intel.com; yang.a.hong@intel.com; harry.chang@intel.com;
> gu.jian1@zte.com.cn; shanjiangh@chinatelecom.cn;
> zhangy.yun@chinatelecom.cn; lixingfu@huachentel.com; wushuai@inspur.com;
> yuyingxia@yxlink.com; fanchenggang@sunyainfo.com;
> davidfgao@tencent.com; liuzhong1@chinaunicom.cn;
> zhaoyong11@huawei.com; oc@yunify.com; jim@netgate.com;
> hongjun.ni@intel.com; j.bromhead@titan-ic.com; deri@ntop.org;
> fc@napatech.com; arthur.su@lionic.com; Thomas Monjalon
> <thomas@monjalon.net>
> Subject: Re: [dpdk-dev] [RFC v5] regexdev: introduce regexdev subsystem
>
> Hi Ori,
>
> Comments below.
>
> Thanks,
> Xiang
>
> On Thu, Feb 27, 2020 at 03:08:35PM +0000, Ori Kam wrote:
> > From: Jerin Jacob <jerinj@marvell.com>
> >
> > Even though there are some vendors which offer Regex HW offload, due to
> > lack of standard API, It is diffcult for DPDK consumer to use them
> > in a portable way.
> >
> > This _RFC_ attempts to standardize the RegEx/DPI offload APIs for DPDK.
> >
> > This RFC crafted based on SW Regex API frameworks such as libpcre and
> > hyperscan and a few of the RegEx HW IPs which I am aware of.
> >
> > RegEx pattern matching applications:
> > * Next Generation Firewalls (NGFW)
> > * Deep Packet and Flow Inspection (DPI)
> > * Intrusion Prevention Systems (IPS)
> > * DDoS Mitigation
> > * Network Monitoring
> > * Data Loss Prevention (DLP)
> > * Smart NICs
> > * Grammar based content processing
> > * URL, spam and adware filtering
> > * Advanced auditing and policing of user/application security policies
> > * Financial data mining - parsing of streamed financial feeds
> > * Application recognition.
> > * Dmemory introspection.
> > * Natural Language Processing (NLP)
> > * Sentiment Analysis.
> > * Big data databse acceleration.
> > * Computational storage.
> >
> > Request to review from HW and SW RegEx vendors and RegEx application
> > users to have portable DPDK API for RegEx.
> >
> > The API schematics are based cryptodev, eventdev and ethdev existing
> > device API.
> >
> > Signed-off-by: Jerin Jacob <jerinj@marvell.com>
> > Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
> > Signed-off-by: Ori Kam <orika@mellanox.com>
> > ---
> > V5:
> > * Remove unused iov struct.
> > V4:
> > * Replace iov with mbuf.
> > * Small ML comments.
> > V3:
> > * Change subject title.
> > V2:
> > * Address ML comments.
> > ---
> > +
> > +#define RTE_REGEX_DEV_SUPP_PCRE_GREEDY_F (1ULL << 6)
> > +/**< RegEx device support PCRE Greedy mode.
> > + * For example if the RegEx is 'AB\d*?' then '*?' represents zero or
> unlimited
> > + * matches. In greedy mode the pattern 'AB12345' will be matched
> completely
> > + * where as the ungreedy mode 'AB' will be returned as the match.
> > + * @see struct rte_regex_dev_info::regex_dev_capa
> > + */
> > +
>
> Hyperscan actually supports "match all" semantic, neither greedy nor
> ungreedy,
> which is different from PCRE. In the case above, AB, AB1, ..., AB12345 will all
> be returned as matches. Do HW solutions support this?
No our HW doesn't support this.
Jerin, does Marvell HW support this?
> Can we add a new flag like RTE_REGEX_DEV_SUPP_PCRE_MATCHALL_F?
> Similarly, we can define a flag RTE_REGEX_PCRE_RULE_MATCHALL_F so
> Hyperscan
> users have to set this flag during rule compile.
>
Sure,
> > +#define RTE_REGEX_DEV_SUPP_PCRE_LOOKAROUND_ASRT_F (1ULL << 7)
> > +/**< RegEx device support PCRE Lookaround assertions
> > + * (Zero-width assertions). Example RegEx is '[a-z]+\d+(?=!{3,})' if
> > + * the given pattern is 'dwad1234!' the RegEx engine doesn't report any
> matches
> > + * because the assert '(?=!{3,})' fails. The pattern 'dwad123!!!' would return
> a
> > + * successful match.
> > + * @see struct rte_regex_dev_info::regex_dev_capa
> > + */
> > +
> > +
> > +/**
> > + * RegEx device information
> > + */
> > +struct rte_regex_dev_info {
> > + const char *driver_name; /**< RegEx driver name. */
> > + struct rte_device *dev; /**< Device information. */
> > + uint16_t max_matches;
> > + /**< Maximum matches per scan supported by this device. */
> > + uint16_t max_queue_pairs;
> > + /**< Maximum queue pairs supported by this device. */
> > + uint16_t max_payload_size;
> > + /**< Maximum payload size for a pattern match request or scan.
> > + * @see RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F
> > + */
> > + uint32_t max_rules_per_group;
> > + /**< Maximum rules supported per group by this device. */
> > + uint16_t max_groups;
> > + /**< Maximum groups supported by this device. */
> > + uint32_t regex_dev_capa;
> > + /**< RegEx device capabilities. @see RTE_REGEX_DEV_CAPA_* */
> > + uint64_t rule_flags;
> > + /**< Supported compiler rule flags.
> > + * @see RTE_REGEX_PCRE_RULE_*, struct rte_regex_rule::rule_flags
> > + */
> > + uint8_t max_scatter_gather;
> > + /**< The max supported number of buffers that can
> > + * be used in a single ops. The total size of all elements
> > + * must be less then max_payload_size.
> > + */
>
> s/then/than
>
Will fix.
> > +};
> > +
> > +/**
> > + * @warning
> > + * @b EXPERIMENTAL: this API may change without prior notice.
> > + *
> > + * Retrieve the contextual information of a RegEx device.
> > + *
> > + * @param dev_id
> > + * The identifier of the device.
> > + *
> > + * @param[out] dev_info
> > + * A pointer to a structure of type *rte_regex_dev_info* to be filled with
> the
> > + * contextual information of the device.
> > + *
> > + * @return
> > + * - 0: Success, driver updates the contextual information of the RegEx
> device
> > + * - <0: Error code returned by the driver info get function.
> > + *
> > + */
> > +__rte_experimental
> > +int
> > +rte_regex_dev_info_get(uint8_t dev_id, struct rte_regex_dev_info
> *dev_info);
> > +
> > +/* Enumerates RegEx device configuration flags */
> > +#define RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F (1ULL << 0)
> > +/**< Cross buffer scan refers to the ability to be able to detect
> > + * matches that occur across buffer boundaries, where the buffers are
> related
> > + * to each other in some way. Enable this flag when to scan payload size
> > + * greater struct struct rte_regex_dev_info::max_payload_size and/or
> > + * matches can present across scan buffer boundaries.
>
> s/struct/than
>
Will fix.
> > + *
> > + * @see struct rte_regex_dev_info::max_payload_size
> > + * @see struct rte_regex_dev_config::dev_cfg_flags,
> rte_regex_dev_configure()
> > + * @see RTE_REGEX_OPS_RSP_PMI_SOJ_F
> > + * @see RTE_REGEX_OPS_RSP_PMI_EOJ_F
> > + */
> > +
> > +
> > +/* Enumerates RegEx response flags. */
> > +#define RTE_REGEX_OPS_RSP_PMI_SOJ_F (1 << 0)
> > +/**< Indicates that the RegEx device has encountered a partial match at the
> > + * start of scan in the given buffer.
> > + *
> > + * @see RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F
> > + */
> > +
> Hyperscan supports cross buffer scan and only reports true matches instead of
> partial matches. Can we have users to config this partial match capability?
>
Do you mean something like this:
RTE_REGEX_OPS_RSP_PMI_FOJ_F
/**< Indicates that the RegEx device has encountered a full match in the current buffer.
* The match was started in previous buffer.
*/
> > +#define RTE_REGEX_OPS_RSP_PMI_EOJ_F (1 << 1)
> > +/**< Indicates that the RegEx device has encountered a partial match at the
> > + * end of scan in the given buffer.
> > + *
> > + * @see RTE_REGEX_DEV_CFG_CROSS_BUFFER_SCAN_F
> > + */
> > +
next prev parent reply other threads:[~2020-03-03 7:44 UTC|newest]
Thread overview: 61+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-27 15:50 [dpdk-dev] [RFC PATCH v1] regexdev: introduce regexdev subsystem jerinj
2019-07-15 4:26 ` Jerin Jacob Kollanukkaran
2019-08-15 9:35 ` Thomas Monjalon
2019-08-15 11:34 ` Thomas Monjalon
2019-08-19 3:09 ` Jerin Jacob Kollanukkaran
2019-08-20 1:54 ` Wang, Xiang W
2019-09-10 8:05 ` Jerin Jacob Kollanukkaran
2019-09-19 13:58 ` Wang Xiang
2019-09-27 14:35 ` Jerin Jacob Kollanukkaran
2019-10-14 13:59 ` Wang Xiang
2020-01-26 11:55 ` Ori Kam
2019-08-21 5:32 ` Shahaf Shuler
2019-08-21 15:12 ` John Bromhead
2019-09-10 10:31 ` Jerin Jacob Kollanukkaran
2019-09-10 11:02 ` Jerin Jacob Kollanukkaran
2019-09-27 14:45 ` Jerin Jacob Kollanukkaran
2019-10-02 5:53 ` Shahaf Shuler
2019-10-02 8:31 ` Jerin Jacob Kollanukkaran
2019-10-02 8:52 ` Shahaf Shuler
2019-10-02 9:34 ` Jerin Jacob Kollanukkaran
2020-01-27 21:19 ` [dpdk-dev] [PATCH v2] net/regexdev: " Ori Kam
2020-01-28 9:00 ` [dpdk-dev] [PATCH v3] regexdev: " Ori Kam
2020-02-22 16:52 ` Jerin Jacob
2020-02-23 8:41 ` Ori Kam
2020-02-23 9:53 ` Jerin Jacob
2020-02-23 12:33 ` Ori Kam
2020-02-25 5:57 ` Jerin Jacob
2020-02-25 7:48 ` Ori Kam
2020-02-26 9:03 ` Wang Xiang
2020-02-26 8:36 ` Ori Kam
2020-02-27 9:25 ` Wang Xiang
2020-02-27 7:31 ` Ori Kam
2020-02-27 9:16 ` Wang Xiang
2020-02-27 14:40 ` [dpdk-dev] [RFC v4] " Ori Kam
2020-02-27 14:55 ` Jerin Jacob
2020-02-27 15:08 ` [dpdk-dev] [RFC v5] " Ori Kam
2020-03-01 6:13 ` [dpdk-dev] [EXT] " Pavan Nikhilesh Bhagavatula
2020-03-01 7:31 ` Ori Kam
2020-03-01 13:23 ` Pavan Nikhilesh Bhagavatula
2020-03-01 14:10 ` Ori Kam
2020-03-01 14:38 ` Pavan Nikhilesh Bhagavatula
2020-03-01 15:41 ` Ori Kam
2020-03-01 15:57 ` Pavan Nikhilesh Bhagavatula
2020-03-02 7:18 ` Jerin Jacob
2020-03-03 7:06 ` Ori Kam
2020-03-02 7:05 ` [dpdk-dev] " Wang Xiang
2020-03-03 7:44 ` Ori Kam [this message]
2020-03-03 7:54 ` Jerin Jacob
2020-03-10 10:32 ` [dpdk-dev] [RFC v6] " Ori Kam
2020-03-10 13:42 ` Pavan Nikhilesh Bhagavatula
2020-03-10 16:23 ` Ori Kam
2020-03-10 16:36 ` Pavan Nikhilesh Bhagavatula
2020-03-10 17:00 ` Ori Kam
2020-03-12 12:13 ` Ori Kam
2020-03-13 1:20 ` Wang Xiang
2020-03-15 10:05 ` Ori Kam
2020-03-16 1:25 ` Wang Xiang
2020-03-16 9:09 ` Ori Kam
2020-03-16 20:48 ` Wang Xiang
2020-03-16 13:49 ` Ori Kam
2020-03-16 21:10 ` Wang Xiang
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=AM6PR05MB51764668F2C742EB1D99D5F9DBE40@AM6PR05MB5176.eurprd05.prod.outlook.com \
--to=orika@mellanox.com \
--cc=alexr@mellanox.com \
--cc=arthur.su@lionic.com \
--cc=bruce.richardson@intel.com \
--cc=davidfgao@tencent.com \
--cc=deri@ntop.org \
--cc=dev@dpdk.org \
--cc=dovrat@marvell.com \
--cc=fanchenggang@sunyainfo.com \
--cc=fc@napatech.com \
--cc=gu.jian1@zte.com.cn \
--cc=harry.chang@intel.com \
--cc=hemant.agrawal@nxp.com \
--cc=hongjun.ni@intel.com \
--cc=j.bromhead@titan-ic.com \
--cc=jerinj@marvell.com \
--cc=jim@netgate.com \
--cc=liuzhong1@chinaunicom.cn \
--cc=lixingfu@huachentel.com \
--cc=nipun.gupta@nxp.com \
--cc=oc@yunify.com \
--cc=opher@mellanox.com \
--cc=pbhagavatula@marvell.com \
--cc=pkapoor@marvell.com \
--cc=shahafs@mellanox.com \
--cc=shanjiangh@chinatelecom.cn \
--cc=thomas@monjalon.net \
--cc=wushuai@inspur.com \
--cc=xiang.w.wang@intel.com \
--cc=yang.a.hong@intel.com \
--cc=yuyingxia@yxlink.com \
--cc=zhangy.yun@chinatelecom.cn \
--cc=zhaoyong11@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).