[dpdk-dev] [PATCH] examples/l3fwd: prefetch the content of the next packet

[dpdk-dev] [PATCH] examples/l3fwd: prefetch the content of the next packet

[Feifei Wang]

The cache-misses problem is very serious when the function
lpm_cb_parse_ptype is called to read the content of packets. That is
because the contents of packages previously stored in the cache are
overwritten by the following instructions or variables.
Thus the prefetch order can be used to prefetch the next packet into
the cache to avoid CPU spending too much time on it.

On Octeon TX platform with built-in NIC, 12% performance gain was
measured by running RFC2544 NDR test with l3fwd. Furthermore, the
cache-misses event of the function lpm_cb_parse_ptype was reduced by
20%, and the CPU task-clock of it dropped from 16.49% to 11.3%, based
on the forwarding test for one minute with the 64B packet.
On the dpaa2 platform, no performance improvement nor drop were seen
with this patch by running RFC2544 NDR test with l3fwd.
On the x86 platform, 15.7% performance gain was measured by running
RFC2544 NDR test with l3fwd.

Signed-off-by: Feifei Wang <feifei.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
---
 examples/l3fwd/l3fwd_lpm.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/examples/l3fwd/l3fwd_lpm.c b/examples/l3fwd/l3fwd_lpm.c
index 4143683..a3a65f7 100644
--- a/examples/l3fwd/l3fwd_lpm.c
+++ b/examples/l3fwd/l3fwd_lpm.c
@@ -400,10 +400,17 @@ lpm_cb_parse_ptype(uint16_t port __rte_unused, uint16_t queue __rte_unused,
 		   uint16_t max_pkts __rte_unused,
 		   void *user_param __rte_unused)
 {
-	unsigned i;
-
-	for (i = 0; i < nb_pkts; ++i)
+	unsigned int i;
+
+	if (unlikely(nb_pkts == 0))
+		return nb_pkts;
+	rte_prefetch0(rte_pktmbuf_mtod(pkts[0], struct ether_hdr *));
+	for (i = 0; i < (unsigned int) (nb_pkts - 1); ++i) {
+		rte_prefetch0(rte_pktmbuf_mtod(pkts[i+1],
+			struct ether_hdr *));
 		lpm_parse_ptype(pkts[i]);
+	}
+	lpm_parse_ptype(pkts[i]);
 
 	return nb_pkts;
 }
-- 
2.7.4

Re: [dpdk-dev] [PATCH] examples/l3fwd: prefetch the content of the next packet

[Thomas Monjalon]

+Cc Konstantin

14/08/2019 10:54, Feifei Wang:
> The cache-misses problem is very serious when the function
> lpm_cb_parse_ptype is called to read the content of packets. That is
> because the contents of packages previously stored in the cache are
> overwritten by the following instructions or variables.
> Thus the prefetch order can be used to prefetch the next packet into
> the cache to avoid CPU spending too much time on it.
> 
> On Octeon TX platform with built-in NIC, 12% performance gain was
> measured by running RFC2544 NDR test with l3fwd. Furthermore, the
> cache-misses event of the function lpm_cb_parse_ptype was reduced by
> 20%, and the CPU task-clock of it dropped from 16.49% to 11.3%, based
> on the forwarding test for one minute with the 64B packet.
> On the dpaa2 platform, no performance improvement nor drop were seen
> with this patch by running RFC2544 NDR test with l3fwd.
> On the x86 platform, 15.7% performance gain was measured by running
> RFC2544 NDR test with l3fwd.
> 
> Signed-off-by: Feifei Wang <feifei.wang@arm.com>
> Reviewed-by: Gavin Hu <gavin.hu@arm.com>
> Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
> Reviewed-by: Phil Yang <phil.yang@arm.com>

Let's test this "improvement" with 19.11-rc1.
If a drawback is seen, the patch can be reverted.

Applied, thanks