* [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff
@ 2020-12-12 17:41 Lorenzo Bianconi
2020-12-12 17:41 ` [PATCH v3 bpf-next 1/2] net: xdp: introduce xdp_init_buff utility routine Lorenzo Bianconi
` (3 more replies)
0 siblings, 4 replies; 19+ messages in thread
From: Lorenzo Bianconi @ 2020-12-12 17:41 UTC (permalink / raw)
To: bpf, netdev
Cc: davem, kuba, ast, daniel, brouer, lorenzo.bianconi,
alexander.duyck, maciej.fijalkowski, saeed
Introduce xdp_init_buff and xdp_prepare_buff utility routines to initialize
xdp_buff data structure and remove duplicated code in all XDP capable
drivers.
Changes since v2:
- precompute xdp->data as hard_start + headroom and save it in a local
variable to reuse it for xdp->data_end and xdp->data_meta in
xdp_prepare_buff()
Changes since v1:
- introduce xdp_prepare_buff utility routine
Lorenzo Bianconi (2):
net: xdp: introduce xdp_init_buff utility routine
net: xdp: introduce xdp_prepare_buff utility routine
drivers/net/ethernet/amazon/ena/ena_netdev.c | 8 +++-----
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 7 ++-----
.../net/ethernet/cavium/thunder/nicvf_main.c | 11 ++++++-----
.../net/ethernet/freescale/dpaa/dpaa_eth.c | 10 ++++------
.../net/ethernet/freescale/dpaa2/dpaa2-eth.c | 13 +++++--------
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 18 +++++++++---------
drivers/net/ethernet/intel/ice/ice_txrx.c | 17 +++++++++--------
drivers/net/ethernet/intel/igb/igb_main.c | 18 +++++++++---------
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 19 +++++++++----------
.../net/ethernet/intel/ixgbevf/ixgbevf_main.c | 19 +++++++++----------
drivers/net/ethernet/marvell/mvneta.c | 9 +++------
.../net/ethernet/marvell/mvpp2/mvpp2_main.c | 13 +++++++------
drivers/net/ethernet/mellanox/mlx4/en_rx.c | 8 +++-----
.../net/ethernet/mellanox/mlx5/core/en_rx.c | 7 ++-----
.../ethernet/netronome/nfp/nfp_net_common.c | 12 ++++++------
drivers/net/ethernet/qlogic/qede/qede_fp.c | 7 ++-----
drivers/net/ethernet/sfc/rx.c | 9 +++------
drivers/net/ethernet/socionext/netsec.c | 8 +++-----
drivers/net/ethernet/ti/cpsw.c | 17 ++++++-----------
drivers/net/ethernet/ti/cpsw_new.c | 17 ++++++-----------
drivers/net/hyperv/netvsc_bpf.c | 7 ++-----
drivers/net/tun.c | 11 ++++-------
drivers/net/veth.c | 14 +++++---------
drivers/net/virtio_net.c | 18 ++++++------------
drivers/net/xen-netfront.c | 8 +++-----
include/net/xdp.h | 19 +++++++++++++++++++
net/bpf/test_run.c | 9 +++------
net/core/dev.c | 18 ++++++++----------
28 files changed, 156 insertions(+), 195 deletions(-)
--
2.29.2
^ permalink raw reply [flat|nested] 19+ messages in thread
* [PATCH v3 bpf-next 1/2] net: xdp: introduce xdp_init_buff utility routine
2020-12-12 17:41 [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff Lorenzo Bianconi
@ 2020-12-12 17:41 ` Lorenzo Bianconi
2020-12-16 8:35 ` Jesper Dangaard Brouer
2020-12-12 17:41 ` [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff " Lorenzo Bianconi
` (2 subsequent siblings)
3 siblings, 1 reply; 19+ messages in thread
From: Lorenzo Bianconi @ 2020-12-12 17:41 UTC (permalink / raw)
To: bpf, netdev
Cc: davem, kuba, ast, daniel, brouer, lorenzo.bianconi,
alexander.duyck, maciej.fijalkowski, saeed
Introduce xdp_init_buff utility routine to initialize xdp_buff fields
const over NAPI iterations (e.g. frame_sz or rxq pointer). Rely on
xdp_init_buff in all XDP capable drivers.
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
drivers/net/ethernet/amazon/ena/ena_netdev.c | 3 +--
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 3 +--
drivers/net/ethernet/cavium/thunder/nicvf_main.c | 4 ++--
drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 4 ++--
drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 8 ++++----
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 6 +++---
drivers/net/ethernet/intel/ice/ice_txrx.c | 6 +++---
drivers/net/ethernet/intel/igb/igb_main.c | 6 +++---
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 7 +++----
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 7 +++----
drivers/net/ethernet/marvell/mvneta.c | 3 +--
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 8 +++++---
drivers/net/ethernet/mellanox/mlx4/en_rx.c | 3 +--
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 3 +--
drivers/net/ethernet/netronome/nfp/nfp_net_common.c | 4 ++--
drivers/net/ethernet/qlogic/qede/qede_fp.c | 3 +--
drivers/net/ethernet/sfc/rx.c | 3 +--
drivers/net/ethernet/socionext/netsec.c | 3 +--
drivers/net/ethernet/ti/cpsw.c | 4 ++--
drivers/net/ethernet/ti/cpsw_new.c | 4 ++--
drivers/net/hyperv/netvsc_bpf.c | 3 +--
drivers/net/tun.c | 7 +++----
drivers/net/veth.c | 8 ++++----
drivers/net/virtio_net.c | 6 ++----
drivers/net/xen-netfront.c | 4 ++--
include/net/xdp.h | 7 +++++++
net/bpf/test_run.c | 4 ++--
net/core/dev.c | 8 ++++----
28 files changed, 67 insertions(+), 72 deletions(-)
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
index 0e98f45c2b22..338dce73927e 100644
--- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
+++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
@@ -1567,8 +1567,7 @@ static int ena_clean_rx_irq(struct ena_ring *rx_ring, struct napi_struct *napi,
netif_dbg(rx_ring->adapter, rx_status, rx_ring->netdev,
"%s qid %d\n", __func__, rx_ring->qid);
res_budget = budget;
- xdp.rxq = &rx_ring->xdp_rxq;
- xdp.frame_sz = ENA_PAGE_SIZE;
+ xdp_init_buff(&xdp, ENA_PAGE_SIZE, &rx_ring->xdp_rxq);
do {
xdp_verdict = XDP_PASS;
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index fcc262064766..b7942c3440c0 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -133,12 +133,11 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
dma_sync_single_for_cpu(&pdev->dev, mapping + offset, *len, bp->rx_dir);
txr = rxr->bnapi->tx_ring;
+ xdp_init_buff(&xdp, PAGE_SIZE, &rxr->xdp_rxq);
xdp.data_hard_start = *data_ptr - offset;
xdp.data = *data_ptr;
xdp_set_data_meta_invalid(&xdp);
xdp.data_end = *data_ptr + *len;
- xdp.rxq = &rxr->xdp_rxq;
- xdp.frame_sz = PAGE_SIZE; /* BNXT_RX_PAGE_MODE(bp) when XDP enabled */
orig_data = xdp.data;
rcu_read_lock();
diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index f3b7b443f964..9fc672f075f2 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -547,12 +547,12 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog,
cpu_addr = (u64)phys_to_virt(cpu_addr);
page = virt_to_page((void *)cpu_addr);
+ xdp_init_buff(&xdp, RCV_FRAG_LEN + XDP_PACKET_HEADROOM,
+ &rq->xdp_rxq);
xdp.data_hard_start = page_address(page);
xdp.data = (void *)cpu_addr;
xdp_set_data_meta_invalid(&xdp);
xdp.data_end = xdp.data + len;
- xdp.rxq = &rq->xdp_rxq;
- xdp.frame_sz = RCV_FRAG_LEN + XDP_PACKET_HEADROOM;
orig_data = xdp.data;
rcu_read_lock();
diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
index e28510c282e5..93030000e0aa 100644
--- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
+++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
@@ -2536,12 +2536,12 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr,
return XDP_PASS;
}
+ xdp_init_buff(&xdp, DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE,
+ &dpaa_fq->xdp_rxq);
xdp.data = vaddr + fd_off;
xdp.data_meta = xdp.data;
xdp.data_hard_start = xdp.data - XDP_PACKET_HEADROOM;
xdp.data_end = xdp.data + qm_fd_get_length(fd);
- xdp.frame_sz = DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE;
- xdp.rxq = &dpaa_fq->xdp_rxq;
/* We reserve a fixed headroom of 256 bytes under the erratum and we
* offer it all to XDP programs to use. If no room is left for the
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
index 91cff93dbdae..a4ade0b5adb0 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
@@ -358,14 +358,14 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv,
if (!xdp_prog)
goto out;
+ xdp_init_buff(&xdp,
+ DPAA2_ETH_RX_BUF_RAW_SIZE -
+ (dpaa2_fd_get_offset(fd) - XDP_PACKET_HEADROOM),
+ &ch->xdp_rxq);
xdp.data = vaddr + dpaa2_fd_get_offset(fd);
xdp.data_end = xdp.data + dpaa2_fd_get_len(fd);
xdp.data_hard_start = xdp.data - XDP_PACKET_HEADROOM;
xdp_set_data_meta_invalid(&xdp);
- xdp.rxq = &ch->xdp_rxq;
-
- xdp.frame_sz = DPAA2_ETH_RX_BUF_RAW_SIZE -
- (dpaa2_fd_get_offset(fd) - XDP_PACKET_HEADROOM);
xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
index 9f73cd7aee09..4dbbbd49c389 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
@@ -2332,7 +2332,7 @@ static void i40e_inc_ntc(struct i40e_ring *rx_ring)
**/
static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
{
- unsigned int total_rx_bytes = 0, total_rx_packets = 0;
+ unsigned int total_rx_bytes = 0, total_rx_packets = 0, frame_sz = 0;
struct sk_buff *skb = rx_ring->skb;
u16 cleaned_count = I40E_DESC_UNUSED(rx_ring);
unsigned int xdp_xmit = 0;
@@ -2340,9 +2340,9 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
struct xdp_buff xdp;
#if (PAGE_SIZE < 8192)
- xdp.frame_sz = i40e_rx_frame_truesize(rx_ring, 0);
+ frame_sz = i40e_rx_frame_truesize(rx_ring, 0);
#endif
- xdp.rxq = &rx_ring->xdp_rxq;
+ xdp_init_buff(&xdp, frame_sz, &rx_ring->xdp_rxq);
while (likely(total_rx_packets < (unsigned int)budget)) {
struct i40e_rx_buffer *rx_buffer;
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index 77d5eae6b4c2..d52d98d56367 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -1077,18 +1077,18 @@ ice_is_non_eop(struct ice_ring *rx_ring, union ice_32b_rx_flex_desc *rx_desc,
*/
int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
{
- unsigned int total_rx_bytes = 0, total_rx_pkts = 0;
+ unsigned int total_rx_bytes = 0, total_rx_pkts = 0, frame_sz = 0;
u16 cleaned_count = ICE_DESC_UNUSED(rx_ring);
unsigned int xdp_res, xdp_xmit = 0;
struct bpf_prog *xdp_prog = NULL;
struct xdp_buff xdp;
bool failure;
- xdp.rxq = &rx_ring->xdp_rxq;
/* Frame size depend on rx_ring setup when PAGE_SIZE=4K */
#if (PAGE_SIZE < 8192)
- xdp.frame_sz = ice_rx_frame_truesize(rx_ring, 0);
+ frame_sz = ice_rx_frame_truesize(rx_ring, 0);
#endif
+ xdp_init_buff(&xdp, frame_sz, &rx_ring->xdp_rxq);
/* start the loop to process Rx packets bounded by 'budget' */
while (likely(total_rx_pkts < (unsigned int)budget)) {
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 6a4ef4934fcf..365dfc0e3b65 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -8666,13 +8666,13 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget)
u16 cleaned_count = igb_desc_unused(rx_ring);
unsigned int xdp_xmit = 0;
struct xdp_buff xdp;
-
- xdp.rxq = &rx_ring->xdp_rxq;
+ u32 frame_sz = 0;
/* Frame size depend on rx_ring setup when PAGE_SIZE=4K */
#if (PAGE_SIZE < 8192)
- xdp.frame_sz = igb_rx_frame_truesize(rx_ring, 0);
+ frame_sz = igb_rx_frame_truesize(rx_ring, 0);
#endif
+ xdp_init_buff(&xdp, frame_sz, &rx_ring->xdp_rxq);
while (likely(total_packets < budget)) {
union e1000_adv_rx_desc *rx_desc;
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index 50e6b8b6ba7b..dcd49cfa36f7 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -2282,7 +2282,7 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,
struct ixgbe_ring *rx_ring,
const int budget)
{
- unsigned int total_rx_bytes = 0, total_rx_packets = 0;
+ unsigned int total_rx_bytes = 0, total_rx_packets = 0, frame_sz = 0;
struct ixgbe_adapter *adapter = q_vector->adapter;
#ifdef IXGBE_FCOE
int ddp_bytes;
@@ -2292,12 +2292,11 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,
unsigned int xdp_xmit = 0;
struct xdp_buff xdp;
- xdp.rxq = &rx_ring->xdp_rxq;
-
/* Frame size depend on rx_ring setup when PAGE_SIZE=4K */
#if (PAGE_SIZE < 8192)
- xdp.frame_sz = ixgbe_rx_frame_truesize(rx_ring, 0);
+ frame_sz = ixgbe_rx_frame_truesize(rx_ring, 0);
#endif
+ xdp_init_buff(&xdp, frame_sz, &rx_ring->xdp_rxq);
while (likely(total_rx_packets < budget)) {
union ixgbe_adv_rx_desc *rx_desc;
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 4061cd7db5dd..624efcd71569 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -1121,19 +1121,18 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
struct ixgbevf_ring *rx_ring,
int budget)
{
- unsigned int total_rx_bytes = 0, total_rx_packets = 0;
+ unsigned int total_rx_bytes = 0, total_rx_packets = 0, frame_sz = 0;
struct ixgbevf_adapter *adapter = q_vector->adapter;
u16 cleaned_count = ixgbevf_desc_unused(rx_ring);
struct sk_buff *skb = rx_ring->skb;
bool xdp_xmit = false;
struct xdp_buff xdp;
- xdp.rxq = &rx_ring->xdp_rxq;
-
/* Frame size depend on rx_ring setup when PAGE_SIZE=4K */
#if (PAGE_SIZE < 8192)
- xdp.frame_sz = ixgbevf_rx_frame_truesize(rx_ring, 0);
+ frame_sz = ixgbevf_rx_frame_truesize(rx_ring, 0);
#endif
+ xdp_init_buff(&xdp, frame_sz, &rx_ring->xdp_rxq);
while (likely(total_rx_packets < budget)) {
struct ixgbevf_rx_buffer *rx_buffer;
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index 563ceac3060f..acbb9cb85ada 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -2363,9 +2363,8 @@ static int mvneta_rx_swbm(struct napi_struct *napi,
u32 desc_status, frame_sz;
struct xdp_buff xdp_buf;
+ xdp_init_buff(&xdp_buf, PAGE_SIZE, &rxq->xdp_rxq);
xdp_buf.data_hard_start = NULL;
- xdp_buf.frame_sz = PAGE_SIZE;
- xdp_buf.rxq = &rxq->xdp_rxq;
sinfo.nr_frags = 0;
diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
index afdd22827223..ca05dfc05058 100644
--- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
+++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
@@ -3562,16 +3562,18 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
frag_size = bm_pool->frag_size;
if (xdp_prog) {
+ struct xdp_rxq_info *xdp_rxq;
+
xdp.data_hard_start = data;
xdp.data = data + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM;
xdp.data_end = xdp.data + rx_bytes;
- xdp.frame_sz = PAGE_SIZE;
if (bm_pool->pkt_size == MVPP2_BM_SHORT_PKT_SIZE)
- xdp.rxq = &rxq->xdp_rxq_short;
+ xdp_rxq = &rxq->xdp_rxq_short;
else
- xdp.rxq = &rxq->xdp_rxq_long;
+ xdp_rxq = &rxq->xdp_rxq_long;
+ xdp_init_buff(&xdp, PAGE_SIZE, xdp_rxq);
xdp_set_data_meta_invalid(&xdp);
ret = mvpp2_run_xdp(port, rxq, xdp_prog, &xdp, pp, &ps);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index 7954c1daf2b6..815381b484ca 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -682,8 +682,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud
/* Protect accesses to: ring->xdp_prog, priv->mac_hash list */
rcu_read_lock();
xdp_prog = rcu_dereference(ring->xdp_prog);
- xdp.rxq = &ring->xdp_rxq;
- xdp.frame_sz = priv->frag_info[0].frag_stride;
+ xdp_init_buff(&xdp, priv->frag_info[0].frag_stride, &ring->xdp_rxq);
doorbell_pending = false;
/* We assume a 1:1 mapping between CQEs and Rx descriptors, so Rx
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index 6628a0197b4e..c68628b1f30b 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -1127,12 +1127,11 @@ struct sk_buff *mlx5e_build_linear_skb(struct mlx5e_rq *rq, void *va,
static void mlx5e_fill_xdp_buff(struct mlx5e_rq *rq, void *va, u16 headroom,
u32 len, struct xdp_buff *xdp)
{
+ xdp_init_buff(xdp, rq->buff.frame0_sz, &rq->xdp_rxq);
xdp->data_hard_start = va;
xdp->data = va + headroom;
xdp_set_data_meta_invalid(xdp);
xdp->data_end = xdp->data + len;
- xdp->rxq = &rq->xdp_rxq;
- xdp->frame_sz = rq->buff.frame0_sz;
}
static struct sk_buff *
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
index b4acf2f41e84..68e03e8257f2 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
@@ -1822,8 +1822,8 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget)
rcu_read_lock();
xdp_prog = READ_ONCE(dp->xdp_prog);
true_bufsz = xdp_prog ? PAGE_SIZE : dp->fl_bufsz;
- xdp.frame_sz = PAGE_SIZE - NFP_NET_RX_BUF_HEADROOM;
- xdp.rxq = &rx_ring->xdp_rxq;
+ xdp_init_buff(&xdp, PAGE_SIZE - NFP_NET_RX_BUF_HEADROOM,
+ &rx_ring->xdp_rxq);
tx_ring = r_vec->xdp_ring;
while (pkts_polled < budget) {
diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c
index a2494bf85007..d40220043883 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_fp.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c
@@ -1090,12 +1090,11 @@ static bool qede_rx_xdp(struct qede_dev *edev,
struct xdp_buff xdp;
enum xdp_action act;
+ xdp_init_buff(&xdp, rxq->rx_buf_seg_size, &rxq->xdp_rxq);
xdp.data_hard_start = page_address(bd->data);
xdp.data = xdp.data_hard_start + *data_offset;
xdp_set_data_meta_invalid(&xdp);
xdp.data_end = xdp.data + *len;
- xdp.rxq = &rxq->xdp_rxq;
- xdp.frame_sz = rxq->rx_buf_seg_size; /* PAGE_SIZE when XDP enabled */
/* Queues always have a full reset currently, so for the time
* being until there's atomic program replace just mark read
diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c
index aaa112877561..eaa6650955d1 100644
--- a/drivers/net/ethernet/sfc/rx.c
+++ b/drivers/net/ethernet/sfc/rx.c
@@ -293,14 +293,13 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel,
memcpy(rx_prefix, *ehp - efx->rx_prefix_size,
efx->rx_prefix_size);
+ xdp_init_buff(&xdp, efx->rx_page_buf_step, &rx_queue->xdp_rxq_info);
xdp.data = *ehp;
xdp.data_hard_start = xdp.data - EFX_XDP_HEADROOM;
/* No support yet for XDP metadata */
xdp_set_data_meta_invalid(&xdp);
xdp.data_end = xdp.data + rx_buf->len;
- xdp.rxq = &rx_queue->xdp_rxq_info;
- xdp.frame_sz = efx->rx_page_buf_step;
xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
rcu_read_unlock();
diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c
index 19d20a6d0d44..945ca9517bf9 100644
--- a/drivers/net/ethernet/socionext/netsec.c
+++ b/drivers/net/ethernet/socionext/netsec.c
@@ -956,8 +956,7 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget)
u32 xdp_act = 0;
int done = 0;
- xdp.rxq = &dring->xdp_rxq;
- xdp.frame_sz = PAGE_SIZE;
+ xdp_init_buff(&xdp, PAGE_SIZE, &dring->xdp_rxq);
rcu_read_lock();
xdp_prog = READ_ONCE(priv->xdp_prog);
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index b0f00b4edd94..78a923391828 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -392,6 +392,8 @@ static void cpsw_rx_handler(void *token, int len, int status)
}
if (priv->xdp_prog) {
+ xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
+
if (status & CPDMA_RX_VLAN_ENCAP) {
xdp.data = pa + CPSW_HEADROOM +
CPSW_RX_VLAN_ENCAP_HDR_SIZE;
@@ -405,8 +407,6 @@ static void cpsw_rx_handler(void *token, int len, int status)
xdp_set_data_meta_invalid(&xdp);
xdp.data_hard_start = pa;
- xdp.rxq = &priv->xdp_rxq[ch];
- xdp.frame_sz = PAGE_SIZE;
port = priv->emac_port + cpsw->data.dual_emac;
ret = cpsw_run_xdp(priv, ch, &xdp, page, port);
diff --git a/drivers/net/ethernet/ti/cpsw_new.c b/drivers/net/ethernet/ti/cpsw_new.c
index 2f5e0ad23ad7..1b3385ec9645 100644
--- a/drivers/net/ethernet/ti/cpsw_new.c
+++ b/drivers/net/ethernet/ti/cpsw_new.c
@@ -335,6 +335,8 @@ static void cpsw_rx_handler(void *token, int len, int status)
}
if (priv->xdp_prog) {
+ xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
+
if (status & CPDMA_RX_VLAN_ENCAP) {
xdp.data = pa + CPSW_HEADROOM +
CPSW_RX_VLAN_ENCAP_HDR_SIZE;
@@ -348,8 +350,6 @@ static void cpsw_rx_handler(void *token, int len, int status)
xdp_set_data_meta_invalid(&xdp);
xdp.data_hard_start = pa;
- xdp.rxq = &priv->xdp_rxq[ch];
- xdp.frame_sz = PAGE_SIZE;
ret = cpsw_run_xdp(priv, ch, &xdp, page, priv->emac_port);
if (ret != CPSW_XDP_PASS)
diff --git a/drivers/net/hyperv/netvsc_bpf.c b/drivers/net/hyperv/netvsc_bpf.c
index 440486d9c999..14a7ee4c6899 100644
--- a/drivers/net/hyperv/netvsc_bpf.c
+++ b/drivers/net/hyperv/netvsc_bpf.c
@@ -44,12 +44,11 @@ u32 netvsc_run_xdp(struct net_device *ndev, struct netvsc_channel *nvchan,
goto out;
}
+ xdp_init_buff(xdp, PAGE_SIZE, &nvchan->xdp_rxq);
xdp->data_hard_start = page_address(page);
xdp->data = xdp->data_hard_start + NETVSC_XDP_HDRM;
xdp_set_data_meta_invalid(xdp);
xdp->data_end = xdp->data + len;
- xdp->rxq = &nvchan->xdp_rxq;
- xdp->frame_sz = PAGE_SIZE;
memcpy(xdp->data, data, len);
diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index fbed05ae7b0f..a82f7823d428 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -1599,12 +1599,11 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun,
struct xdp_buff xdp;
u32 act;
+ xdp_init_buff(&xdp, buflen, &tfile->xdp_rxq);
xdp.data_hard_start = buf;
xdp.data = buf + pad;
xdp_set_data_meta_invalid(&xdp);
xdp.data_end = xdp.data + len;
- xdp.rxq = &tfile->xdp_rxq;
- xdp.frame_sz = buflen;
act = bpf_prog_run_xdp(xdp_prog, &xdp);
if (act == XDP_REDIRECT || act == XDP_TX) {
@@ -2344,9 +2343,9 @@ static int tun_xdp_one(struct tun_struct *tun,
skb_xdp = true;
goto build;
}
+
+ xdp_init_buff(xdp, buflen, &tfile->xdp_rxq);
xdp_set_data_meta_invalid(xdp);
- xdp->rxq = &tfile->xdp_rxq;
- xdp->frame_sz = buflen;
act = bpf_prog_run_xdp(xdp_prog, xdp);
err = tun_xdp_act(tun, xdp_prog, xdp, act);
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 02bfcdf50a7a..25f3601fb6dd 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -654,7 +654,7 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq,
struct veth_xdp_tx_bq *bq,
struct veth_stats *stats)
{
- u32 pktlen, headroom, act, metalen;
+ u32 pktlen, headroom, act, metalen, frame_sz;
void *orig_data, *orig_data_end;
struct bpf_prog *xdp_prog;
int mac_len, delta, off;
@@ -714,11 +714,11 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq,
xdp.data = skb_mac_header(skb);
xdp.data_end = xdp.data + pktlen;
xdp.data_meta = xdp.data;
- xdp.rxq = &rq->xdp_rxq;
/* SKB "head" area always have tailroom for skb_shared_info */
- xdp.frame_sz = (void *)skb_end_pointer(skb) - xdp.data_hard_start;
- xdp.frame_sz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+ frame_sz = (void *)skb_end_pointer(skb) - xdp.data_hard_start;
+ frame_sz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+ xdp_init_buff(&xdp, frame_sz, &rq->xdp_rxq);
orig_data = xdp.data;
orig_data_end = xdp.data_end;
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 052975ea0af4..a22ce87bcd9c 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -689,12 +689,11 @@ static struct sk_buff *receive_small(struct net_device *dev,
page = xdp_page;
}
+ xdp_init_buff(&xdp, buflen, &rq->xdp_rxq);
xdp.data_hard_start = buf + VIRTNET_RX_PAD + vi->hdr_len;
xdp.data = xdp.data_hard_start + xdp_headroom;
xdp.data_end = xdp.data + len;
xdp.data_meta = xdp.data;
- xdp.rxq = &rq->xdp_rxq;
- xdp.frame_sz = buflen;
orig_data = xdp.data;
act = bpf_prog_run_xdp(xdp_prog, &xdp);
stats->xdp_packets++;
@@ -859,12 +858,11 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
* the descriptor on if we get an XDP_TX return code.
*/
data = page_address(xdp_page) + offset;
+ xdp_init_buff(&xdp, frame_sz - vi->hdr_len, &rq->xdp_rxq);
xdp.data_hard_start = data - VIRTIO_XDP_HEADROOM + vi->hdr_len;
xdp.data = data + vi->hdr_len;
xdp.data_end = xdp.data + (len - vi->hdr_len);
xdp.data_meta = xdp.data;
- xdp.rxq = &rq->xdp_rxq;
- xdp.frame_sz = frame_sz - vi->hdr_len;
act = bpf_prog_run_xdp(xdp_prog, &xdp);
stats->xdp_packets++;
diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index b01848ef4649..329397c60d84 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -864,12 +864,12 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
u32 act;
int err;
+ xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
+ &queue->xdp_rxq);
xdp->data_hard_start = page_address(pdata);
xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
xdp_set_data_meta_invalid(xdp);
xdp->data_end = xdp->data + len;
- xdp->rxq = &queue->xdp_rxq;
- xdp->frame_sz = XEN_PAGE_SIZE - XDP_PACKET_HEADROOM;
act = bpf_prog_run_xdp(prog, xdp);
switch (act) {
diff --git a/include/net/xdp.h b/include/net/xdp.h
index 700ad5db7f5d..3fb3a9aa1b71 100644
--- a/include/net/xdp.h
+++ b/include/net/xdp.h
@@ -76,6 +76,13 @@ struct xdp_buff {
u32 frame_sz; /* frame size to deduce data_hard_end/reserved tailroom*/
};
+static inline void
+xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
+{
+ xdp->frame_sz = frame_sz;
+ xdp->rxq = rxq;
+}
+
/* Reserve memory area at end-of data area.
*
* This macro reserves tailroom in the XDP buffer by limiting the
diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
index c1c30a9f76f3..a8fa5a9e4137 100644
--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
@@ -640,10 +640,10 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr,
xdp.data = data + headroom;
xdp.data_meta = xdp.data;
xdp.data_end = xdp.data + size;
- xdp.frame_sz = headroom + max_data_sz + tailroom;
rxqueue = __netif_get_rx_queue(current->nsproxy->net_ns->loopback_dev, 0);
- xdp.rxq = &rxqueue->xdp_rxq;
+ xdp_init_buff(&xdp, headroom + max_data_sz + tailroom,
+ &rxqueue->xdp_rxq);
bpf_prog_change_xdp(NULL, prog);
ret = bpf_test_run(prog, &xdp, repeat, &retval, &duration, true);
if (ret)
diff --git a/net/core/dev.c b/net/core/dev.c
index ce8fea2e2788..bac56afcf6bc 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4588,11 +4588,11 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
struct netdev_rx_queue *rxqueue;
void *orig_data, *orig_data_end;
u32 metalen, act = XDP_DROP;
+ u32 mac_len, frame_sz;
__be16 orig_eth_type;
struct ethhdr *eth;
bool orig_bcast;
int hlen, off;
- u32 mac_len;
/* Reinjected packets coming from act_mirred or similar should
* not get XDP generic processing.
@@ -4631,8 +4631,8 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
xdp->data_hard_start = skb->data - skb_headroom(skb);
/* SKB "head" area always have tailroom for skb_shared_info */
- xdp->frame_sz = (void *)skb_end_pointer(skb) - xdp->data_hard_start;
- xdp->frame_sz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+ frame_sz = (void *)skb_end_pointer(skb) - xdp->data_hard_start;
+ frame_sz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
orig_data_end = xdp->data_end;
orig_data = xdp->data;
@@ -4641,7 +4641,7 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
orig_eth_type = eth->h_proto;
rxqueue = netif_get_rxqueue(skb);
- xdp->rxq = &rxqueue->xdp_rxq;
+ xdp_init_buff(xdp, frame_sz, &rxqueue->xdp_rxq);
act = bpf_prog_run_xdp(xdp_prog, xdp);
--
2.29.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-12 17:41 [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff Lorenzo Bianconi
2020-12-12 17:41 ` [PATCH v3 bpf-next 1/2] net: xdp: introduce xdp_init_buff utility routine Lorenzo Bianconi
@ 2020-12-12 17:41 ` Lorenzo Bianconi
2020-12-15 12:36 ` Maciej Fijalkowski
2020-12-14 15:32 ` [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff Martin Habets
2020-12-14 17:53 ` Camelia Alexandra Groza
3 siblings, 1 reply; 19+ messages in thread
From: Lorenzo Bianconi @ 2020-12-12 17:41 UTC (permalink / raw)
To: bpf, netdev
Cc: davem, kuba, ast, daniel, brouer, lorenzo.bianconi,
alexander.duyck, maciej.fijalkowski, saeed
Introduce xdp_prepare_buff utility routine to initialize per-descriptor
xdp_buff fields (e.g. xdp_buff pointers). Rely on xdp_prepare_buff() in
all XDP capable drivers.
Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
---
drivers/net/ethernet/amazon/ena/ena_netdev.c | 5 ++---
drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 4 +---
drivers/net/ethernet/cavium/thunder/nicvf_main.c | 7 ++++---
drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 6 ++----
drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 13 +++++--------
drivers/net/ethernet/intel/i40e/i40e_txrx.c | 12 ++++++------
drivers/net/ethernet/intel/ice/ice_txrx.c | 11 ++++++-----
drivers/net/ethernet/intel/igb/igb_main.c | 12 ++++++------
drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 12 ++++++------
drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 12 ++++++------
drivers/net/ethernet/marvell/mvneta.c | 6 ++----
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 7 +++----
drivers/net/ethernet/mellanox/mlx4/en_rx.c | 5 ++---
drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 4 +---
.../net/ethernet/netronome/nfp/nfp_net_common.c | 8 ++++----
drivers/net/ethernet/qlogic/qede/qede_fp.c | 4 +---
drivers/net/ethernet/sfc/rx.c | 6 ++----
drivers/net/ethernet/socionext/netsec.c | 5 ++---
drivers/net/ethernet/ti/cpsw.c | 15 +++++----------
drivers/net/ethernet/ti/cpsw_new.c | 15 +++++----------
drivers/net/hyperv/netvsc_bpf.c | 4 +---
drivers/net/tun.c | 4 +---
drivers/net/veth.c | 6 +-----
drivers/net/virtio_net.c | 12 ++++--------
drivers/net/xen-netfront.c | 4 +---
include/net/xdp.h | 12 ++++++++++++
net/bpf/test_run.c | 5 +----
net/core/dev.c | 10 ++++------
28 files changed, 96 insertions(+), 130 deletions(-)
diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
index 338dce73927e..1cfd0c98677e 100644
--- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
+++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
@@ -1519,10 +1519,9 @@ static int ena_xdp_handle_buff(struct ena_ring *rx_ring, struct xdp_buff *xdp)
int ret;
rx_info = &rx_ring->rx_buffer_info[rx_ring->ena_bufs[0].req_id];
- xdp->data = page_address(rx_info->page) + rx_info->page_offset;
+ xdp_prepare_buff(xdp, page_address(rx_info->page),
+ rx_info->page_offset, rx_ring->ena_bufs[0].len);
xdp_set_data_meta_invalid(xdp);
- xdp->data_hard_start = page_address(rx_info->page);
- xdp->data_end = xdp->data + rx_ring->ena_bufs[0].len;
/* If for some reason we received a bigger packet than
* we expect, then we simply drop it
*/
diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
index b7942c3440c0..e1664b86a7b8 100644
--- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
+++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
@@ -134,10 +134,8 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
txr = rxr->bnapi->tx_ring;
xdp_init_buff(&xdp, PAGE_SIZE, &rxr->xdp_rxq);
- xdp.data_hard_start = *data_ptr - offset;
- xdp.data = *data_ptr;
+ xdp_prepare_buff(&xdp, *data_ptr - offset, offset, *len);
xdp_set_data_meta_invalid(&xdp);
- xdp.data_end = *data_ptr + *len;
orig_data = xdp.data;
rcu_read_lock();
diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
index 9fc672f075f2..9bdac04359c6 100644
--- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
+++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
@@ -530,6 +530,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog,
struct cqe_rx_t *cqe_rx, struct snd_queue *sq,
struct rcv_queue *rq, struct sk_buff **skb)
{
+ unsigned char *hard_start, *data;
struct xdp_buff xdp;
struct page *page;
u32 action;
@@ -549,10 +550,10 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog,
xdp_init_buff(&xdp, RCV_FRAG_LEN + XDP_PACKET_HEADROOM,
&rq->xdp_rxq);
- xdp.data_hard_start = page_address(page);
- xdp.data = (void *)cpu_addr;
+ hard_start = page_address(page);
+ data = (unsigned char *)cpu_addr;
+ xdp_prepare_buff(&xdp, hard_start, data - hard_start, len);
xdp_set_data_meta_invalid(&xdp);
- xdp.data_end = xdp.data + len;
orig_data = xdp.data;
rcu_read_lock();
diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
index 93030000e0aa..86ee07c90154 100644
--- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
+++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
@@ -2538,10 +2538,8 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr,
xdp_init_buff(&xdp, DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE,
&dpaa_fq->xdp_rxq);
- xdp.data = vaddr + fd_off;
- xdp.data_meta = xdp.data;
- xdp.data_hard_start = xdp.data - XDP_PACKET_HEADROOM;
- xdp.data_end = xdp.data + qm_fd_get_length(fd);
+ xdp_prepare_buff(&xdp, vaddr + fd_off - XDP_PACKET_HEADROOM,
+ XDP_PACKET_HEADROOM, qm_fd_get_length(fd));
/* We reserve a fixed headroom of 256 bytes under the erratum and we
* offer it all to XDP programs to use. If no room is left for the
diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
index a4ade0b5adb0..12358f5d59d6 100644
--- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
+++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
@@ -350,7 +350,7 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv,
struct bpf_prog *xdp_prog;
struct xdp_buff xdp;
u32 xdp_act = XDP_PASS;
- int err;
+ int err, offset;
rcu_read_lock();
@@ -358,13 +358,10 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv,
if (!xdp_prog)
goto out;
- xdp_init_buff(&xdp,
- DPAA2_ETH_RX_BUF_RAW_SIZE -
- (dpaa2_fd_get_offset(fd) - XDP_PACKET_HEADROOM),
- &ch->xdp_rxq);
- xdp.data = vaddr + dpaa2_fd_get_offset(fd);
- xdp.data_end = xdp.data + dpaa2_fd_get_len(fd);
- xdp.data_hard_start = xdp.data - XDP_PACKET_HEADROOM;
+ offset = dpaa2_fd_get_offset(fd) - XDP_PACKET_HEADROOM;
+ xdp_init_buff(&xdp, DPAA2_ETH_RX_BUF_RAW_SIZE - offset, &ch->xdp_rxq);
+ xdp_prepare_buff(&xdp, vaddr + offset, XDP_PACKET_HEADROOM,
+ dpaa2_fd_get_len(fd));
xdp_set_data_meta_invalid(&xdp);
xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
index 4dbbbd49c389..fcd1ca3343fb 100644
--- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
+++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
@@ -2393,12 +2393,12 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
/* retrieve a buffer from the ring */
if (!skb) {
- xdp.data = page_address(rx_buffer->page) +
- rx_buffer->page_offset;
- xdp.data_meta = xdp.data;
- xdp.data_hard_start = xdp.data -
- i40e_rx_offset(rx_ring);
- xdp.data_end = xdp.data + size;
+ unsigned int offset = i40e_rx_offset(rx_ring);
+ unsigned char *hard_start;
+
+ hard_start = page_address(rx_buffer->page) +
+ rx_buffer->page_offset - offset;
+ xdp_prepare_buff(&xdp, hard_start, offset, size);
#if (PAGE_SIZE > 4096)
/* At larger PAGE_SIZE, frame_sz depend on len size */
xdp.frame_sz = i40e_rx_frame_truesize(rx_ring, size);
diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
index d52d98d56367..a7a00060f520 100644
--- a/drivers/net/ethernet/intel/ice/ice_txrx.c
+++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
@@ -1094,8 +1094,9 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
while (likely(total_rx_pkts < (unsigned int)budget)) {
union ice_32b_rx_flex_desc *rx_desc;
struct ice_rx_buf *rx_buf;
+ unsigned int size, offset;
+ unsigned char *hard_start;
struct sk_buff *skb;
- unsigned int size;
u16 stat_err_bits;
u16 vlan_tag = 0;
u8 rx_ptype;
@@ -1138,10 +1139,10 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
goto construct_skb;
}
- xdp.data = page_address(rx_buf->page) + rx_buf->page_offset;
- xdp.data_hard_start = xdp.data - ice_rx_offset(rx_ring);
- xdp.data_meta = xdp.data;
- xdp.data_end = xdp.data + size;
+ offset = ice_rx_offset(rx_ring);
+ hard_start = page_address(rx_buf->page) + rx_buf->page_offset -
+ offset;
+ xdp_prepare_buff(&xdp, hard_start, offset, size);
#if (PAGE_SIZE > 4096)
/* At larger PAGE_SIZE, frame_sz depend on len size */
xdp.frame_sz = ice_rx_frame_truesize(rx_ring, size);
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 365dfc0e3b65..070b2bb4e9ca 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -8700,12 +8700,12 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget)
/* retrieve a buffer from the ring */
if (!skb) {
- xdp.data = page_address(rx_buffer->page) +
- rx_buffer->page_offset;
- xdp.data_meta = xdp.data;
- xdp.data_hard_start = xdp.data -
- igb_rx_offset(rx_ring);
- xdp.data_end = xdp.data + size;
+ unsigned int offset = igb_rx_offset(rx_ring);
+ unsigned char *hard_start;
+
+ hard_start = page_address(rx_buffer->page) +
+ rx_buffer->page_offset - offset;
+ xdp_prepare_buff(&xdp, hard_start, offset, size);
#if (PAGE_SIZE > 4096)
/* At larger PAGE_SIZE, frame_sz depend on len size */
xdp.frame_sz = igb_rx_frame_truesize(rx_ring, size);
diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
index dcd49cfa36f7..e34054433c7a 100644
--- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
+++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
@@ -2325,12 +2325,12 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,
/* retrieve a buffer from the ring */
if (!skb) {
- xdp.data = page_address(rx_buffer->page) +
- rx_buffer->page_offset;
- xdp.data_meta = xdp.data;
- xdp.data_hard_start = xdp.data -
- ixgbe_rx_offset(rx_ring);
- xdp.data_end = xdp.data + size;
+ unsigned int offset = ixgbe_rx_offset(rx_ring);
+ unsigned char *hard_start;
+
+ hard_start = page_address(rx_buffer->page) +
+ rx_buffer->page_offset - offset;
+ xdp_prepare_buff(&xdp, hard_start, offset, size);
#if (PAGE_SIZE > 4096)
/* At larger PAGE_SIZE, frame_sz depend on len size */
xdp.frame_sz = ixgbe_rx_frame_truesize(rx_ring, size);
diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
index 624efcd71569..51df79005ccb 100644
--- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
+++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
@@ -1160,12 +1160,12 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
/* retrieve a buffer from the ring */
if (!skb) {
- xdp.data = page_address(rx_buffer->page) +
- rx_buffer->page_offset;
- xdp.data_meta = xdp.data;
- xdp.data_hard_start = xdp.data -
- ixgbevf_rx_offset(rx_ring);
- xdp.data_end = xdp.data + size;
+ unsigned int offset = ixgbevf_rx_offset(rx_ring);
+ unsigned char *hard_start;
+
+ hard_start = page_address(rx_buffer->page) +
+ rx_buffer->page_offset - offset;
+ xdp_prepare_buff(&xdp, hard_start, offset, size);
#if (PAGE_SIZE > 4096)
/* At larger PAGE_SIZE, frame_sz depend on len size */
xdp.frame_sz = ixgbevf_rx_frame_truesize(rx_ring, size);
diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
index acbb9cb85ada..af6c9cf59809 100644
--- a/drivers/net/ethernet/marvell/mvneta.c
+++ b/drivers/net/ethernet/marvell/mvneta.c
@@ -2263,10 +2263,8 @@ mvneta_swbm_rx_frame(struct mvneta_port *pp,
/* Prefetch header */
prefetch(data);
-
- xdp->data_hard_start = data;
- xdp->data = data + pp->rx_offset_correction + MVNETA_MH_SIZE;
- xdp->data_end = xdp->data + data_len;
+ xdp_prepare_buff(xdp, data, pp->rx_offset_correction + MVNETA_MH_SIZE,
+ data_len);
xdp_set_data_meta_invalid(xdp);
sinfo = xdp_get_shared_info_from_buff(xdp);
diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
index ca05dfc05058..8c2197b96515 100644
--- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
+++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
@@ -3564,16 +3564,15 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
if (xdp_prog) {
struct xdp_rxq_info *xdp_rxq;
- xdp.data_hard_start = data;
- xdp.data = data + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM;
- xdp.data_end = xdp.data + rx_bytes;
-
if (bm_pool->pkt_size == MVPP2_BM_SHORT_PKT_SIZE)
xdp_rxq = &rxq->xdp_rxq_short;
else
xdp_rxq = &rxq->xdp_rxq_long;
xdp_init_buff(&xdp, PAGE_SIZE, xdp_rxq);
+ xdp_prepare_buff(&xdp, data,
+ MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM,
+ rx_bytes);
xdp_set_data_meta_invalid(&xdp);
ret = mvpp2_run_xdp(port, rxq, xdp_prog, &xdp, pp, &ps);
diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
index 815381b484ca..86c63dedc689 100644
--- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
@@ -776,10 +776,9 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud
priv->frag_info[0].frag_size,
DMA_FROM_DEVICE);
- xdp.data_hard_start = va - frags[0].page_offset;
- xdp.data = va;
+ xdp_prepare_buff(&xdp, va - frags[0].page_offset,
+ frags[0].page_offset, length);
xdp_set_data_meta_invalid(&xdp);
- xdp.data_end = xdp.data + length;
orig_data = xdp.data;
act = bpf_prog_run_xdp(xdp_prog, &xdp);
diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
index c68628b1f30b..a2f4f0ce427f 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
@@ -1128,10 +1128,8 @@ static void mlx5e_fill_xdp_buff(struct mlx5e_rq *rq, void *va, u16 headroom,
u32 len, struct xdp_buff *xdp)
{
xdp_init_buff(xdp, rq->buff.frame0_sz, &rq->xdp_rxq);
- xdp->data_hard_start = va;
- xdp->data = va + headroom;
+ xdp_prepare_buff(xdp, va, headroom, len);
xdp_set_data_meta_invalid(xdp);
- xdp->data_end = xdp->data + len;
}
static struct sk_buff *
diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
index 68e03e8257f2..5d0046c24b8c 100644
--- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
+++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
@@ -1914,10 +1914,10 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget)
unsigned int dma_off;
int act;
- xdp.data_hard_start = rxbuf->frag + NFP_NET_RX_BUF_HEADROOM;
- xdp.data = orig_data;
- xdp.data_meta = orig_data;
- xdp.data_end = orig_data + pkt_len;
+ xdp_prepare_buff(&xdp,
+ rxbuf->frag + NFP_NET_RX_BUF_HEADROOM,
+ pkt_off - NFP_NET_RX_BUF_HEADROOM,
+ pkt_len);
act = bpf_prog_run_xdp(xdp_prog, &xdp);
diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c
index d40220043883..9c50df499046 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_fp.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c
@@ -1091,10 +1091,8 @@ static bool qede_rx_xdp(struct qede_dev *edev,
enum xdp_action act;
xdp_init_buff(&xdp, rxq->rx_buf_seg_size, &rxq->xdp_rxq);
- xdp.data_hard_start = page_address(bd->data);
- xdp.data = xdp.data_hard_start + *data_offset;
+ xdp_prepare_buff(&xdp, page_address(bd->data), *data_offset, *len);
xdp_set_data_meta_invalid(&xdp);
- xdp.data_end = xdp.data + *len;
/* Queues always have a full reset currently, so for the time
* being until there's atomic program replace just mark read
diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c
index eaa6650955d1..9015a1639234 100644
--- a/drivers/net/ethernet/sfc/rx.c
+++ b/drivers/net/ethernet/sfc/rx.c
@@ -294,12 +294,10 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel,
efx->rx_prefix_size);
xdp_init_buff(&xdp, efx->rx_page_buf_step, &rx_queue->xdp_rxq_info);
- xdp.data = *ehp;
- xdp.data_hard_start = xdp.data - EFX_XDP_HEADROOM;
-
+ xdp_prepare_buff(&xdp, *ehp - EFX_XDP_HEADROOM, EFX_XDP_HEADROOM,
+ rx_buf->len);
/* No support yet for XDP metadata */
xdp_set_data_meta_invalid(&xdp);
- xdp.data_end = xdp.data + rx_buf->len;
xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
rcu_read_unlock();
diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c
index 945ca9517bf9..80bb1a6612b1 100644
--- a/drivers/net/ethernet/socionext/netsec.c
+++ b/drivers/net/ethernet/socionext/netsec.c
@@ -1015,10 +1015,9 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget)
dma_dir);
prefetch(desc->addr);
- xdp.data_hard_start = desc->addr;
- xdp.data = desc->addr + NETSEC_RXBUF_HEADROOM;
+ xdp_prepare_buff(&xdp, desc->addr, NETSEC_RXBUF_HEADROOM,
+ pkt_len);
xdp_set_data_meta_invalid(&xdp);
- xdp.data_end = xdp.data + pkt_len;
if (xdp_prog) {
xdp_result = netsec_run_xdp(priv, xdp_prog, &xdp);
diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
index 78a923391828..c08fd6a6be9b 100644
--- a/drivers/net/ethernet/ti/cpsw.c
+++ b/drivers/net/ethernet/ti/cpsw.c
@@ -392,22 +392,17 @@ static void cpsw_rx_handler(void *token, int len, int status)
}
if (priv->xdp_prog) {
- xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
+ int headroom = CPSW_HEADROOM, size = len;
+ xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
if (status & CPDMA_RX_VLAN_ENCAP) {
- xdp.data = pa + CPSW_HEADROOM +
- CPSW_RX_VLAN_ENCAP_HDR_SIZE;
- xdp.data_end = xdp.data + len -
- CPSW_RX_VLAN_ENCAP_HDR_SIZE;
- } else {
- xdp.data = pa + CPSW_HEADROOM;
- xdp.data_end = xdp.data + len;
+ headroom += CPSW_RX_VLAN_ENCAP_HDR_SIZE;
+ size -= CPSW_RX_VLAN_ENCAP_HDR_SIZE;
}
+ xdp_prepare_buff(&xdp, pa, headroom, size);
xdp_set_data_meta_invalid(&xdp);
- xdp.data_hard_start = pa;
-
port = priv->emac_port + cpsw->data.dual_emac;
ret = cpsw_run_xdp(priv, ch, &xdp, page, port);
if (ret != CPSW_XDP_PASS)
diff --git a/drivers/net/ethernet/ti/cpsw_new.c b/drivers/net/ethernet/ti/cpsw_new.c
index 1b3385ec9645..c74c997d1cf2 100644
--- a/drivers/net/ethernet/ti/cpsw_new.c
+++ b/drivers/net/ethernet/ti/cpsw_new.c
@@ -335,22 +335,17 @@ static void cpsw_rx_handler(void *token, int len, int status)
}
if (priv->xdp_prog) {
- xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
+ int headroom = CPSW_HEADROOM, size = len;
+ xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
if (status & CPDMA_RX_VLAN_ENCAP) {
- xdp.data = pa + CPSW_HEADROOM +
- CPSW_RX_VLAN_ENCAP_HDR_SIZE;
- xdp.data_end = xdp.data + len -
- CPSW_RX_VLAN_ENCAP_HDR_SIZE;
- } else {
- xdp.data = pa + CPSW_HEADROOM;
- xdp.data_end = xdp.data + len;
+ headroom += CPSW_RX_VLAN_ENCAP_HDR_SIZE;
+ size -= CPSW_RX_VLAN_ENCAP_HDR_SIZE;
}
+ xdp_prepare_buff(&xdp, pa, headroom, size);
xdp_set_data_meta_invalid(&xdp);
- xdp.data_hard_start = pa;
-
ret = cpsw_run_xdp(priv, ch, &xdp, page, priv->emac_port);
if (ret != CPSW_XDP_PASS)
goto requeue;
diff --git a/drivers/net/hyperv/netvsc_bpf.c b/drivers/net/hyperv/netvsc_bpf.c
index 14a7ee4c6899..93c202d6aff5 100644
--- a/drivers/net/hyperv/netvsc_bpf.c
+++ b/drivers/net/hyperv/netvsc_bpf.c
@@ -45,10 +45,8 @@ u32 netvsc_run_xdp(struct net_device *ndev, struct netvsc_channel *nvchan,
}
xdp_init_buff(xdp, PAGE_SIZE, &nvchan->xdp_rxq);
- xdp->data_hard_start = page_address(page);
- xdp->data = xdp->data_hard_start + NETVSC_XDP_HDRM;
+ xdp_prepare_buff(xdp, page_address(page), NETVSC_XDP_HDRM, len);
xdp_set_data_meta_invalid(xdp);
- xdp->data_end = xdp->data + len;
memcpy(xdp->data, data, len);
diff --git a/drivers/net/tun.c b/drivers/net/tun.c
index a82f7823d428..c7cbd058b345 100644
--- a/drivers/net/tun.c
+++ b/drivers/net/tun.c
@@ -1600,10 +1600,8 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun,
u32 act;
xdp_init_buff(&xdp, buflen, &tfile->xdp_rxq);
- xdp.data_hard_start = buf;
- xdp.data = buf + pad;
+ xdp_prepare_buff(&xdp, buf, pad, len);
xdp_set_data_meta_invalid(&xdp);
- xdp.data_end = xdp.data + len;
act = bpf_prog_run_xdp(xdp_prog, &xdp);
if (act == XDP_REDIRECT || act == XDP_TX) {
diff --git a/drivers/net/veth.c b/drivers/net/veth.c
index 25f3601fb6dd..30a7f2ad39c3 100644
--- a/drivers/net/veth.c
+++ b/drivers/net/veth.c
@@ -710,11 +710,7 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq,
skb = nskb;
}
- xdp.data_hard_start = skb->head;
- xdp.data = skb_mac_header(skb);
- xdp.data_end = xdp.data + pktlen;
- xdp.data_meta = xdp.data;
-
+ xdp_prepare_buff(&xdp, skb->head, skb->mac_header, pktlen);
/* SKB "head" area always have tailroom for skb_shared_info */
frame_sz = (void *)skb_end_pointer(skb) - xdp.data_hard_start;
frame_sz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index a22ce87bcd9c..e57b2d452cbc 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -690,10 +690,8 @@ static struct sk_buff *receive_small(struct net_device *dev,
}
xdp_init_buff(&xdp, buflen, &rq->xdp_rxq);
- xdp.data_hard_start = buf + VIRTNET_RX_PAD + vi->hdr_len;
- xdp.data = xdp.data_hard_start + xdp_headroom;
- xdp.data_end = xdp.data + len;
- xdp.data_meta = xdp.data;
+ xdp_prepare_buff(&xdp, buf + VIRTNET_RX_PAD + vi->hdr_len,
+ xdp_headroom, len);
orig_data = xdp.data;
act = bpf_prog_run_xdp(xdp_prog, &xdp);
stats->xdp_packets++;
@@ -859,10 +857,8 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
*/
data = page_address(xdp_page) + offset;
xdp_init_buff(&xdp, frame_sz - vi->hdr_len, &rq->xdp_rxq);
- xdp.data_hard_start = data - VIRTIO_XDP_HEADROOM + vi->hdr_len;
- xdp.data = data + vi->hdr_len;
- xdp.data_end = xdp.data + (len - vi->hdr_len);
- xdp.data_meta = xdp.data;
+ xdp_prepare_buff(&xdp, data - VIRTIO_XDP_HEADROOM + vi->hdr_len,
+ VIRTIO_XDP_HEADROOM, len - vi->hdr_len);
act = bpf_prog_run_xdp(xdp_prog, &xdp);
stats->xdp_packets++;
diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
index 329397c60d84..61d3f5f8b7f3 100644
--- a/drivers/net/xen-netfront.c
+++ b/drivers/net/xen-netfront.c
@@ -866,10 +866,8 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
&queue->xdp_rxq);
- xdp->data_hard_start = page_address(pdata);
- xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
+ xdp_prepare_buff(xdp, page_address(pdata), XDP_PACKET_HEADROOM, len);
xdp_set_data_meta_invalid(xdp);
- xdp->data_end = xdp->data + len;
act = bpf_prog_run_xdp(prog, xdp);
switch (act) {
diff --git a/include/net/xdp.h b/include/net/xdp.h
index 3fb3a9aa1b71..66d8a4b317a3 100644
--- a/include/net/xdp.h
+++ b/include/net/xdp.h
@@ -83,6 +83,18 @@ xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
xdp->rxq = rxq;
}
+static inline void
+xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
+ int headroom, int data_len)
+{
+ unsigned char *data = hard_start + headroom;
+
+ xdp->data_hard_start = hard_start;
+ xdp->data = data;
+ xdp->data_end = data + data_len;
+ xdp->data_meta = data;
+}
+
/* Reserve memory area at end-of data area.
*
* This macro reserves tailroom in the XDP buffer by limiting the
diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
index a8fa5a9e4137..fe5a80d396e3 100644
--- a/net/bpf/test_run.c
+++ b/net/bpf/test_run.c
@@ -636,10 +636,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr,
if (IS_ERR(data))
return PTR_ERR(data);
- xdp.data_hard_start = data;
- xdp.data = data + headroom;
- xdp.data_meta = xdp.data;
- xdp.data_end = xdp.data + size;
+ xdp_prepare_buff(&xdp, data, headroom, size);
rxqueue = __netif_get_rx_queue(current->nsproxy->net_ns->loopback_dev, 0);
xdp_init_buff(&xdp, headroom + max_data_sz + tailroom,
diff --git a/net/core/dev.c b/net/core/dev.c
index bac56afcf6bc..2997177876cc 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -4592,7 +4592,7 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
__be16 orig_eth_type;
struct ethhdr *eth;
bool orig_bcast;
- int hlen, off;
+ int off;
/* Reinjected packets coming from act_mirred or similar should
* not get XDP generic processing.
@@ -4624,11 +4624,9 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
* header.
*/
mac_len = skb->data - skb_mac_header(skb);
- hlen = skb_headlen(skb) + mac_len;
- xdp->data = skb->data - mac_len;
- xdp->data_meta = xdp->data;
- xdp->data_end = xdp->data + hlen;
- xdp->data_hard_start = skb->data - skb_headroom(skb);
+ xdp_prepare_buff(xdp, skb->data - skb_headroom(skb),
+ skb_headroom(skb) - mac_len,
+ skb_headlen(skb) + mac_len);
/* SKB "head" area always have tailroom for skb_shared_info */
frame_sz = (void *)skb_end_pointer(skb) - xdp->data_hard_start;
--
2.29.2
^ permalink raw reply related [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff
2020-12-12 17:41 [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff Lorenzo Bianconi
2020-12-12 17:41 ` [PATCH v3 bpf-next 1/2] net: xdp: introduce xdp_init_buff utility routine Lorenzo Bianconi
2020-12-12 17:41 ` [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff " Lorenzo Bianconi
@ 2020-12-14 15:32 ` Martin Habets
2020-12-14 17:53 ` Camelia Alexandra Groza
3 siblings, 0 replies; 19+ messages in thread
From: Martin Habets @ 2020-12-14 15:32 UTC (permalink / raw)
To: Lorenzo Bianconi
Cc: bpf, netdev, davem, kuba, ast, daniel, brouer, lorenzo.bianconi,
alexander.duyck, maciej.fijalkowski, saeed
On Sat, Dec 12, 2020 at 06:41:47PM +0100, Lorenzo Bianconi wrote:
> Introduce xdp_init_buff and xdp_prepare_buff utility routines to initialize
> xdp_buff data structure and remove duplicated code in all XDP capable
> drivers.
>
> Changes since v2:
> - precompute xdp->data as hard_start + headroom and save it in a local
> variable to reuse it for xdp->data_end and xdp->data_meta in
> xdp_prepare_buff()
>
> Changes since v1:
> - introduce xdp_prepare_buff utility routine
>
> Lorenzo Bianconi (2):
> net: xdp: introduce xdp_init_buff utility routine
> net: xdp: introduce xdp_prepare_buff utility routine
For changes in drivers/net/ethernet/sfc:
Acked-by: Martin Habets <habetsm.xilinx@gmail.com>
> drivers/net/ethernet/amazon/ena/ena_netdev.c | 8 +++-----
> drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 7 ++-----
> .../net/ethernet/cavium/thunder/nicvf_main.c | 11 ++++++-----
> .../net/ethernet/freescale/dpaa/dpaa_eth.c | 10 ++++------
> .../net/ethernet/freescale/dpaa2/dpaa2-eth.c | 13 +++++--------
> drivers/net/ethernet/intel/i40e/i40e_txrx.c | 18 +++++++++---------
> drivers/net/ethernet/intel/ice/ice_txrx.c | 17 +++++++++--------
> drivers/net/ethernet/intel/igb/igb_main.c | 18 +++++++++---------
> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 19 +++++++++----------
> .../net/ethernet/intel/ixgbevf/ixgbevf_main.c | 19 +++++++++----------
> drivers/net/ethernet/marvell/mvneta.c | 9 +++------
> .../net/ethernet/marvell/mvpp2/mvpp2_main.c | 13 +++++++------
> drivers/net/ethernet/mellanox/mlx4/en_rx.c | 8 +++-----
> .../net/ethernet/mellanox/mlx5/core/en_rx.c | 7 ++-----
> .../ethernet/netronome/nfp/nfp_net_common.c | 12 ++++++------
> drivers/net/ethernet/qlogic/qede/qede_fp.c | 7 ++-----
> drivers/net/ethernet/sfc/rx.c | 9 +++------
> drivers/net/ethernet/socionext/netsec.c | 8 +++-----
> drivers/net/ethernet/ti/cpsw.c | 17 ++++++-----------
> drivers/net/ethernet/ti/cpsw_new.c | 17 ++++++-----------
> drivers/net/hyperv/netvsc_bpf.c | 7 ++-----
> drivers/net/tun.c | 11 ++++-------
> drivers/net/veth.c | 14 +++++---------
> drivers/net/virtio_net.c | 18 ++++++------------
> drivers/net/xen-netfront.c | 8 +++-----
> include/net/xdp.h | 19 +++++++++++++++++++
> net/bpf/test_run.c | 9 +++------
> net/core/dev.c | 18 ++++++++----------
> 28 files changed, 156 insertions(+), 195 deletions(-)
>
> --
> 2.29.2
--
Martin Habets <habetsm.xilinx@gmail.com>
^ permalink raw reply [flat|nested] 19+ messages in thread
* RE: [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff
2020-12-12 17:41 [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff Lorenzo Bianconi
` (2 preceding siblings ...)
2020-12-14 15:32 ` [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff Martin Habets
@ 2020-12-14 17:53 ` Camelia Alexandra Groza
3 siblings, 0 replies; 19+ messages in thread
From: Camelia Alexandra Groza @ 2020-12-14 17:53 UTC (permalink / raw)
To: Lorenzo Bianconi, bpf, netdev
Cc: davem, kuba, ast, daniel, brouer, lorenzo.bianconi,
alexander.duyck, maciej.fijalkowski, saeed
> -----Original Message-----
> From: Lorenzo Bianconi <lorenzo@kernel.org>
> Sent: Saturday, December 12, 2020 19:42
> To: bpf@vger.kernel.org; netdev@vger.kernel.org
> Cc: davem@davemloft.net; kuba@kernel.org; ast@kernel.org;
> daniel@iogearbox.net; brouer@redhat.com; lorenzo.bianconi@redhat.com;
> alexander.duyck@gmail.com; maciej.fijalkowski@intel.com;
> saeed@kernel.org
> Subject: [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff
>
> Introduce xdp_init_buff and xdp_prepare_buff utility routines to initialize
> xdp_buff data structure and remove duplicated code in all XDP capable
> drivers.
>
> Changes since v2:
> - precompute xdp->data as hard_start + headroom and save it in a local
> variable to reuse it for xdp->data_end and xdp->data_meta in
> xdp_prepare_buff()
>
> Changes since v1:
> - introduce xdp_prepare_buff utility routine
>
> Lorenzo Bianconi (2):
> net: xdp: introduce xdp_init_buff utility routine
> net: xdp: introduce xdp_prepare_buff utility routine
For the drivers/net/ethernet/freescale/dpaa changes:
Acked-by: Camelia Groza <camelia.groza@nxp.com>
> drivers/net/ethernet/amazon/ena/ena_netdev.c | 8 +++-----
> drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 7 ++-----
> .../net/ethernet/cavium/thunder/nicvf_main.c | 11 ++++++-----
> .../net/ethernet/freescale/dpaa/dpaa_eth.c | 10 ++++------
> .../net/ethernet/freescale/dpaa2/dpaa2-eth.c | 13 +++++--------
> drivers/net/ethernet/intel/i40e/i40e_txrx.c | 18 +++++++++---------
> drivers/net/ethernet/intel/ice/ice_txrx.c | 17 +++++++++--------
> drivers/net/ethernet/intel/igb/igb_main.c | 18 +++++++++---------
> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 19 +++++++++----------
> .../net/ethernet/intel/ixgbevf/ixgbevf_main.c | 19 +++++++++----------
> drivers/net/ethernet/marvell/mvneta.c | 9 +++------
> .../net/ethernet/marvell/mvpp2/mvpp2_main.c | 13 +++++++------
> drivers/net/ethernet/mellanox/mlx4/en_rx.c | 8 +++-----
> .../net/ethernet/mellanox/mlx5/core/en_rx.c | 7 ++-----
> .../ethernet/netronome/nfp/nfp_net_common.c | 12 ++++++------
> drivers/net/ethernet/qlogic/qede/qede_fp.c | 7 ++-----
> drivers/net/ethernet/sfc/rx.c | 9 +++------
> drivers/net/ethernet/socionext/netsec.c | 8 +++-----
> drivers/net/ethernet/ti/cpsw.c | 17 ++++++-----------
> drivers/net/ethernet/ti/cpsw_new.c | 17 ++++++-----------
> drivers/net/hyperv/netvsc_bpf.c | 7 ++-----
> drivers/net/tun.c | 11 ++++-------
> drivers/net/veth.c | 14 +++++---------
> drivers/net/virtio_net.c | 18 ++++++------------
> drivers/net/xen-netfront.c | 8 +++-----
> include/net/xdp.h | 19 +++++++++++++++++++
> net/bpf/test_run.c | 9 +++------
> net/core/dev.c | 18 ++++++++----------
> 28 files changed, 156 insertions(+), 195 deletions(-)
>
> --
> 2.29.2
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-12 17:41 ` [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff " Lorenzo Bianconi
@ 2020-12-15 12:36 ` Maciej Fijalkowski
2020-12-15 13:47 ` Lorenzo Bianconi
0 siblings, 1 reply; 19+ messages in thread
From: Maciej Fijalkowski @ 2020-12-15 12:36 UTC (permalink / raw)
To: Lorenzo Bianconi
Cc: bpf, netdev, davem, kuba, ast, daniel, brouer, lorenzo.bianconi,
alexander.duyck, saeed
On Sat, Dec 12, 2020 at 06:41:49PM +0100, Lorenzo Bianconi wrote:
> Introduce xdp_prepare_buff utility routine to initialize per-descriptor
> xdp_buff fields (e.g. xdp_buff pointers). Rely on xdp_prepare_buff() in
> all XDP capable drivers.
>
> Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org>
> ---
> drivers/net/ethernet/amazon/ena/ena_netdev.c | 5 ++---
> drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c | 4 +---
> drivers/net/ethernet/cavium/thunder/nicvf_main.c | 7 ++++---
> drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 6 ++----
> drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 13 +++++--------
> drivers/net/ethernet/intel/i40e/i40e_txrx.c | 12 ++++++------
> drivers/net/ethernet/intel/ice/ice_txrx.c | 11 ++++++-----
> drivers/net/ethernet/intel/igb/igb_main.c | 12 ++++++------
> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 12 ++++++------
> drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 12 ++++++------
> drivers/net/ethernet/marvell/mvneta.c | 6 ++----
> drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 7 +++----
> drivers/net/ethernet/mellanox/mlx4/en_rx.c | 5 ++---
> drivers/net/ethernet/mellanox/mlx5/core/en_rx.c | 4 +---
> .../net/ethernet/netronome/nfp/nfp_net_common.c | 8 ++++----
> drivers/net/ethernet/qlogic/qede/qede_fp.c | 4 +---
> drivers/net/ethernet/sfc/rx.c | 6 ++----
> drivers/net/ethernet/socionext/netsec.c | 5 ++---
> drivers/net/ethernet/ti/cpsw.c | 15 +++++----------
> drivers/net/ethernet/ti/cpsw_new.c | 15 +++++----------
> drivers/net/hyperv/netvsc_bpf.c | 4 +---
> drivers/net/tun.c | 4 +---
> drivers/net/veth.c | 6 +-----
> drivers/net/virtio_net.c | 12 ++++--------
> drivers/net/xen-netfront.c | 4 +---
> include/net/xdp.h | 12 ++++++++++++
> net/bpf/test_run.c | 5 +----
> net/core/dev.c | 10 ++++------
> 28 files changed, 96 insertions(+), 130 deletions(-)
>
> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c b/drivers/net/ethernet/amazon/ena/ena_netdev.c
> index 338dce73927e..1cfd0c98677e 100644
> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
> @@ -1519,10 +1519,9 @@ static int ena_xdp_handle_buff(struct ena_ring *rx_ring, struct xdp_buff *xdp)
> int ret;
>
> rx_info = &rx_ring->rx_buffer_info[rx_ring->ena_bufs[0].req_id];
> - xdp->data = page_address(rx_info->page) + rx_info->page_offset;
> + xdp_prepare_buff(xdp, page_address(rx_info->page),
> + rx_info->page_offset, rx_ring->ena_bufs[0].len);
> xdp_set_data_meta_invalid(xdp);
> - xdp->data_hard_start = page_address(rx_info->page);
> - xdp->data_end = xdp->data + rx_ring->ena_bufs[0].len;
> /* If for some reason we received a bigger packet than
> * we expect, then we simply drop it
> */
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> index b7942c3440c0..e1664b86a7b8 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> @@ -134,10 +134,8 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
>
> txr = rxr->bnapi->tx_ring;
> xdp_init_buff(&xdp, PAGE_SIZE, &rxr->xdp_rxq);
> - xdp.data_hard_start = *data_ptr - offset;
> - xdp.data = *data_ptr;
> + xdp_prepare_buff(&xdp, *data_ptr - offset, offset, *len);
> xdp_set_data_meta_invalid(&xdp);
> - xdp.data_end = *data_ptr + *len;
> orig_data = xdp.data;
>
> rcu_read_lock();
> diff --git a/drivers/net/ethernet/cavium/thunder/nicvf_main.c b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
> index 9fc672f075f2..9bdac04359c6 100644
> --- a/drivers/net/ethernet/cavium/thunder/nicvf_main.c
> +++ b/drivers/net/ethernet/cavium/thunder/nicvf_main.c
> @@ -530,6 +530,7 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog,
> struct cqe_rx_t *cqe_rx, struct snd_queue *sq,
> struct rcv_queue *rq, struct sk_buff **skb)
> {
> + unsigned char *hard_start, *data;
> struct xdp_buff xdp;
> struct page *page;
> u32 action;
> @@ -549,10 +550,10 @@ static inline bool nicvf_xdp_rx(struct nicvf *nic, struct bpf_prog *prog,
>
> xdp_init_buff(&xdp, RCV_FRAG_LEN + XDP_PACKET_HEADROOM,
> &rq->xdp_rxq);
> - xdp.data_hard_start = page_address(page);
> - xdp.data = (void *)cpu_addr;
> + hard_start = page_address(page);
> + data = (unsigned char *)cpu_addr;
> + xdp_prepare_buff(&xdp, hard_start, data - hard_start, len);
> xdp_set_data_meta_invalid(&xdp);
> - xdp.data_end = xdp.data + len;
> orig_data = xdp.data;
>
> rcu_read_lock();
> diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
> index 93030000e0aa..86ee07c90154 100644
> --- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
> +++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c
> @@ -2538,10 +2538,8 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr,
>
> xdp_init_buff(&xdp, DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE,
> &dpaa_fq->xdp_rxq);
> - xdp.data = vaddr + fd_off;
> - xdp.data_meta = xdp.data;
> - xdp.data_hard_start = xdp.data - XDP_PACKET_HEADROOM;
> - xdp.data_end = xdp.data + qm_fd_get_length(fd);
> + xdp_prepare_buff(&xdp, vaddr + fd_off - XDP_PACKET_HEADROOM,
> + XDP_PACKET_HEADROOM, qm_fd_get_length(fd));
>
> /* We reserve a fixed headroom of 256 bytes under the erratum and we
> * offer it all to XDP programs to use. If no room is left for the
> diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
> index a4ade0b5adb0..12358f5d59d6 100644
> --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
> +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c
> @@ -350,7 +350,7 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv,
> struct bpf_prog *xdp_prog;
> struct xdp_buff xdp;
> u32 xdp_act = XDP_PASS;
> - int err;
> + int err, offset;
>
> rcu_read_lock();
>
> @@ -358,13 +358,10 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv,
> if (!xdp_prog)
> goto out;
>
> - xdp_init_buff(&xdp,
> - DPAA2_ETH_RX_BUF_RAW_SIZE -
> - (dpaa2_fd_get_offset(fd) - XDP_PACKET_HEADROOM),
> - &ch->xdp_rxq);
> - xdp.data = vaddr + dpaa2_fd_get_offset(fd);
> - xdp.data_end = xdp.data + dpaa2_fd_get_len(fd);
> - xdp.data_hard_start = xdp.data - XDP_PACKET_HEADROOM;
> + offset = dpaa2_fd_get_offset(fd) - XDP_PACKET_HEADROOM;
> + xdp_init_buff(&xdp, DPAA2_ETH_RX_BUF_RAW_SIZE - offset, &ch->xdp_rxq);
> + xdp_prepare_buff(&xdp, vaddr + offset, XDP_PACKET_HEADROOM,
> + dpaa2_fd_get_len(fd));
> xdp_set_data_meta_invalid(&xdp);
>
> xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> index 4dbbbd49c389..fcd1ca3343fb 100644
> --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> @@ -2393,12 +2393,12 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
>
> /* retrieve a buffer from the ring */
> if (!skb) {
> - xdp.data = page_address(rx_buffer->page) +
> - rx_buffer->page_offset;
> - xdp.data_meta = xdp.data;
> - xdp.data_hard_start = xdp.data -
> - i40e_rx_offset(rx_ring);
> - xdp.data_end = xdp.data + size;
> + unsigned int offset = i40e_rx_offset(rx_ring);
I now see that we could call the i40e_rx_offset() once per napi, so can
you pull this variable out and have it initialized a single time? Applies
to other intel drivers as well.
I also feel like it's sub-optimal for drivers that are calculating the
data_hard_start out of data (intel, bnxt, sfc and mlx4 have this approach)
due to additional add, but I don't have a solution for that. Would be
weird to have another helper. Not sure what other people think, but I have
in mind a "death by 1000 cuts" phrase :)
> + unsigned char *hard_start;
> +
> + hard_start = page_address(rx_buffer->page) +
> + rx_buffer->page_offset - offset;
> + xdp_prepare_buff(&xdp, hard_start, offset, size);
> #if (PAGE_SIZE > 4096)
> /* At larger PAGE_SIZE, frame_sz depend on len size */
> xdp.frame_sz = i40e_rx_frame_truesize(rx_ring, size);
> diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
> index d52d98d56367..a7a00060f520 100644
> --- a/drivers/net/ethernet/intel/ice/ice_txrx.c
> +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
> @@ -1094,8 +1094,9 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
> while (likely(total_rx_pkts < (unsigned int)budget)) {
> union ice_32b_rx_flex_desc *rx_desc;
> struct ice_rx_buf *rx_buf;
> + unsigned int size, offset;
> + unsigned char *hard_start;
> struct sk_buff *skb;
> - unsigned int size;
> u16 stat_err_bits;
> u16 vlan_tag = 0;
> u8 rx_ptype;
> @@ -1138,10 +1139,10 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
> goto construct_skb;
> }
>
> - xdp.data = page_address(rx_buf->page) + rx_buf->page_offset;
> - xdp.data_hard_start = xdp.data - ice_rx_offset(rx_ring);
> - xdp.data_meta = xdp.data;
> - xdp.data_end = xdp.data + size;
> + offset = ice_rx_offset(rx_ring);
> + hard_start = page_address(rx_buf->page) + rx_buf->page_offset -
> + offset;
> + xdp_prepare_buff(&xdp, hard_start, offset, size);
> #if (PAGE_SIZE > 4096)
> /* At larger PAGE_SIZE, frame_sz depend on len size */
> xdp.frame_sz = ice_rx_frame_truesize(rx_ring, size);
> diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> index 365dfc0e3b65..070b2bb4e9ca 100644
> --- a/drivers/net/ethernet/intel/igb/igb_main.c
> +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> @@ -8700,12 +8700,12 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget)
>
> /* retrieve a buffer from the ring */
> if (!skb) {
> - xdp.data = page_address(rx_buffer->page) +
> - rx_buffer->page_offset;
> - xdp.data_meta = xdp.data;
> - xdp.data_hard_start = xdp.data -
> - igb_rx_offset(rx_ring);
> - xdp.data_end = xdp.data + size;
> + unsigned int offset = igb_rx_offset(rx_ring);
> + unsigned char *hard_start;
> +
> + hard_start = page_address(rx_buffer->page) +
> + rx_buffer->page_offset - offset;
> + xdp_prepare_buff(&xdp, hard_start, offset, size);
> #if (PAGE_SIZE > 4096)
> /* At larger PAGE_SIZE, frame_sz depend on len size */
> xdp.frame_sz = igb_rx_frame_truesize(rx_ring, size);
> diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> index dcd49cfa36f7..e34054433c7a 100644
> --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> @@ -2325,12 +2325,12 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,
>
> /* retrieve a buffer from the ring */
> if (!skb) {
> - xdp.data = page_address(rx_buffer->page) +
> - rx_buffer->page_offset;
> - xdp.data_meta = xdp.data;
> - xdp.data_hard_start = xdp.data -
> - ixgbe_rx_offset(rx_ring);
> - xdp.data_end = xdp.data + size;
> + unsigned int offset = ixgbe_rx_offset(rx_ring);
> + unsigned char *hard_start;
> +
> + hard_start = page_address(rx_buffer->page) +
> + rx_buffer->page_offset - offset;
> + xdp_prepare_buff(&xdp, hard_start, offset, size);
> #if (PAGE_SIZE > 4096)
> /* At larger PAGE_SIZE, frame_sz depend on len size */
> xdp.frame_sz = ixgbe_rx_frame_truesize(rx_ring, size);
> diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
> index 624efcd71569..51df79005ccb 100644
> --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
> +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
> @@ -1160,12 +1160,12 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
>
> /* retrieve a buffer from the ring */
> if (!skb) {
> - xdp.data = page_address(rx_buffer->page) +
> - rx_buffer->page_offset;
> - xdp.data_meta = xdp.data;
> - xdp.data_hard_start = xdp.data -
> - ixgbevf_rx_offset(rx_ring);
> - xdp.data_end = xdp.data + size;
> + unsigned int offset = ixgbevf_rx_offset(rx_ring);
> + unsigned char *hard_start;
> +
> + hard_start = page_address(rx_buffer->page) +
> + rx_buffer->page_offset - offset;
> + xdp_prepare_buff(&xdp, hard_start, offset, size);
> #if (PAGE_SIZE > 4096)
> /* At larger PAGE_SIZE, frame_sz depend on len size */
> xdp.frame_sz = ixgbevf_rx_frame_truesize(rx_ring, size);
> diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
> index acbb9cb85ada..af6c9cf59809 100644
> --- a/drivers/net/ethernet/marvell/mvneta.c
> +++ b/drivers/net/ethernet/marvell/mvneta.c
> @@ -2263,10 +2263,8 @@ mvneta_swbm_rx_frame(struct mvneta_port *pp,
>
> /* Prefetch header */
> prefetch(data);
> -
> - xdp->data_hard_start = data;
> - xdp->data = data + pp->rx_offset_correction + MVNETA_MH_SIZE;
> - xdp->data_end = xdp->data + data_len;
> + xdp_prepare_buff(xdp, data, pp->rx_offset_correction + MVNETA_MH_SIZE,
> + data_len);
> xdp_set_data_meta_invalid(xdp);
>
> sinfo = xdp_get_shared_info_from_buff(xdp);
> diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> index ca05dfc05058..8c2197b96515 100644
> --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> @@ -3564,16 +3564,15 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
> if (xdp_prog) {
> struct xdp_rxq_info *xdp_rxq;
>
> - xdp.data_hard_start = data;
> - xdp.data = data + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM;
> - xdp.data_end = xdp.data + rx_bytes;
> -
> if (bm_pool->pkt_size == MVPP2_BM_SHORT_PKT_SIZE)
> xdp_rxq = &rxq->xdp_rxq_short;
> else
> xdp_rxq = &rxq->xdp_rxq_long;
>
> xdp_init_buff(&xdp, PAGE_SIZE, xdp_rxq);
> + xdp_prepare_buff(&xdp, data,
> + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM,
> + rx_bytes);
> xdp_set_data_meta_invalid(&xdp);
>
> ret = mvpp2_run_xdp(port, rxq, xdp_prog, &xdp, pp, &ps);
> diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> index 815381b484ca..86c63dedc689 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> @@ -776,10 +776,9 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud
> priv->frag_info[0].frag_size,
> DMA_FROM_DEVICE);
>
> - xdp.data_hard_start = va - frags[0].page_offset;
> - xdp.data = va;
> + xdp_prepare_buff(&xdp, va - frags[0].page_offset,
> + frags[0].page_offset, length);
> xdp_set_data_meta_invalid(&xdp);
> - xdp.data_end = xdp.data + length;
> orig_data = xdp.data;
>
> act = bpf_prog_run_xdp(xdp_prog, &xdp);
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> index c68628b1f30b..a2f4f0ce427f 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> @@ -1128,10 +1128,8 @@ static void mlx5e_fill_xdp_buff(struct mlx5e_rq *rq, void *va, u16 headroom,
> u32 len, struct xdp_buff *xdp)
> {
> xdp_init_buff(xdp, rq->buff.frame0_sz, &rq->xdp_rxq);
> - xdp->data_hard_start = va;
> - xdp->data = va + headroom;
> + xdp_prepare_buff(xdp, va, headroom, len);
> xdp_set_data_meta_invalid(xdp);
> - xdp->data_end = xdp->data + len;
> }
>
> static struct sk_buff *
> diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
> index 68e03e8257f2..5d0046c24b8c 100644
> --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
> +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
> @@ -1914,10 +1914,10 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget)
> unsigned int dma_off;
> int act;
>
> - xdp.data_hard_start = rxbuf->frag + NFP_NET_RX_BUF_HEADROOM;
> - xdp.data = orig_data;
> - xdp.data_meta = orig_data;
> - xdp.data_end = orig_data + pkt_len;
> + xdp_prepare_buff(&xdp,
> + rxbuf->frag + NFP_NET_RX_BUF_HEADROOM,
> + pkt_off - NFP_NET_RX_BUF_HEADROOM,
> + pkt_len);
>
> act = bpf_prog_run_xdp(xdp_prog, &xdp);
>
> diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c
> index d40220043883..9c50df499046 100644
> --- a/drivers/net/ethernet/qlogic/qede/qede_fp.c
> +++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c
> @@ -1091,10 +1091,8 @@ static bool qede_rx_xdp(struct qede_dev *edev,
> enum xdp_action act;
>
> xdp_init_buff(&xdp, rxq->rx_buf_seg_size, &rxq->xdp_rxq);
> - xdp.data_hard_start = page_address(bd->data);
> - xdp.data = xdp.data_hard_start + *data_offset;
> + xdp_prepare_buff(&xdp, page_address(bd->data), *data_offset, *len);
> xdp_set_data_meta_invalid(&xdp);
> - xdp.data_end = xdp.data + *len;
>
> /* Queues always have a full reset currently, so for the time
> * being until there's atomic program replace just mark read
> diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c
> index eaa6650955d1..9015a1639234 100644
> --- a/drivers/net/ethernet/sfc/rx.c
> +++ b/drivers/net/ethernet/sfc/rx.c
> @@ -294,12 +294,10 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel,
> efx->rx_prefix_size);
>
> xdp_init_buff(&xdp, efx->rx_page_buf_step, &rx_queue->xdp_rxq_info);
> - xdp.data = *ehp;
> - xdp.data_hard_start = xdp.data - EFX_XDP_HEADROOM;
> -
> + xdp_prepare_buff(&xdp, *ehp - EFX_XDP_HEADROOM, EFX_XDP_HEADROOM,
> + rx_buf->len);
> /* No support yet for XDP metadata */
> xdp_set_data_meta_invalid(&xdp);
> - xdp.data_end = xdp.data + rx_buf->len;
>
> xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> rcu_read_unlock();
> diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c
> index 945ca9517bf9..80bb1a6612b1 100644
> --- a/drivers/net/ethernet/socionext/netsec.c
> +++ b/drivers/net/ethernet/socionext/netsec.c
> @@ -1015,10 +1015,9 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget)
> dma_dir);
> prefetch(desc->addr);
>
> - xdp.data_hard_start = desc->addr;
> - xdp.data = desc->addr + NETSEC_RXBUF_HEADROOM;
> + xdp_prepare_buff(&xdp, desc->addr, NETSEC_RXBUF_HEADROOM,
> + pkt_len);
> xdp_set_data_meta_invalid(&xdp);
> - xdp.data_end = xdp.data + pkt_len;
>
> if (xdp_prog) {
> xdp_result = netsec_run_xdp(priv, xdp_prog, &xdp);
> diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
> index 78a923391828..c08fd6a6be9b 100644
> --- a/drivers/net/ethernet/ti/cpsw.c
> +++ b/drivers/net/ethernet/ti/cpsw.c
> @@ -392,22 +392,17 @@ static void cpsw_rx_handler(void *token, int len, int status)
> }
>
> if (priv->xdp_prog) {
> - xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
> + int headroom = CPSW_HEADROOM, size = len;
>
> + xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
> if (status & CPDMA_RX_VLAN_ENCAP) {
> - xdp.data = pa + CPSW_HEADROOM +
> - CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> - xdp.data_end = xdp.data + len -
> - CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> - } else {
> - xdp.data = pa + CPSW_HEADROOM;
> - xdp.data_end = xdp.data + len;
> + headroom += CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> + size -= CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> }
>
> + xdp_prepare_buff(&xdp, pa, headroom, size);
> xdp_set_data_meta_invalid(&xdp);
>
> - xdp.data_hard_start = pa;
> -
> port = priv->emac_port + cpsw->data.dual_emac;
> ret = cpsw_run_xdp(priv, ch, &xdp, page, port);
> if (ret != CPSW_XDP_PASS)
> diff --git a/drivers/net/ethernet/ti/cpsw_new.c b/drivers/net/ethernet/ti/cpsw_new.c
> index 1b3385ec9645..c74c997d1cf2 100644
> --- a/drivers/net/ethernet/ti/cpsw_new.c
> +++ b/drivers/net/ethernet/ti/cpsw_new.c
> @@ -335,22 +335,17 @@ static void cpsw_rx_handler(void *token, int len, int status)
> }
>
> if (priv->xdp_prog) {
> - xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
> + int headroom = CPSW_HEADROOM, size = len;
>
> + xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
> if (status & CPDMA_RX_VLAN_ENCAP) {
> - xdp.data = pa + CPSW_HEADROOM +
> - CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> - xdp.data_end = xdp.data + len -
> - CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> - } else {
> - xdp.data = pa + CPSW_HEADROOM;
> - xdp.data_end = xdp.data + len;
> + headroom += CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> + size -= CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> }
>
> + xdp_prepare_buff(&xdp, pa, headroom, size);
> xdp_set_data_meta_invalid(&xdp);
>
> - xdp.data_hard_start = pa;
> -
> ret = cpsw_run_xdp(priv, ch, &xdp, page, priv->emac_port);
> if (ret != CPSW_XDP_PASS)
> goto requeue;
> diff --git a/drivers/net/hyperv/netvsc_bpf.c b/drivers/net/hyperv/netvsc_bpf.c
> index 14a7ee4c6899..93c202d6aff5 100644
> --- a/drivers/net/hyperv/netvsc_bpf.c
> +++ b/drivers/net/hyperv/netvsc_bpf.c
> @@ -45,10 +45,8 @@ u32 netvsc_run_xdp(struct net_device *ndev, struct netvsc_channel *nvchan,
> }
>
> xdp_init_buff(xdp, PAGE_SIZE, &nvchan->xdp_rxq);
> - xdp->data_hard_start = page_address(page);
> - xdp->data = xdp->data_hard_start + NETVSC_XDP_HDRM;
> + xdp_prepare_buff(xdp, page_address(page), NETVSC_XDP_HDRM, len);
> xdp_set_data_meta_invalid(xdp);
> - xdp->data_end = xdp->data + len;
>
> memcpy(xdp->data, data, len);
>
> diff --git a/drivers/net/tun.c b/drivers/net/tun.c
> index a82f7823d428..c7cbd058b345 100644
> --- a/drivers/net/tun.c
> +++ b/drivers/net/tun.c
> @@ -1600,10 +1600,8 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun,
> u32 act;
>
> xdp_init_buff(&xdp, buflen, &tfile->xdp_rxq);
> - xdp.data_hard_start = buf;
> - xdp.data = buf + pad;
> + xdp_prepare_buff(&xdp, buf, pad, len);
> xdp_set_data_meta_invalid(&xdp);
> - xdp.data_end = xdp.data + len;
>
> act = bpf_prog_run_xdp(xdp_prog, &xdp);
> if (act == XDP_REDIRECT || act == XDP_TX) {
> diff --git a/drivers/net/veth.c b/drivers/net/veth.c
> index 25f3601fb6dd..30a7f2ad39c3 100644
> --- a/drivers/net/veth.c
> +++ b/drivers/net/veth.c
> @@ -710,11 +710,7 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq,
> skb = nskb;
> }
>
> - xdp.data_hard_start = skb->head;
> - xdp.data = skb_mac_header(skb);
> - xdp.data_end = xdp.data + pktlen;
> - xdp.data_meta = xdp.data;
> -
> + xdp_prepare_buff(&xdp, skb->head, skb->mac_header, pktlen);
> /* SKB "head" area always have tailroom for skb_shared_info */
> frame_sz = (void *)skb_end_pointer(skb) - xdp.data_hard_start;
> frame_sz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> index a22ce87bcd9c..e57b2d452cbc 100644
> --- a/drivers/net/virtio_net.c
> +++ b/drivers/net/virtio_net.c
> @@ -690,10 +690,8 @@ static struct sk_buff *receive_small(struct net_device *dev,
> }
>
> xdp_init_buff(&xdp, buflen, &rq->xdp_rxq);
> - xdp.data_hard_start = buf + VIRTNET_RX_PAD + vi->hdr_len;
> - xdp.data = xdp.data_hard_start + xdp_headroom;
> - xdp.data_end = xdp.data + len;
> - xdp.data_meta = xdp.data;
> + xdp_prepare_buff(&xdp, buf + VIRTNET_RX_PAD + vi->hdr_len,
> + xdp_headroom, len);
> orig_data = xdp.data;
> act = bpf_prog_run_xdp(xdp_prog, &xdp);
> stats->xdp_packets++;
> @@ -859,10 +857,8 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
> */
> data = page_address(xdp_page) + offset;
> xdp_init_buff(&xdp, frame_sz - vi->hdr_len, &rq->xdp_rxq);
> - xdp.data_hard_start = data - VIRTIO_XDP_HEADROOM + vi->hdr_len;
> - xdp.data = data + vi->hdr_len;
> - xdp.data_end = xdp.data + (len - vi->hdr_len);
> - xdp.data_meta = xdp.data;
> + xdp_prepare_buff(&xdp, data - VIRTIO_XDP_HEADROOM + vi->hdr_len,
> + VIRTIO_XDP_HEADROOM, len - vi->hdr_len);
>
> act = bpf_prog_run_xdp(xdp_prog, &xdp);
> stats->xdp_packets++;
> diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
> index 329397c60d84..61d3f5f8b7f3 100644
> --- a/drivers/net/xen-netfront.c
> +++ b/drivers/net/xen-netfront.c
> @@ -866,10 +866,8 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
>
> xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
> &queue->xdp_rxq);
> - xdp->data_hard_start = page_address(pdata);
> - xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
> + xdp_prepare_buff(xdp, page_address(pdata), XDP_PACKET_HEADROOM, len);
> xdp_set_data_meta_invalid(xdp);
> - xdp->data_end = xdp->data + len;
>
> act = bpf_prog_run_xdp(prog, xdp);
> switch (act) {
> diff --git a/include/net/xdp.h b/include/net/xdp.h
> index 3fb3a9aa1b71..66d8a4b317a3 100644
> --- a/include/net/xdp.h
> +++ b/include/net/xdp.h
> @@ -83,6 +83,18 @@ xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
> xdp->rxq = rxq;
> }
>
> +static inline void
> +xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> + int headroom, int data_len)
> +{
> + unsigned char *data = hard_start + headroom;
> +
> + xdp->data_hard_start = hard_start;
> + xdp->data = data;
> + xdp->data_end = data + data_len;
> + xdp->data_meta = data;
> +}
> +
> /* Reserve memory area at end-of data area.
> *
> * This macro reserves tailroom in the XDP buffer by limiting the
> diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
> index a8fa5a9e4137..fe5a80d396e3 100644
> --- a/net/bpf/test_run.c
> +++ b/net/bpf/test_run.c
> @@ -636,10 +636,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr,
> if (IS_ERR(data))
> return PTR_ERR(data);
>
> - xdp.data_hard_start = data;
> - xdp.data = data + headroom;
> - xdp.data_meta = xdp.data;
> - xdp.data_end = xdp.data + size;
> + xdp_prepare_buff(&xdp, data, headroom, size);
>
> rxqueue = __netif_get_rx_queue(current->nsproxy->net_ns->loopback_dev, 0);
> xdp_init_buff(&xdp, headroom + max_data_sz + tailroom,
> diff --git a/net/core/dev.c b/net/core/dev.c
> index bac56afcf6bc..2997177876cc 100644
> --- a/net/core/dev.c
> +++ b/net/core/dev.c
> @@ -4592,7 +4592,7 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
> __be16 orig_eth_type;
> struct ethhdr *eth;
> bool orig_bcast;
> - int hlen, off;
> + int off;
>
> /* Reinjected packets coming from act_mirred or similar should
> * not get XDP generic processing.
> @@ -4624,11 +4624,9 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
> * header.
> */
> mac_len = skb->data - skb_mac_header(skb);
> - hlen = skb_headlen(skb) + mac_len;
> - xdp->data = skb->data - mac_len;
> - xdp->data_meta = xdp->data;
> - xdp->data_end = xdp->data + hlen;
> - xdp->data_hard_start = skb->data - skb_headroom(skb);
> + xdp_prepare_buff(xdp, skb->data - skb_headroom(skb),
> + skb_headroom(skb) - mac_len,
> + skb_headlen(skb) + mac_len);
>
> /* SKB "head" area always have tailroom for skb_shared_info */
> frame_sz = (void *)skb_end_pointer(skb) - xdp->data_hard_start;
> --
> 2.29.2
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-15 12:36 ` Maciej Fijalkowski
@ 2020-12-15 13:47 ` Lorenzo Bianconi
2020-12-15 14:51 ` Daniel Borkmann
2020-12-16 8:52 ` Jesper Dangaard Brouer
0 siblings, 2 replies; 19+ messages in thread
From: Lorenzo Bianconi @ 2020-12-15 13:47 UTC (permalink / raw)
To: Maciej Fijalkowski
Cc: Lorenzo Bianconi, bpf, netdev, davem, kuba, ast, daniel, brouer,
alexander.duyck, saeed
[-- Attachment #1: Type: text/plain, Size: 22259 bytes --]
[...]
> > xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > index 4dbbbd49c389..fcd1ca3343fb 100644
> > --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > @@ -2393,12 +2393,12 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
> >
> > /* retrieve a buffer from the ring */
> > if (!skb) {
> > - xdp.data = page_address(rx_buffer->page) +
> > - rx_buffer->page_offset;
> > - xdp.data_meta = xdp.data;
> > - xdp.data_hard_start = xdp.data -
> > - i40e_rx_offset(rx_ring);
> > - xdp.data_end = xdp.data + size;
> > + unsigned int offset = i40e_rx_offset(rx_ring);
>
> I now see that we could call the i40e_rx_offset() once per napi, so can
> you pull this variable out and have it initialized a single time? Applies
> to other intel drivers as well.
ack, fine. I will fix in v4.
Regards,
Lorenzo
>
> I also feel like it's sub-optimal for drivers that are calculating the
> data_hard_start out of data (intel, bnxt, sfc and mlx4 have this approach)
> due to additional add, but I don't have a solution for that. Would be
> weird to have another helper. Not sure what other people think, but I have
> in mind a "death by 1000 cuts" phrase :)
>
> > + unsigned char *hard_start;
> > +
> > + hard_start = page_address(rx_buffer->page) +
> > + rx_buffer->page_offset - offset;
> > + xdp_prepare_buff(&xdp, hard_start, offset, size);
> > #if (PAGE_SIZE > 4096)
> > /* At larger PAGE_SIZE, frame_sz depend on len size */
> > xdp.frame_sz = i40e_rx_frame_truesize(rx_ring, size);
> > diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c
> > index d52d98d56367..a7a00060f520 100644
> > --- a/drivers/net/ethernet/intel/ice/ice_txrx.c
> > +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c
> > @@ -1094,8 +1094,9 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
> > while (likely(total_rx_pkts < (unsigned int)budget)) {
> > union ice_32b_rx_flex_desc *rx_desc;
> > struct ice_rx_buf *rx_buf;
> > + unsigned int size, offset;
> > + unsigned char *hard_start;
> > struct sk_buff *skb;
> > - unsigned int size;
> > u16 stat_err_bits;
> > u16 vlan_tag = 0;
> > u8 rx_ptype;
> > @@ -1138,10 +1139,10 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget)
> > goto construct_skb;
> > }
> >
> > - xdp.data = page_address(rx_buf->page) + rx_buf->page_offset;
> > - xdp.data_hard_start = xdp.data - ice_rx_offset(rx_ring);
> > - xdp.data_meta = xdp.data;
> > - xdp.data_end = xdp.data + size;
> > + offset = ice_rx_offset(rx_ring);
> > + hard_start = page_address(rx_buf->page) + rx_buf->page_offset -
> > + offset;
> > + xdp_prepare_buff(&xdp, hard_start, offset, size);
> > #if (PAGE_SIZE > 4096)
> > /* At larger PAGE_SIZE, frame_sz depend on len size */
> > xdp.frame_sz = ice_rx_frame_truesize(rx_ring, size);
> > diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> > index 365dfc0e3b65..070b2bb4e9ca 100644
> > --- a/drivers/net/ethernet/intel/igb/igb_main.c
> > +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> > @@ -8700,12 +8700,12 @@ static int igb_clean_rx_irq(struct igb_q_vector *q_vector, const int budget)
> >
> > /* retrieve a buffer from the ring */
> > if (!skb) {
> > - xdp.data = page_address(rx_buffer->page) +
> > - rx_buffer->page_offset;
> > - xdp.data_meta = xdp.data;
> > - xdp.data_hard_start = xdp.data -
> > - igb_rx_offset(rx_ring);
> > - xdp.data_end = xdp.data + size;
> > + unsigned int offset = igb_rx_offset(rx_ring);
> > + unsigned char *hard_start;
> > +
> > + hard_start = page_address(rx_buffer->page) +
> > + rx_buffer->page_offset - offset;
> > + xdp_prepare_buff(&xdp, hard_start, offset, size);
> > #if (PAGE_SIZE > 4096)
> > /* At larger PAGE_SIZE, frame_sz depend on len size */
> > xdp.frame_sz = igb_rx_frame_truesize(rx_ring, size);
> > diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> > index dcd49cfa36f7..e34054433c7a 100644
> > --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> > +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c
> > @@ -2325,12 +2325,12 @@ static int ixgbe_clean_rx_irq(struct ixgbe_q_vector *q_vector,
> >
> > /* retrieve a buffer from the ring */
> > if (!skb) {
> > - xdp.data = page_address(rx_buffer->page) +
> > - rx_buffer->page_offset;
> > - xdp.data_meta = xdp.data;
> > - xdp.data_hard_start = xdp.data -
> > - ixgbe_rx_offset(rx_ring);
> > - xdp.data_end = xdp.data + size;
> > + unsigned int offset = ixgbe_rx_offset(rx_ring);
> > + unsigned char *hard_start;
> > +
> > + hard_start = page_address(rx_buffer->page) +
> > + rx_buffer->page_offset - offset;
> > + xdp_prepare_buff(&xdp, hard_start, offset, size);
> > #if (PAGE_SIZE > 4096)
> > /* At larger PAGE_SIZE, frame_sz depend on len size */
> > xdp.frame_sz = ixgbe_rx_frame_truesize(rx_ring, size);
> > diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
> > index 624efcd71569..51df79005ccb 100644
> > --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
> > +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c
> > @@ -1160,12 +1160,12 @@ static int ixgbevf_clean_rx_irq(struct ixgbevf_q_vector *q_vector,
> >
> > /* retrieve a buffer from the ring */
> > if (!skb) {
> > - xdp.data = page_address(rx_buffer->page) +
> > - rx_buffer->page_offset;
> > - xdp.data_meta = xdp.data;
> > - xdp.data_hard_start = xdp.data -
> > - ixgbevf_rx_offset(rx_ring);
> > - xdp.data_end = xdp.data + size;
> > + unsigned int offset = ixgbevf_rx_offset(rx_ring);
> > + unsigned char *hard_start;
> > +
> > + hard_start = page_address(rx_buffer->page) +
> > + rx_buffer->page_offset - offset;
> > + xdp_prepare_buff(&xdp, hard_start, offset, size);
> > #if (PAGE_SIZE > 4096)
> > /* At larger PAGE_SIZE, frame_sz depend on len size */
> > xdp.frame_sz = ixgbevf_rx_frame_truesize(rx_ring, size);
> > diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
> > index acbb9cb85ada..af6c9cf59809 100644
> > --- a/drivers/net/ethernet/marvell/mvneta.c
> > +++ b/drivers/net/ethernet/marvell/mvneta.c
> > @@ -2263,10 +2263,8 @@ mvneta_swbm_rx_frame(struct mvneta_port *pp,
> >
> > /* Prefetch header */
> > prefetch(data);
> > -
> > - xdp->data_hard_start = data;
> > - xdp->data = data + pp->rx_offset_correction + MVNETA_MH_SIZE;
> > - xdp->data_end = xdp->data + data_len;
> > + xdp_prepare_buff(xdp, data, pp->rx_offset_correction + MVNETA_MH_SIZE,
> > + data_len);
> > xdp_set_data_meta_invalid(xdp);
> >
> > sinfo = xdp_get_shared_info_from_buff(xdp);
> > diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> > index ca05dfc05058..8c2197b96515 100644
> > --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> > +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> > @@ -3564,16 +3564,15 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
> > if (xdp_prog) {
> > struct xdp_rxq_info *xdp_rxq;
> >
> > - xdp.data_hard_start = data;
> > - xdp.data = data + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM;
> > - xdp.data_end = xdp.data + rx_bytes;
> > -
> > if (bm_pool->pkt_size == MVPP2_BM_SHORT_PKT_SIZE)
> > xdp_rxq = &rxq->xdp_rxq_short;
> > else
> > xdp_rxq = &rxq->xdp_rxq_long;
> >
> > xdp_init_buff(&xdp, PAGE_SIZE, xdp_rxq);
> > + xdp_prepare_buff(&xdp, data,
> > + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM,
> > + rx_bytes);
> > xdp_set_data_meta_invalid(&xdp);
> >
> > ret = mvpp2_run_xdp(port, rxq, xdp_prog, &xdp, pp, &ps);
> > diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> > index 815381b484ca..86c63dedc689 100644
> > --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> > +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c
> > @@ -776,10 +776,9 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud
> > priv->frag_info[0].frag_size,
> > DMA_FROM_DEVICE);
> >
> > - xdp.data_hard_start = va - frags[0].page_offset;
> > - xdp.data = va;
> > + xdp_prepare_buff(&xdp, va - frags[0].page_offset,
> > + frags[0].page_offset, length);
> > xdp_set_data_meta_invalid(&xdp);
> > - xdp.data_end = xdp.data + length;
> > orig_data = xdp.data;
> >
> > act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> > index c68628b1f30b..a2f4f0ce427f 100644
> > --- a/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> > +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_rx.c
> > @@ -1128,10 +1128,8 @@ static void mlx5e_fill_xdp_buff(struct mlx5e_rq *rq, void *va, u16 headroom,
> > u32 len, struct xdp_buff *xdp)
> > {
> > xdp_init_buff(xdp, rq->buff.frame0_sz, &rq->xdp_rxq);
> > - xdp->data_hard_start = va;
> > - xdp->data = va + headroom;
> > + xdp_prepare_buff(xdp, va, headroom, len);
> > xdp_set_data_meta_invalid(xdp);
> > - xdp->data_end = xdp->data + len;
> > }
> >
> > static struct sk_buff *
> > diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
> > index 68e03e8257f2..5d0046c24b8c 100644
> > --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
> > +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c
> > @@ -1914,10 +1914,10 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget)
> > unsigned int dma_off;
> > int act;
> >
> > - xdp.data_hard_start = rxbuf->frag + NFP_NET_RX_BUF_HEADROOM;
> > - xdp.data = orig_data;
> > - xdp.data_meta = orig_data;
> > - xdp.data_end = orig_data + pkt_len;
> > + xdp_prepare_buff(&xdp,
> > + rxbuf->frag + NFP_NET_RX_BUF_HEADROOM,
> > + pkt_off - NFP_NET_RX_BUF_HEADROOM,
> > + pkt_len);
> >
> > act = bpf_prog_run_xdp(xdp_prog, &xdp);
> >
> > diff --git a/drivers/net/ethernet/qlogic/qede/qede_fp.c b/drivers/net/ethernet/qlogic/qede/qede_fp.c
> > index d40220043883..9c50df499046 100644
> > --- a/drivers/net/ethernet/qlogic/qede/qede_fp.c
> > +++ b/drivers/net/ethernet/qlogic/qede/qede_fp.c
> > @@ -1091,10 +1091,8 @@ static bool qede_rx_xdp(struct qede_dev *edev,
> > enum xdp_action act;
> >
> > xdp_init_buff(&xdp, rxq->rx_buf_seg_size, &rxq->xdp_rxq);
> > - xdp.data_hard_start = page_address(bd->data);
> > - xdp.data = xdp.data_hard_start + *data_offset;
> > + xdp_prepare_buff(&xdp, page_address(bd->data), *data_offset, *len);
> > xdp_set_data_meta_invalid(&xdp);
> > - xdp.data_end = xdp.data + *len;
> >
> > /* Queues always have a full reset currently, so for the time
> > * being until there's atomic program replace just mark read
> > diff --git a/drivers/net/ethernet/sfc/rx.c b/drivers/net/ethernet/sfc/rx.c
> > index eaa6650955d1..9015a1639234 100644
> > --- a/drivers/net/ethernet/sfc/rx.c
> > +++ b/drivers/net/ethernet/sfc/rx.c
> > @@ -294,12 +294,10 @@ static bool efx_do_xdp(struct efx_nic *efx, struct efx_channel *channel,
> > efx->rx_prefix_size);
> >
> > xdp_init_buff(&xdp, efx->rx_page_buf_step, &rx_queue->xdp_rxq_info);
> > - xdp.data = *ehp;
> > - xdp.data_hard_start = xdp.data - EFX_XDP_HEADROOM;
> > -
> > + xdp_prepare_buff(&xdp, *ehp - EFX_XDP_HEADROOM, EFX_XDP_HEADROOM,
> > + rx_buf->len);
> > /* No support yet for XDP metadata */
> > xdp_set_data_meta_invalid(&xdp);
> > - xdp.data_end = xdp.data + rx_buf->len;
> >
> > xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > rcu_read_unlock();
> > diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c
> > index 945ca9517bf9..80bb1a6612b1 100644
> > --- a/drivers/net/ethernet/socionext/netsec.c
> > +++ b/drivers/net/ethernet/socionext/netsec.c
> > @@ -1015,10 +1015,9 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget)
> > dma_dir);
> > prefetch(desc->addr);
> >
> > - xdp.data_hard_start = desc->addr;
> > - xdp.data = desc->addr + NETSEC_RXBUF_HEADROOM;
> > + xdp_prepare_buff(&xdp, desc->addr, NETSEC_RXBUF_HEADROOM,
> > + pkt_len);
> > xdp_set_data_meta_invalid(&xdp);
> > - xdp.data_end = xdp.data + pkt_len;
> >
> > if (xdp_prog) {
> > xdp_result = netsec_run_xdp(priv, xdp_prog, &xdp);
> > diff --git a/drivers/net/ethernet/ti/cpsw.c b/drivers/net/ethernet/ti/cpsw.c
> > index 78a923391828..c08fd6a6be9b 100644
> > --- a/drivers/net/ethernet/ti/cpsw.c
> > +++ b/drivers/net/ethernet/ti/cpsw.c
> > @@ -392,22 +392,17 @@ static void cpsw_rx_handler(void *token, int len, int status)
> > }
> >
> > if (priv->xdp_prog) {
> > - xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
> > + int headroom = CPSW_HEADROOM, size = len;
> >
> > + xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
> > if (status & CPDMA_RX_VLAN_ENCAP) {
> > - xdp.data = pa + CPSW_HEADROOM +
> > - CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> > - xdp.data_end = xdp.data + len -
> > - CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> > - } else {
> > - xdp.data = pa + CPSW_HEADROOM;
> > - xdp.data_end = xdp.data + len;
> > + headroom += CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> > + size -= CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> > }
> >
> > + xdp_prepare_buff(&xdp, pa, headroom, size);
> > xdp_set_data_meta_invalid(&xdp);
> >
> > - xdp.data_hard_start = pa;
> > -
> > port = priv->emac_port + cpsw->data.dual_emac;
> > ret = cpsw_run_xdp(priv, ch, &xdp, page, port);
> > if (ret != CPSW_XDP_PASS)
> > diff --git a/drivers/net/ethernet/ti/cpsw_new.c b/drivers/net/ethernet/ti/cpsw_new.c
> > index 1b3385ec9645..c74c997d1cf2 100644
> > --- a/drivers/net/ethernet/ti/cpsw_new.c
> > +++ b/drivers/net/ethernet/ti/cpsw_new.c
> > @@ -335,22 +335,17 @@ static void cpsw_rx_handler(void *token, int len, int status)
> > }
> >
> > if (priv->xdp_prog) {
> > - xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
> > + int headroom = CPSW_HEADROOM, size = len;
> >
> > + xdp_init_buff(&xdp, PAGE_SIZE, &priv->xdp_rxq[ch]);
> > if (status & CPDMA_RX_VLAN_ENCAP) {
> > - xdp.data = pa + CPSW_HEADROOM +
> > - CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> > - xdp.data_end = xdp.data + len -
> > - CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> > - } else {
> > - xdp.data = pa + CPSW_HEADROOM;
> > - xdp.data_end = xdp.data + len;
> > + headroom += CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> > + size -= CPSW_RX_VLAN_ENCAP_HDR_SIZE;
> > }
> >
> > + xdp_prepare_buff(&xdp, pa, headroom, size);
> > xdp_set_data_meta_invalid(&xdp);
> >
> > - xdp.data_hard_start = pa;
> > -
> > ret = cpsw_run_xdp(priv, ch, &xdp, page, priv->emac_port);
> > if (ret != CPSW_XDP_PASS)
> > goto requeue;
> > diff --git a/drivers/net/hyperv/netvsc_bpf.c b/drivers/net/hyperv/netvsc_bpf.c
> > index 14a7ee4c6899..93c202d6aff5 100644
> > --- a/drivers/net/hyperv/netvsc_bpf.c
> > +++ b/drivers/net/hyperv/netvsc_bpf.c
> > @@ -45,10 +45,8 @@ u32 netvsc_run_xdp(struct net_device *ndev, struct netvsc_channel *nvchan,
> > }
> >
> > xdp_init_buff(xdp, PAGE_SIZE, &nvchan->xdp_rxq);
> > - xdp->data_hard_start = page_address(page);
> > - xdp->data = xdp->data_hard_start + NETVSC_XDP_HDRM;
> > + xdp_prepare_buff(xdp, page_address(page), NETVSC_XDP_HDRM, len);
> > xdp_set_data_meta_invalid(xdp);
> > - xdp->data_end = xdp->data + len;
> >
> > memcpy(xdp->data, data, len);
> >
> > diff --git a/drivers/net/tun.c b/drivers/net/tun.c
> > index a82f7823d428..c7cbd058b345 100644
> > --- a/drivers/net/tun.c
> > +++ b/drivers/net/tun.c
> > @@ -1600,10 +1600,8 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun,
> > u32 act;
> >
> > xdp_init_buff(&xdp, buflen, &tfile->xdp_rxq);
> > - xdp.data_hard_start = buf;
> > - xdp.data = buf + pad;
> > + xdp_prepare_buff(&xdp, buf, pad, len);
> > xdp_set_data_meta_invalid(&xdp);
> > - xdp.data_end = xdp.data + len;
> >
> > act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > if (act == XDP_REDIRECT || act == XDP_TX) {
> > diff --git a/drivers/net/veth.c b/drivers/net/veth.c
> > index 25f3601fb6dd..30a7f2ad39c3 100644
> > --- a/drivers/net/veth.c
> > +++ b/drivers/net/veth.c
> > @@ -710,11 +710,7 @@ static struct sk_buff *veth_xdp_rcv_skb(struct veth_rq *rq,
> > skb = nskb;
> > }
> >
> > - xdp.data_hard_start = skb->head;
> > - xdp.data = skb_mac_header(skb);
> > - xdp.data_end = xdp.data + pktlen;
> > - xdp.data_meta = xdp.data;
> > -
> > + xdp_prepare_buff(&xdp, skb->head, skb->mac_header, pktlen);
> > /* SKB "head" area always have tailroom for skb_shared_info */
> > frame_sz = (void *)skb_end_pointer(skb) - xdp.data_hard_start;
> > frame_sz += SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
> > diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
> > index a22ce87bcd9c..e57b2d452cbc 100644
> > --- a/drivers/net/virtio_net.c
> > +++ b/drivers/net/virtio_net.c
> > @@ -690,10 +690,8 @@ static struct sk_buff *receive_small(struct net_device *dev,
> > }
> >
> > xdp_init_buff(&xdp, buflen, &rq->xdp_rxq);
> > - xdp.data_hard_start = buf + VIRTNET_RX_PAD + vi->hdr_len;
> > - xdp.data = xdp.data_hard_start + xdp_headroom;
> > - xdp.data_end = xdp.data + len;
> > - xdp.data_meta = xdp.data;
> > + xdp_prepare_buff(&xdp, buf + VIRTNET_RX_PAD + vi->hdr_len,
> > + xdp_headroom, len);
> > orig_data = xdp.data;
> > act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > stats->xdp_packets++;
> > @@ -859,10 +857,8 @@ static struct sk_buff *receive_mergeable(struct net_device *dev,
> > */
> > data = page_address(xdp_page) + offset;
> > xdp_init_buff(&xdp, frame_sz - vi->hdr_len, &rq->xdp_rxq);
> > - xdp.data_hard_start = data - VIRTIO_XDP_HEADROOM + vi->hdr_len;
> > - xdp.data = data + vi->hdr_len;
> > - xdp.data_end = xdp.data + (len - vi->hdr_len);
> > - xdp.data_meta = xdp.data;
> > + xdp_prepare_buff(&xdp, data - VIRTIO_XDP_HEADROOM + vi->hdr_len,
> > + VIRTIO_XDP_HEADROOM, len - vi->hdr_len);
> >
> > act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > stats->xdp_packets++;
> > diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
> > index 329397c60d84..61d3f5f8b7f3 100644
> > --- a/drivers/net/xen-netfront.c
> > +++ b/drivers/net/xen-netfront.c
> > @@ -866,10 +866,8 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
> >
> > xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
> > &queue->xdp_rxq);
> > - xdp->data_hard_start = page_address(pdata);
> > - xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
> > + xdp_prepare_buff(xdp, page_address(pdata), XDP_PACKET_HEADROOM, len);
> > xdp_set_data_meta_invalid(xdp);
> > - xdp->data_end = xdp->data + len;
> >
> > act = bpf_prog_run_xdp(prog, xdp);
> > switch (act) {
> > diff --git a/include/net/xdp.h b/include/net/xdp.h
> > index 3fb3a9aa1b71..66d8a4b317a3 100644
> > --- a/include/net/xdp.h
> > +++ b/include/net/xdp.h
> > @@ -83,6 +83,18 @@ xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
> > xdp->rxq = rxq;
> > }
> >
> > +static inline void
> > +xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> > + int headroom, int data_len)
> > +{
> > + unsigned char *data = hard_start + headroom;
> > +
> > + xdp->data_hard_start = hard_start;
> > + xdp->data = data;
> > + xdp->data_end = data + data_len;
> > + xdp->data_meta = data;
> > +}
> > +
> > /* Reserve memory area at end-of data area.
> > *
> > * This macro reserves tailroom in the XDP buffer by limiting the
> > diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c
> > index a8fa5a9e4137..fe5a80d396e3 100644
> > --- a/net/bpf/test_run.c
> > +++ b/net/bpf/test_run.c
> > @@ -636,10 +636,7 @@ int bpf_prog_test_run_xdp(struct bpf_prog *prog, const union bpf_attr *kattr,
> > if (IS_ERR(data))
> > return PTR_ERR(data);
> >
> > - xdp.data_hard_start = data;
> > - xdp.data = data + headroom;
> > - xdp.data_meta = xdp.data;
> > - xdp.data_end = xdp.data + size;
> > + xdp_prepare_buff(&xdp, data, headroom, size);
> >
> > rxqueue = __netif_get_rx_queue(current->nsproxy->net_ns->loopback_dev, 0);
> > xdp_init_buff(&xdp, headroom + max_data_sz + tailroom,
> > diff --git a/net/core/dev.c b/net/core/dev.c
> > index bac56afcf6bc..2997177876cc 100644
> > --- a/net/core/dev.c
> > +++ b/net/core/dev.c
> > @@ -4592,7 +4592,7 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
> > __be16 orig_eth_type;
> > struct ethhdr *eth;
> > bool orig_bcast;
> > - int hlen, off;
> > + int off;
> >
> > /* Reinjected packets coming from act_mirred or similar should
> > * not get XDP generic processing.
> > @@ -4624,11 +4624,9 @@ static u32 netif_receive_generic_xdp(struct sk_buff *skb,
> > * header.
> > */
> > mac_len = skb->data - skb_mac_header(skb);
> > - hlen = skb_headlen(skb) + mac_len;
> > - xdp->data = skb->data - mac_len;
> > - xdp->data_meta = xdp->data;
> > - xdp->data_end = xdp->data + hlen;
> > - xdp->data_hard_start = skb->data - skb_headroom(skb);
> > + xdp_prepare_buff(xdp, skb->data - skb_headroom(skb),
> > + skb_headroom(skb) - mac_len,
> > + skb_headlen(skb) + mac_len);
> >
> > /* SKB "head" area always have tailroom for skb_shared_info */
> > frame_sz = (void *)skb_end_pointer(skb) - xdp->data_hard_start;
> > --
> > 2.29.2
> >
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-15 13:47 ` Lorenzo Bianconi
@ 2020-12-15 14:51 ` Daniel Borkmann
2020-12-15 15:06 ` Lorenzo Bianconi
2020-12-16 8:52 ` Jesper Dangaard Brouer
1 sibling, 1 reply; 19+ messages in thread
From: Daniel Borkmann @ 2020-12-15 14:51 UTC (permalink / raw)
To: Lorenzo Bianconi, Maciej Fijalkowski
Cc: Lorenzo Bianconi, bpf, netdev, davem, kuba, ast, brouer,
alexander.duyck, saeed
On 12/15/20 2:47 PM, Lorenzo Bianconi wrote:
[...]
>>> diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
>>> index 329397c60d84..61d3f5f8b7f3 100644
>>> --- a/drivers/net/xen-netfront.c
>>> +++ b/drivers/net/xen-netfront.c
>>> @@ -866,10 +866,8 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
>>>
>>> xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
>>> &queue->xdp_rxq);
>>> - xdp->data_hard_start = page_address(pdata);
>>> - xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
>>> + xdp_prepare_buff(xdp, page_address(pdata), XDP_PACKET_HEADROOM, len);
>>> xdp_set_data_meta_invalid(xdp);
>>> - xdp->data_end = xdp->data + len;
>>>
>>> act = bpf_prog_run_xdp(prog, xdp);
>>> switch (act) {
>>> diff --git a/include/net/xdp.h b/include/net/xdp.h
>>> index 3fb3a9aa1b71..66d8a4b317a3 100644
>>> --- a/include/net/xdp.h
>>> +++ b/include/net/xdp.h
>>> @@ -83,6 +83,18 @@ xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
>>> xdp->rxq = rxq;
>>> }
>>>
>>> +static inline void
nit: maybe __always_inline
>>> +xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
>>> + int headroom, int data_len)
>>> +{
>>> + unsigned char *data = hard_start + headroom;
>>> +
>>> + xdp->data_hard_start = hard_start;
>>> + xdp->data = data;
>>> + xdp->data_end = data + data_len;
>>> + xdp->data_meta = data;
>>> +}
>>> +
>>> /* Reserve memory area at end-of data area.
>>> *
For the drivers with xdp_set_data_meta_invalid(), we're basically setting xdp->data_meta
twice unless compiler is smart enough to optimize the first one away (did you double check?).
Given this is supposed to be a cleanup, why not integrate this logic as well so the
xdp_set_data_meta_invalid() doesn't get extra treatment?
Thanks,
Daniel
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-15 14:51 ` Daniel Borkmann
@ 2020-12-15 15:06 ` Lorenzo Bianconi
2020-12-15 15:13 ` Maciej Fijalkowski
0 siblings, 1 reply; 19+ messages in thread
From: Lorenzo Bianconi @ 2020-12-15 15:06 UTC (permalink / raw)
To: Daniel Borkmann
Cc: Maciej Fijalkowski, Lorenzo Bianconi, bpf, netdev, davem, kuba,
ast, brouer, alexander.duyck, saeed
[-- Attachment #1: Type: text/plain, Size: 2509 bytes --]
> On 12/15/20 2:47 PM, Lorenzo Bianconi wrote:
> [...]
> > > > diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
> > > > index 329397c60d84..61d3f5f8b7f3 100644
> > > > --- a/drivers/net/xen-netfront.c
> > > > +++ b/drivers/net/xen-netfront.c
> > > > @@ -866,10 +866,8 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
> > > > xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
> > > > &queue->xdp_rxq);
> > > > - xdp->data_hard_start = page_address(pdata);
> > > > - xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
> > > > + xdp_prepare_buff(xdp, page_address(pdata), XDP_PACKET_HEADROOM, len);
> > > > xdp_set_data_meta_invalid(xdp);
> > > > - xdp->data_end = xdp->data + len;
> > > > act = bpf_prog_run_xdp(prog, xdp);
> > > > switch (act) {
> > > > diff --git a/include/net/xdp.h b/include/net/xdp.h
> > > > index 3fb3a9aa1b71..66d8a4b317a3 100644
> > > > --- a/include/net/xdp.h
> > > > +++ b/include/net/xdp.h
> > > > @@ -83,6 +83,18 @@ xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
> > > > xdp->rxq = rxq;
> > > > }
> > > > +static inline void
>
> nit: maybe __always_inline
ack, I will add in v4
>
> > > > +xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> > > > + int headroom, int data_len)
> > > > +{
> > > > + unsigned char *data = hard_start + headroom;
> > > > +
> > > > + xdp->data_hard_start = hard_start;
> > > > + xdp->data = data;
> > > > + xdp->data_end = data + data_len;
> > > > + xdp->data_meta = data;
> > > > +}
> > > > +
> > > > /* Reserve memory area at end-of data area.
> > > > *
>
> For the drivers with xdp_set_data_meta_invalid(), we're basically setting xdp->data_meta
> twice unless compiler is smart enough to optimize the first one away (did you double check?).
> Given this is supposed to be a cleanup, why not integrate this logic as well so the
> xdp_set_data_meta_invalid() doesn't get extra treatment?
we discussed it before, but I am fine to add it in v4. Something like:
static __always_inline void
xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
int headroom, int data_len, bool meta_valid)
{
unsigned char *data = hard_start + headroom;
xdp->data_hard_start = hard_start;
xdp->data = data;
xdp->data_end = data + data_len;
xdp->data_meta = meta_valid ? data : data + 1;
}
Regards,
Lorenzo
>
> Thanks,
> Daniel
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-15 15:06 ` Lorenzo Bianconi
@ 2020-12-15 15:13 ` Maciej Fijalkowski
2020-12-15 20:36 ` Lorenzo Bianconi
2020-12-16 8:30 ` Jesper Dangaard Brouer
0 siblings, 2 replies; 19+ messages in thread
From: Maciej Fijalkowski @ 2020-12-15 15:13 UTC (permalink / raw)
To: Lorenzo Bianconi
Cc: Daniel Borkmann, Lorenzo Bianconi, bpf, netdev, davem, kuba, ast,
brouer, alexander.duyck, saeed
On Tue, Dec 15, 2020 at 04:06:20PM +0100, Lorenzo Bianconi wrote:
> > On 12/15/20 2:47 PM, Lorenzo Bianconi wrote:
> > [...]
> > > > > diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
> > > > > index 329397c60d84..61d3f5f8b7f3 100644
> > > > > --- a/drivers/net/xen-netfront.c
> > > > > +++ b/drivers/net/xen-netfront.c
> > > > > @@ -866,10 +866,8 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
> > > > > xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
> > > > > &queue->xdp_rxq);
> > > > > - xdp->data_hard_start = page_address(pdata);
> > > > > - xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
> > > > > + xdp_prepare_buff(xdp, page_address(pdata), XDP_PACKET_HEADROOM, len);
> > > > > xdp_set_data_meta_invalid(xdp);
> > > > > - xdp->data_end = xdp->data + len;
> > > > > act = bpf_prog_run_xdp(prog, xdp);
> > > > > switch (act) {
> > > > > diff --git a/include/net/xdp.h b/include/net/xdp.h
> > > > > index 3fb3a9aa1b71..66d8a4b317a3 100644
> > > > > --- a/include/net/xdp.h
> > > > > +++ b/include/net/xdp.h
> > > > > @@ -83,6 +83,18 @@ xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
> > > > > xdp->rxq = rxq;
> > > > > }
> > > > > +static inline void
> >
> > nit: maybe __always_inline
>
> ack, I will add in v4
>
> >
> > > > > +xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> > > > > + int headroom, int data_len)
> > > > > +{
> > > > > + unsigned char *data = hard_start + headroom;
> > > > > +
> > > > > + xdp->data_hard_start = hard_start;
> > > > > + xdp->data = data;
> > > > > + xdp->data_end = data + data_len;
> > > > > + xdp->data_meta = data;
> > > > > +}
> > > > > +
> > > > > /* Reserve memory area at end-of data area.
> > > > > *
> >
> > For the drivers with xdp_set_data_meta_invalid(), we're basically setting xdp->data_meta
> > twice unless compiler is smart enough to optimize the first one away (did you double check?).
> > Given this is supposed to be a cleanup, why not integrate this logic as well so the
> > xdp_set_data_meta_invalid() doesn't get extra treatment?
That's what I was trying to say previously.
>
> we discussed it before, but I am fine to add it in v4. Something like:
>
> static __always_inline void
> xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> int headroom, int data_len, bool meta_valid)
> {
> unsigned char *data = hard_start + headroom;
>
> xdp->data_hard_start = hard_start;
> xdp->data = data;
> xdp->data_end = data + data_len;
> xdp->data_meta = meta_valid ? data : data + 1;
This will introduce branch, so for intel drivers we're getting the
overhead of one add and a branch. I'm still opting for a separate helper.
static __always_inline void
xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
int headroom, int data_len)
{
unsigned char *data = hard_start + headroom;
xdp->data_hard_start = hard_start;
xdp->data = data;
xdp->data_end = data + data_len;
xdp_set_data_meta_invalid(xdp);
}
static __always_inline void
xdp_prepare_buff_meta(struct xdp_buff *xdp, unsigned char *hard_start,
int headroom, int data_len)
{
unsigned char *data = hard_start + headroom;
xdp->data_hard_start = hard_start;
xdp->data = data;
xdp->data_end = data + data_len;
xdp->data_meta = data;
}
> }
>
> Regards,
> Lorenzo
>
> >
> > Thanks,
> > Daniel
> >
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-15 15:13 ` Maciej Fijalkowski
@ 2020-12-15 20:36 ` Lorenzo Bianconi
2020-12-16 8:30 ` Jesper Dangaard Brouer
1 sibling, 0 replies; 19+ messages in thread
From: Lorenzo Bianconi @ 2020-12-15 20:36 UTC (permalink / raw)
To: Maciej Fijalkowski
Cc: Lorenzo Bianconi, Daniel Borkmann, bpf, netdev, davem, kuba, ast,
brouer, alexander.duyck, saeed
[-- Attachment #1: Type: text/plain, Size: 4204 bytes --]
> On Tue, Dec 15, 2020 at 04:06:20PM +0100, Lorenzo Bianconi wrote:
> > > On 12/15/20 2:47 PM, Lorenzo Bianconi wrote:
> > > [...]
> > > > > > diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
> > > > > > index 329397c60d84..61d3f5f8b7f3 100644
> > > > > > --- a/drivers/net/xen-netfront.c
> > > > > > +++ b/drivers/net/xen-netfront.c
> > > > > > @@ -866,10 +866,8 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
> > > > > > xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
> > > > > > &queue->xdp_rxq);
> > > > > > - xdp->data_hard_start = page_address(pdata);
> > > > > > - xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
> > > > > > + xdp_prepare_buff(xdp, page_address(pdata), XDP_PACKET_HEADROOM, len);
> > > > > > xdp_set_data_meta_invalid(xdp);
> > > > > > - xdp->data_end = xdp->data + len;
> > > > > > act = bpf_prog_run_xdp(prog, xdp);
> > > > > > switch (act) {
> > > > > > diff --git a/include/net/xdp.h b/include/net/xdp.h
> > > > > > index 3fb3a9aa1b71..66d8a4b317a3 100644
> > > > > > --- a/include/net/xdp.h
> > > > > > +++ b/include/net/xdp.h
> > > > > > @@ -83,6 +83,18 @@ xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
> > > > > > xdp->rxq = rxq;
> > > > > > }
> > > > > > +static inline void
> > >
> > > nit: maybe __always_inline
> >
> > ack, I will add in v4
> >
> > >
> > > > > > +xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> > > > > > + int headroom, int data_len)
> > > > > > +{
> > > > > > + unsigned char *data = hard_start + headroom;
> > > > > > +
> > > > > > + xdp->data_hard_start = hard_start;
> > > > > > + xdp->data = data;
> > > > > > + xdp->data_end = data + data_len;
> > > > > > + xdp->data_meta = data;
> > > > > > +}
> > > > > > +
> > > > > > /* Reserve memory area at end-of data area.
> > > > > > *
> > >
> > > For the drivers with xdp_set_data_meta_invalid(), we're basically setting xdp->data_meta
> > > twice unless compiler is smart enough to optimize the first one away (did you double check?).
> > > Given this is supposed to be a cleanup, why not integrate this logic as well so the
> > > xdp_set_data_meta_invalid() doesn't get extra treatment?
>
> That's what I was trying to say previously.
>
> >
> > we discussed it before, but I am fine to add it in v4. Something like:
> >
> > static __always_inline void
> > xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> > int headroom, int data_len, bool meta_valid)
> > {
> > unsigned char *data = hard_start + headroom;
> >
> > xdp->data_hard_start = hard_start;
> > xdp->data = data;
> > xdp->data_end = data + data_len;
> > xdp->data_meta = meta_valid ? data : data + 1;
>
> This will introduce branch, so for intel drivers we're getting the
> overhead of one add and a branch. I'm still opting for a separate helper.
>
> static __always_inline void
> xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> int headroom, int data_len)
> {
> unsigned char *data = hard_start + headroom;
>
> xdp->data_hard_start = hard_start;
> xdp->data = data;
> xdp->data_end = data + data_len;
> xdp_set_data_meta_invalid(xdp);
> }
>
> static __always_inline void
> xdp_prepare_buff_meta(struct xdp_buff *xdp, unsigned char *hard_start,
> int headroom, int data_len)
> {
> unsigned char *data = hard_start + headroom;
>
> xdp->data_hard_start = hard_start;
> xdp->data = data;
> xdp->data_end = data + data_len;
> xdp->data_meta = data;
> }
yes, to follow-up the possible approaches we have here are:
- have 2 different helpers (xdp_prepare_buff_meta and xdp_prepare_buff) as
suggested by Maciej
- move the data_meta initialization out of the helper and do it in each
driver
- use the current approach and overwrite data_meta with
xdp_set_data_meta_invalid() when necessary
- introduce a branch in order to have just one helper
what is the best for you?
Regards,
Lorenzo
>
> > }
> >
> > Regards,
> > Lorenzo
> >
> > >
> > > Thanks,
> > > Daniel
> > >
>
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-15 15:13 ` Maciej Fijalkowski
2020-12-15 20:36 ` Lorenzo Bianconi
@ 2020-12-16 8:30 ` Jesper Dangaard Brouer
1 sibling, 0 replies; 19+ messages in thread
From: Jesper Dangaard Brouer @ 2020-12-16 8:30 UTC (permalink / raw)
To: Maciej Fijalkowski
Cc: Lorenzo Bianconi, Daniel Borkmann, Lorenzo Bianconi, bpf, netdev,
davem, kuba, ast, alexander.duyck, saeed, brouer
On Tue, 15 Dec 2020 16:13:44 +0100
Maciej Fijalkowski <maciej.fijalkowski@intel.com> wrote:
> On Tue, Dec 15, 2020 at 04:06:20PM +0100, Lorenzo Bianconi wrote:
> > > On 12/15/20 2:47 PM, Lorenzo Bianconi wrote:
> > > [...]
> > > > > > diff --git a/drivers/net/xen-netfront.c b/drivers/net/xen-netfront.c
> > > > > > index 329397c60d84..61d3f5f8b7f3 100644
> > > > > > --- a/drivers/net/xen-netfront.c
> > > > > > +++ b/drivers/net/xen-netfront.c
> > > > > > @@ -866,10 +866,8 @@ static u32 xennet_run_xdp(struct netfront_queue *queue, struct page *pdata,
> > > > > > xdp_init_buff(xdp, XEN_PAGE_SIZE - XDP_PACKET_HEADROOM,
> > > > > > &queue->xdp_rxq);
> > > > > > - xdp->data_hard_start = page_address(pdata);
> > > > > > - xdp->data = xdp->data_hard_start + XDP_PACKET_HEADROOM;
> > > > > > + xdp_prepare_buff(xdp, page_address(pdata), XDP_PACKET_HEADROOM, len);
> > > > > > xdp_set_data_meta_invalid(xdp);
> > > > > > - xdp->data_end = xdp->data + len;
> > > > > > act = bpf_prog_run_xdp(prog, xdp);
> > > > > > switch (act) {
> > > > > > diff --git a/include/net/xdp.h b/include/net/xdp.h
> > > > > > index 3fb3a9aa1b71..66d8a4b317a3 100644
> > > > > > --- a/include/net/xdp.h
> > > > > > +++ b/include/net/xdp.h
> > > > > > @@ -83,6 +83,18 @@ xdp_init_buff(struct xdp_buff *xdp, u32 frame_sz, struct xdp_rxq_info *rxq)
> > > > > > xdp->rxq = rxq;
> > > > > > }
> > > > > > +static inline void
> > >
> > > nit: maybe __always_inline
> >
> > ack, I will add in v4
> >
> > >
> > > > > > +xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> > > > > > + int headroom, int data_len)
> > > > > > +{
> > > > > > + unsigned char *data = hard_start + headroom;
> > > > > > +
> > > > > > + xdp->data_hard_start = hard_start;
> > > > > > + xdp->data = data;
> > > > > > + xdp->data_end = data + data_len;
> > > > > > + xdp->data_meta = data;
> > > > > > +}
> > > > > > +
> > > > > > /* Reserve memory area at end-of data area.
> > > > > > *
> > >
> > > For the drivers with xdp_set_data_meta_invalid(), we're basically setting xdp->data_meta
> > > twice unless compiler is smart enough to optimize the first one away (did you double check?).
> > > Given this is supposed to be a cleanup, why not integrate this logic as well so the
> > > xdp_set_data_meta_invalid() doesn't get extra treatment?
>
> That's what I was trying to say previously.
>
> >
> > we discussed it before, but I am fine to add it in v4. Something like:
> >
> > static __always_inline void
> > xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> > int headroom, int data_len, bool meta_valid)
> > {
> > unsigned char *data = hard_start + headroom;
> >
> > xdp->data_hard_start = hard_start;
> > xdp->data = data;
> > xdp->data_end = data + data_len;
> > xdp->data_meta = meta_valid ? data : data + 1;
>
> This will introduce branch, so for intel drivers we're getting the
> overhead of one add and a branch. I'm still opting for a separate helper.
I should think, as this gets inlined the compiler should be able to
remove the branch. I assume that the usage of 'meta_valid' will be a
const in the drivers. Maybe we should have the API be 'const bool meta_valid'?
> static __always_inline void
> xdp_prepare_buff(struct xdp_buff *xdp, unsigned char *hard_start,
> int headroom, int data_len)
> {
> unsigned char *data = hard_start + headroom;
>
> xdp->data_hard_start = hard_start;
> xdp->data = data;
> xdp->data_end = data + data_len;
> xdp_set_data_meta_invalid(xdp);
> }
>
> static __always_inline void
> xdp_prepare_buff_meta(struct xdp_buff *xdp, unsigned char *hard_start,
> int headroom, int data_len)
> {
> unsigned char *data = hard_start + headroom;
>
> xdp->data_hard_start = hard_start;
> xdp->data = data;
> xdp->data_end = data + data_len;
> xdp->data_meta = data;
> }
Thanks to you Maciej for reviewing this! :-)
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 1/2] net: xdp: introduce xdp_init_buff utility routine
2020-12-12 17:41 ` [PATCH v3 bpf-next 1/2] net: xdp: introduce xdp_init_buff utility routine Lorenzo Bianconi
@ 2020-12-16 8:35 ` Jesper Dangaard Brouer
2020-12-16 14:56 ` Lorenzo Bianconi
0 siblings, 1 reply; 19+ messages in thread
From: Jesper Dangaard Brouer @ 2020-12-16 8:35 UTC (permalink / raw)
To: Lorenzo Bianconi
Cc: bpf, netdev, davem, kuba, ast, daniel, lorenzo.bianconi,
alexander.duyck, maciej.fijalkowski, saeed, brouer
On Sat, 12 Dec 2020 18:41:48 +0100
Lorenzo Bianconi <lorenzo@kernel.org> wrote:
> diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> index fcc262064766..b7942c3440c0 100644
> --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> @@ -133,12 +133,11 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
> dma_sync_single_for_cpu(&pdev->dev, mapping + offset, *len, bp->rx_dir);
>
> txr = rxr->bnapi->tx_ring;
> + xdp_init_buff(&xdp, PAGE_SIZE, &rxr->xdp_rxq);
> xdp.data_hard_start = *data_ptr - offset;
> xdp.data = *data_ptr;
> xdp_set_data_meta_invalid(&xdp);
> xdp.data_end = *data_ptr + *len;
> - xdp.rxq = &rxr->xdp_rxq;
> - xdp.frame_sz = PAGE_SIZE; /* BNXT_RX_PAGE_MODE(bp) when XDP enabled */
> orig_data = xdp.data;
I don't like loosing the comment here. Other developers reading this
code might assume that size is always PAGE_SIZE, which is only the case
when XDP is enabled. Lets save them from making this mistake.
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-15 13:47 ` Lorenzo Bianconi
2020-12-15 14:51 ` Daniel Borkmann
@ 2020-12-16 8:52 ` Jesper Dangaard Brouer
2020-12-16 15:01 ` Lorenzo Bianconi
1 sibling, 1 reply; 19+ messages in thread
From: Jesper Dangaard Brouer @ 2020-12-16 8:52 UTC (permalink / raw)
To: Lorenzo Bianconi
Cc: Maciej Fijalkowski, Lorenzo Bianconi, bpf, netdev, davem, kuba,
ast, daniel, alexander.duyck, saeed, brouer
On Tue, 15 Dec 2020 14:47:10 +0100
Lorenzo Bianconi <lorenzo.bianconi@redhat.com> wrote:
> [...]
> > > xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > > diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > index 4dbbbd49c389..fcd1ca3343fb 100644
> > > --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > @@ -2393,12 +2393,12 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
> > >
> > > /* retrieve a buffer from the ring */
> > > if (!skb) {
> > > - xdp.data = page_address(rx_buffer->page) +
> > > - rx_buffer->page_offset;
> > > - xdp.data_meta = xdp.data;
> > > - xdp.data_hard_start = xdp.data -
> > > - i40e_rx_offset(rx_ring);
> > > - xdp.data_end = xdp.data + size;
> > > + unsigned int offset = i40e_rx_offset(rx_ring);
> >
> > I now see that we could call the i40e_rx_offset() once per napi, so can
> > you pull this variable out and have it initialized a single time? Applies
> > to other intel drivers as well.
>
> ack, fine. I will fix in v4.
Be careful with the Intel drivers. They have two modes (at compile
time) depending on PAGE_SIZE in system. In one of the modes (default
one) you can place init of xdp.frame_sz outside the NAPI loop and init a
single time. In the other mode you cannot, and it becomes dynamic per
packet. Intel review this carefully, please!
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 1/2] net: xdp: introduce xdp_init_buff utility routine
2020-12-16 8:35 ` Jesper Dangaard Brouer
@ 2020-12-16 14:56 ` Lorenzo Bianconi
0 siblings, 0 replies; 19+ messages in thread
From: Lorenzo Bianconi @ 2020-12-16 14:56 UTC (permalink / raw)
To: Jesper Dangaard Brouer
Cc: Lorenzo Bianconi, bpf, netdev, davem, kuba, ast, daniel,
alexander.duyck, maciej.fijalkowski, saeed
[-- Attachment #1: Type: text/plain, Size: 1369 bytes --]
> On Sat, 12 Dec 2020 18:41:48 +0100
> Lorenzo Bianconi <lorenzo@kernel.org> wrote:
>
> > diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> > index fcc262064766..b7942c3440c0 100644
> > --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> > +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c
> > @@ -133,12 +133,11 @@ bool bnxt_rx_xdp(struct bnxt *bp, struct bnxt_rx_ring_info *rxr, u16 cons,
> > dma_sync_single_for_cpu(&pdev->dev, mapping + offset, *len, bp->rx_dir);
> >
> > txr = rxr->bnapi->tx_ring;
> > + xdp_init_buff(&xdp, PAGE_SIZE, &rxr->xdp_rxq);
> > xdp.data_hard_start = *data_ptr - offset;
> > xdp.data = *data_ptr;
> > xdp_set_data_meta_invalid(&xdp);
> > xdp.data_end = *data_ptr + *len;
> > - xdp.rxq = &rxr->xdp_rxq;
> > - xdp.frame_sz = PAGE_SIZE; /* BNXT_RX_PAGE_MODE(bp) when XDP enabled */
> > orig_data = xdp.data;
>
> I don't like loosing the comment here. Other developers reading this
> code might assume that size is always PAGE_SIZE, which is only the case
> when XDP is enabled. Lets save them from making this mistake.
ack, I will add it back in v4.
Regards,
Lorenzo
>
> --
> Best regards,
> Jesper Dangaard Brouer
> MSc.CS, Principal Kernel Engineer at Red Hat
> LinkedIn: http://www.linkedin.com/in/brouer
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-16 8:52 ` Jesper Dangaard Brouer
@ 2020-12-16 15:01 ` Lorenzo Bianconi
2020-12-17 18:16 ` Saeed Mahameed
0 siblings, 1 reply; 19+ messages in thread
From: Lorenzo Bianconi @ 2020-12-16 15:01 UTC (permalink / raw)
To: Jesper Dangaard Brouer
Cc: Maciej Fijalkowski, Lorenzo Bianconi, bpf, netdev, davem, kuba,
ast, daniel, alexander.duyck, saeed
[-- Attachment #1: Type: text/plain, Size: 1848 bytes --]
> On Tue, 15 Dec 2020 14:47:10 +0100
> Lorenzo Bianconi <lorenzo.bianconi@redhat.com> wrote:
>
> > [...]
> > > > xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > > > diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > index 4dbbbd49c389..fcd1ca3343fb 100644
> > > > --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > @@ -2393,12 +2393,12 @@ static int i40e_clean_rx_irq(struct i40e_ring *rx_ring, int budget)
> > > >
> > > > /* retrieve a buffer from the ring */
> > > > if (!skb) {
> > > > - xdp.data = page_address(rx_buffer->page) +
> > > > - rx_buffer->page_offset;
> > > > - xdp.data_meta = xdp.data;
> > > > - xdp.data_hard_start = xdp.data -
> > > > - i40e_rx_offset(rx_ring);
> > > > - xdp.data_end = xdp.data + size;
> > > > + unsigned int offset = i40e_rx_offset(rx_ring);
> > >
> > > I now see that we could call the i40e_rx_offset() once per napi, so can
> > > you pull this variable out and have it initialized a single time? Applies
> > > to other intel drivers as well.
> >
> > ack, fine. I will fix in v4.
>
> Be careful with the Intel drivers. They have two modes (at compile
> time) depending on PAGE_SIZE in system. In one of the modes (default
> one) you can place init of xdp.frame_sz outside the NAPI loop and init a
> single time. In the other mode you cannot, and it becomes dynamic per
> packet. Intel review this carefully, please!
ack. Actully I kept the xdp.frame_sz configuration in the NAPI loop but
an Intel review will be nice.
Regards,
Lorenzo
>
> --
> Best regards,
> Jesper Dangaard Brouer
> MSc.CS, Principal Kernel Engineer at Red Hat
> LinkedIn: http://www.linkedin.com/in/brouer
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-16 15:01 ` Lorenzo Bianconi
@ 2020-12-17 18:16 ` Saeed Mahameed
2020-12-17 18:28 ` Maciej Fijalkowski
0 siblings, 1 reply; 19+ messages in thread
From: Saeed Mahameed @ 2020-12-17 18:16 UTC (permalink / raw)
To: Lorenzo Bianconi, Jesper Dangaard Brouer
Cc: Maciej Fijalkowski, Lorenzo Bianconi, bpf, netdev, davem, kuba,
ast, daniel, alexander.duyck
On Wed, 2020-12-16 at 16:01 +0100, Lorenzo Bianconi wrote:
> > On Tue, 15 Dec 2020 14:47:10 +0100
> > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> wrote:
> >
> > > [...]
> > > > > xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > > > > diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > index 4dbbbd49c389..fcd1ca3343fb 100644
> > > > > --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > @@ -2393,12 +2393,12 @@ static int i40e_clean_rx_irq(struct
> > > > > i40e_ring *rx_ring, int budget)
> > > > >
> > > > > /* retrieve a buffer from the ring */
> > > > > if (!skb) {
> > > > > - xdp.data = page_address(rx_buffer-
> > > > > >page) +
> > > > > - rx_buffer->page_offset;
> > > > > - xdp.data_meta = xdp.data;
> > > > > - xdp.data_hard_start = xdp.data -
> > > > > - i40e_rx_offset(rx
> > > > > _ring);
> > > > > - xdp.data_end = xdp.data + size;
> > > > > + unsigned int offset =
> > > > > i40e_rx_offset(rx_ring);
> > > >
> > > > I now see that we could call the i40e_rx_offset() once per
> > > > napi, so can
> > > > you pull this variable out and have it initialized a single
> > > > time? Applies
> > > > to other intel drivers as well.
> > >
How is this related to this series? i suggest to keep this series clean
of vendor specific unrelated optimizations, this must be done in a
separate patchset.
> > > ack, fine. I will fix in v4.
> >
> > Be careful with the Intel drivers. They have two modes (at compile
> > time) depending on PAGE_SIZE in system. In one of the modes
> > (default
> > one) you can place init of xdp.frame_sz outside the NAPI loop and
> > init a
> > single time. In the other mode you cannot, and it becomes dynamic
> > per
> > packet. Intel review this carefully, please!
>
> ack. Actully I kept the xdp.frame_sz configuration in the NAPI loop
> but
> an Intel review will be nice.
>
> Regards,
> Lorenzo
>
> > --
> > Best regards,
> > Jesper Dangaard Brouer
> > MSc.CS, Principal Kernel Engineer at Red Hat
> > LinkedIn: http://www.linkedin.com/in/brouer
> >
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-17 18:16 ` Saeed Mahameed
@ 2020-12-17 18:28 ` Maciej Fijalkowski
2020-12-17 20:31 ` Saeed Mahameed
0 siblings, 1 reply; 19+ messages in thread
From: Maciej Fijalkowski @ 2020-12-17 18:28 UTC (permalink / raw)
To: Saeed Mahameed
Cc: Lorenzo Bianconi, Jesper Dangaard Brouer, Lorenzo Bianconi, bpf,
netdev, davem, kuba, ast, daniel, alexander.duyck
On Thu, Dec 17, 2020 at 10:16:06AM -0800, Saeed Mahameed wrote:
> On Wed, 2020-12-16 at 16:01 +0100, Lorenzo Bianconi wrote:
> > > On Tue, 15 Dec 2020 14:47:10 +0100
> > > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> wrote:
> > >
> > > > [...]
> > > > > > xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > > > > > diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > > b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > > index 4dbbbd49c389..fcd1ca3343fb 100644
> > > > > > --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > > +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > > @@ -2393,12 +2393,12 @@ static int i40e_clean_rx_irq(struct
> > > > > > i40e_ring *rx_ring, int budget)
> > > > > >
> > > > > > /* retrieve a buffer from the ring */
> > > > > > if (!skb) {
> > > > > > - xdp.data = page_address(rx_buffer-
> > > > > > >page) +
> > > > > > - rx_buffer->page_offset;
> > > > > > - xdp.data_meta = xdp.data;
> > > > > > - xdp.data_hard_start = xdp.data -
> > > > > > - i40e_rx_offset(rx
> > > > > > _ring);
> > > > > > - xdp.data_end = xdp.data + size;
> > > > > > + unsigned int offset =
> > > > > > i40e_rx_offset(rx_ring);
> > > > >
> > > > > I now see that we could call the i40e_rx_offset() once per
> > > > > napi, so can
> > > > > you pull this variable out and have it initialized a single
> > > > > time? Applies
> > > > > to other intel drivers as well.
> > > >
>
> How is this related to this series? i suggest to keep this series clean
> of vendor specific unrelated optimizations, this must be done in a
> separate patchset.
Well, Lorenzo explicitly is touching the thing that I referred to, so I
just ask if he can optimize it while he's at it.
Of course I'm fine with addressing this by myself once -next opens :)
>
>
> > > > ack, fine. I will fix in v4.
> > >
> > > Be careful with the Intel drivers. They have two modes (at compile
> > > time) depending on PAGE_SIZE in system. In one of the modes
> > > (default
> > > one) you can place init of xdp.frame_sz outside the NAPI loop and
> > > init a
> > > single time. In the other mode you cannot, and it becomes dynamic
> > > per
> > > packet. Intel review this carefully, please!
> >
> > ack. Actully I kept the xdp.frame_sz configuration in the NAPI loop
> > but
> > an Intel review will be nice.
> >
> > Regards,
> > Lorenzo
> >
> > > --
> > > Best regards,
> > > Jesper Dangaard Brouer
> > > MSc.CS, Principal Kernel Engineer at Red Hat
> > > LinkedIn: http://www.linkedin.com/in/brouer
> > >
>
^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff utility routine
2020-12-17 18:28 ` Maciej Fijalkowski
@ 2020-12-17 20:31 ` Saeed Mahameed
0 siblings, 0 replies; 19+ messages in thread
From: Saeed Mahameed @ 2020-12-17 20:31 UTC (permalink / raw)
To: Maciej Fijalkowski
Cc: Lorenzo Bianconi, Jesper Dangaard Brouer, Lorenzo Bianconi, bpf,
netdev, davem, kuba, ast, daniel, alexander.duyck
On Thu, 2020-12-17 at 19:28 +0100, Maciej Fijalkowski wrote:
> On Thu, Dec 17, 2020 at 10:16:06AM -0800, Saeed Mahameed wrote:
> > On Wed, 2020-12-16 at 16:01 +0100, Lorenzo Bianconi wrote:
> > > > On Tue, 15 Dec 2020 14:47:10 +0100
> > > > Lorenzo Bianconi <lorenzo.bianconi@redhat.com> wrote:
> > > >
> > > > > [...]
> > > > > > > xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp);
> > > > > > > diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > > > b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > > > index 4dbbbd49c389..fcd1ca3343fb 100644
> > > > > > > --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > > > +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c
> > > > > > > @@ -2393,12 +2393,12 @@ static int
> > > > > > > i40e_clean_rx_irq(struct
> > > > > > > i40e_ring *rx_ring, int budget)
> > > > > > >
> > > > > > > /* retrieve a buffer from the ring */
> > > > > > > if (!skb) {
> > > > > > > - xdp.data = page_address(rx_buffer-
> > > > > > > > page) +
> > > > > > > - rx_buffer->page_offset;
> > > > > > > - xdp.data_meta = xdp.data;
> > > > > > > - xdp.data_hard_start = xdp.data -
> > > > > > > - i40e_rx_offset(rx
> > > > > > > _ring);
> > > > > > > - xdp.data_end = xdp.data + size;
> > > > > > > + unsigned int offset =
> > > > > > > i40e_rx_offset(rx_ring);
> > > > > >
> > > > > > I now see that we could call the i40e_rx_offset() once per
> > > > > > napi, so can
> > > > > > you pull this variable out and have it initialized a single
> > > > > > time? Applies
> > > > > > to other intel drivers as well.
> >
> > How is this related to this series? i suggest to keep this series
> > clean
> > of vendor specific unrelated optimizations, this must be done in a
> > separate patchset.
>
> Well, Lorenzo explicitly is touching the thing that I referred to, so
> I
> just ask if he can optimize it while he's at it.
>
> Of course I'm fine with addressing this by myself once -next opens :)
>
Oh, don't get me wrong I am ok with doing this now, and i can do it my
self if you want :), but it shouldn't be part of the this series, so we
won't confuse others who want to implement XDP in the future, that's
all.
^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2020-12-17 20:31 UTC | newest]
Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-12 17:41 [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff Lorenzo Bianconi
2020-12-12 17:41 ` [PATCH v3 bpf-next 1/2] net: xdp: introduce xdp_init_buff utility routine Lorenzo Bianconi
2020-12-16 8:35 ` Jesper Dangaard Brouer
2020-12-16 14:56 ` Lorenzo Bianconi
2020-12-12 17:41 ` [PATCH v3 bpf-next 2/2] net: xdp: introduce xdp_prepare_buff " Lorenzo Bianconi
2020-12-15 12:36 ` Maciej Fijalkowski
2020-12-15 13:47 ` Lorenzo Bianconi
2020-12-15 14:51 ` Daniel Borkmann
2020-12-15 15:06 ` Lorenzo Bianconi
2020-12-15 15:13 ` Maciej Fijalkowski
2020-12-15 20:36 ` Lorenzo Bianconi
2020-12-16 8:30 ` Jesper Dangaard Brouer
2020-12-16 8:52 ` Jesper Dangaard Brouer
2020-12-16 15:01 ` Lorenzo Bianconi
2020-12-17 18:16 ` Saeed Mahameed
2020-12-17 18:28 ` Maciej Fijalkowski
2020-12-17 20:31 ` Saeed Mahameed
2020-12-14 15:32 ` [PATCH v3 bpf-next 0/2] introduce xdp_init_buff/xdp_prepare_buff Martin Habets
2020-12-14 17:53 ` Camelia Alexandra Groza
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.