* [PATCH nft 1/2] tests: py: remove verdict from closing end interval
@ 2021-11-04 21:40 Pablo Neira Ayuso
2021-11-04 21:40 ` [PATCH nft 2/2] mnl: do not build nftnl_set element list Pablo Neira Ayuso
0 siblings, 1 reply; 2+ messages in thread
From: Pablo Neira Ayuso @ 2021-11-04 21:40 UTC (permalink / raw)
To: netfilter-devel
Kernel does not allow for NFT_SET_ELEM_INTERVAL_END flag and
NFTA_SET_ELEM_DATA. The closing end interval represents a mismatch,
therefore, no verdict can be applied. The existing payload files show
the drop verdict when this is unset (because NF_DROP=0).
This update is required to fix payload warnings in tests/py after
libnftnl's ("set: use NFTNL_SET_ELEM_VERDICT to print verdict").
Fixes: 6671d9d137f6 ("mnl: Set NFTNL_SET_DATA_TYPE before dumping set elements")
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
tests/py/ip/ip.t.payload | 2 +-
tests/py/ip/ip.t.payload.bridge | 2 +-
tests/py/ip/ip.t.payload.inet | 2 +-
tests/py/ip/ip.t.payload.netdev | 2 +-
tests/py/ip6/vmap.t.payload.inet | 2 +-
tests/py/ip6/vmap.t.payload.ip6 | 2 +-
tests/py/ip6/vmap.t.payload.netdev | 2 +-
7 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/tests/py/ip/ip.t.payload b/tests/py/ip/ip.t.payload
index 49d1a0fb03e8..b9fcb5158e9d 100644
--- a/tests/py/ip/ip.t.payload
+++ b/tests/py/ip/ip.t.payload
@@ -430,7 +430,7 @@ ip test-ip4 input
# ip hdrlength vmap { 0-4 : drop, 5 : accept, 6 : continue } counter
__map%d test-ip4 f size 4
__map%d test-ip4 0
- element 00000000 : drop 0 [end] element 00000005 : accept 0 [end] element 00000006 : continue 0 [end] element 00000007 : drop 1 [end]
+ element 00000000 : drop 0 [end] element 00000005 : accept 0 [end] element 00000006 : continue 0 [end] element 00000007 : 1 [end]
ip test-ip4 input
[ payload load 1b @ network header + 0 => reg 1 ]
[ bitwise reg 1 = ( reg 1 & 0x0000000f ) ^ 0x00000000 ]
diff --git a/tests/py/ip/ip.t.payload.bridge b/tests/py/ip/ip.t.payload.bridge
index dac8654395f4..c6f8d4e5575b 100644
--- a/tests/py/ip/ip.t.payload.bridge
+++ b/tests/py/ip/ip.t.payload.bridge
@@ -566,7 +566,7 @@ bridge test-bridge input
# ip hdrlength vmap { 0-4 : drop, 5 : accept, 6 : continue } counter
__map%d test-bridge f size 4
__map%d test-bridge 0
- element 00000000 : drop 0 [end] element 00000005 : accept 0 [end] element 00000006 : continue 0 [end] element 00000007 : drop 1 [end]
+ element 00000000 : drop 0 [end] element 00000005 : accept 0 [end] element 00000006 : continue 0 [end] element 00000007 : 1 [end]
bridge test-bridge input
[ meta load protocol => reg 1 ]
[ cmp eq reg 1 0x00000008 ]
diff --git a/tests/py/ip/ip.t.payload.inet b/tests/py/ip/ip.t.payload.inet
index 64371650480f..e26d0dac47be 100644
--- a/tests/py/ip/ip.t.payload.inet
+++ b/tests/py/ip/ip.t.payload.inet
@@ -566,7 +566,7 @@ inet test-inet input
# ip hdrlength vmap { 0-4 : drop, 5 : accept, 6 : continue } counter
__map%d test-inet f size 4
__map%d test-inet 0
- element 00000000 : drop 0 [end] element 00000005 : accept 0 [end] element 00000006 : continue 0 [end] element 00000007 : drop 1 [end]
+ element 00000000 : drop 0 [end] element 00000005 : accept 0 [end] element 00000006 : continue 0 [end] element 00000007 : 1 [end]
inet test-inet input
[ meta load nfproto => reg 1 ]
[ cmp eq reg 1 0x00000002 ]
diff --git a/tests/py/ip/ip.t.payload.netdev b/tests/py/ip/ip.t.payload.netdev
index 65f8c96a9470..de990f5bba12 100644
--- a/tests/py/ip/ip.t.payload.netdev
+++ b/tests/py/ip/ip.t.payload.netdev
@@ -465,7 +465,7 @@ netdev test-netdev ingress
# ip hdrlength vmap { 0-4 : drop, 5 : accept, 6 : continue } counter
__map%d test-netdev f size 4
__map%d test-netdev 0
- element 00000000 : drop 0 [end] element 00000005 : accept 0 [end] element 00000006 : continue 0 [end] element 00000007 : drop 1 [end]
+ element 00000000 : drop 0 [end] element 00000005 : accept 0 [end] element 00000006 : continue 0 [end] element 00000007 : 1 [end]
netdev test-netdev ingress
[ meta load protocol => reg 1 ]
[ cmp eq reg 1 0x00000008 ]
diff --git a/tests/py/ip6/vmap.t.payload.inet b/tests/py/ip6/vmap.t.payload.inet
index 522564a31044..931cc6bd75a7 100644
--- a/tests/py/ip6/vmap.t.payload.inet
+++ b/tests/py/ip6/vmap.t.payload.inet
@@ -371,7 +371,7 @@ inet test-inet input
# ip6 saddr vmap { ::/64 : accept}
__map%d test-inet f
__map%d test-inet 0
- element 00000000 00000000 00000000 00000000 : accept 0 [end] element 00000000 01000000 00000000 00000000 : drop 1 [end]
+ element 00000000 00000000 00000000 00000000 : accept 0 [end] element 00000000 01000000 00000000 00000000 : 1 [end]
inet test-inet input
[ meta load nfproto => reg 1 ]
[ cmp eq reg 1 0x0000000a ]
diff --git a/tests/py/ip6/vmap.t.payload.ip6 b/tests/py/ip6/vmap.t.payload.ip6
index 3850a7c58fd1..6e077b27a5f0 100644
--- a/tests/py/ip6/vmap.t.payload.ip6
+++ b/tests/py/ip6/vmap.t.payload.ip6
@@ -297,7 +297,7 @@ ip6 test-ip6 input
# ip6 saddr vmap { ::/64 : accept}
__map%d test-ip6 f
__map%d test-ip6 0
- element 00000000 00000000 00000000 00000000 : accept 0 [end] element 00000000 01000000 00000000 00000000 : drop 1 [end]
+ element 00000000 00000000 00000000 00000000 : accept 0 [end] element 00000000 01000000 00000000 00000000 : 1 [end]
ip6 test-ip6 input
[ payload load 16b @ network header + 8 => reg 1 ]
[ lookup reg 1 set __map%d dreg 0 ]
diff --git a/tests/py/ip6/vmap.t.payload.netdev b/tests/py/ip6/vmap.t.payload.netdev
index d9cbad5877ae..45f2c0b01e9c 100644
--- a/tests/py/ip6/vmap.t.payload.netdev
+++ b/tests/py/ip6/vmap.t.payload.netdev
@@ -371,7 +371,7 @@ netdev test-netdev ingress
# ip6 saddr vmap { ::/64 : accept}
__map%d test-netdev f
__map%d test-netdev 0
- element 00000000 00000000 00000000 00000000 : accept 0 [end] element 00000000 01000000 00000000 00000000 : drop 1 [end]
+ element 00000000 00000000 00000000 00000000 : accept 0 [end] element 00000000 01000000 00000000 00000000 : 1 [end]
netdev test-netdev ingress
[ meta load protocol => reg 1 ]
[ cmp eq reg 1 0x0000dd86 ]
--
2.30.2
^ permalink raw reply related [flat|nested] 2+ messages in thread
* [PATCH nft 2/2] mnl: do not build nftnl_set element list
2021-11-04 21:40 [PATCH nft 1/2] tests: py: remove verdict from closing end interval Pablo Neira Ayuso
@ 2021-11-04 21:40 ` Pablo Neira Ayuso
0 siblings, 0 replies; 2+ messages in thread
From: Pablo Neira Ayuso @ 2021-11-04 21:40 UTC (permalink / raw)
To: netfilter-devel
Do not call alloc_setelem_cache() to build the set element list in
nftnl_set. Instead, translate one single set element expression to
nftnl_set_elem object at a time and use this object to build the netlink
header.
This is reduce userspace memory consumption, using a huge test set
containing 1.1 million element blocklist, this patch is reducing
userspace memory consumption by 40%.
Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
include/netlink.h | 2 +
src/mnl.c | 112 ++++++++++++++++++++++++++++++++++++----------
src/netlink.c | 4 +-
3 files changed, 93 insertions(+), 25 deletions(-)
diff --git a/include/netlink.h b/include/netlink.h
index 2467ff82a520..c1d7d3189455 100644
--- a/include/netlink.h
+++ b/include/netlink.h
@@ -72,6 +72,8 @@ struct netlink_ctx {
extern struct nftnl_expr *alloc_nft_expr(const char *name);
extern void alloc_setelem_cache(const struct expr *set, struct nftnl_set *nls);
+struct nftnl_set_elem *alloc_nftnl_setelem(const struct expr *set,
+ const struct expr *expr);
extern struct nftnl_table *netlink_table_alloc(const struct nlmsghdr *nlh);
extern struct nftnl_chain *netlink_chain_alloc(const struct nlmsghdr *nlh);
diff --git a/src/mnl.c b/src/mnl.c
index 2d5afdfe8c7f..4a10647f9f17 100644
--- a/src/mnl.c
+++ b/src/mnl.c
@@ -1518,33 +1518,102 @@ static int set_elem_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
-static int mnl_nft_setelem_batch(struct nftnl_set *nls,
+static bool mnl_nft_attr_nest_overflow(struct nlmsghdr *nlh,
+ const struct nlattr *from,
+ const struct nlattr *to)
+{
+ int len = (void *)to + to->nla_len - (void *)from;
+
+ /* The attribute length field is 16 bits long, thus the maximum payload
+ * that an attribute can convey is UINT16_MAX. In case of overflow,
+ * discard the last attribute that did not fit into the nest.
+ */
+ if (len > UINT16_MAX) {
+ nlh->nlmsg_len -= to->nla_len;
+ return true;
+ }
+ return false;
+}
+
+static void netlink_dump_setelem(const struct nftnl_set_elem *nlse,
+ struct netlink_ctx *ctx)
+{
+ FILE *fp = ctx->nft->output.output_fp;
+ char buf[4096];
+
+ if (!(ctx->nft->debug_mask & NFT_DEBUG_NETLINK) || !fp)
+ return;
+
+ nftnl_set_elem_snprintf(buf, sizeof(buf), nlse, NFTNL_OUTPUT_DEFAULT, 0);
+ fprintf(fp, "\t%s", buf);
+}
+
+static void netlink_dump_setelem_done(struct netlink_ctx *ctx)
+{
+ FILE *fp = ctx->nft->output.output_fp;
+
+ if (!(ctx->nft->debug_mask & NFT_DEBUG_NETLINK) || !fp)
+ return;
+
+ fprintf(fp, "\n");
+}
+
+static int mnl_nft_setelem_batch(const struct nftnl_set *nls,
struct nftnl_batch *batch,
enum nf_tables_msg_types cmd,
- unsigned int flags, uint32_t seqnum)
+ unsigned int flags, uint32_t seqnum,
+ const struct expr *set,
+ struct netlink_ctx *ctx)
{
+ struct nlattr *nest1, *nest2;
+ struct nftnl_set_elem *nlse;
struct nlmsghdr *nlh;
- struct nftnl_set_elems_iter *iter;
- int ret;
-
- iter = nftnl_set_elems_iter_create(nls);
- if (iter == NULL)
- memory_allocation_error();
+ struct expr *expr = NULL;
+ int i = 0;
if (cmd == NFT_MSG_NEWSETELEM)
flags |= NLM_F_CREATE;
- while (nftnl_set_elems_iter_cur(iter)) {
- nlh = nftnl_nlmsg_build_hdr(nftnl_batch_buffer(batch), cmd,
- nftnl_set_get_u32(nls, NFTNL_SET_FAMILY),
- flags, seqnum);
- ret = nftnl_set_elems_nlmsg_build_payload_iter(nlh, iter);
- mnl_nft_batch_continue(batch);
- if (ret <= 0)
- break;
+ if (set)
+ expr = list_first_entry(&set->expressions, struct expr, list);
+
+next:
+ nlh = nftnl_nlmsg_build_hdr(nftnl_batch_buffer(batch), cmd,
+ nftnl_set_get_u32(nls, NFTNL_SET_FAMILY),
+ flags, seqnum);
+
+ if (nftnl_set_is_set(nls, NFTNL_SET_TABLE)) {
+ mnl_attr_put_strz(nlh, NFTA_SET_ELEM_LIST_TABLE,
+ nftnl_set_get_str(nls, NFTNL_SET_TABLE));
+ }
+ if (nftnl_set_is_set(nls, NFTNL_SET_NAME)) {
+ mnl_attr_put_strz(nlh, NFTA_SET_ELEM_LIST_SET,
+ nftnl_set_get_str(nls, NFTNL_SET_NAME));
}
+ if (nftnl_set_is_set(nls, NFTNL_SET_ID)) {
+ mnl_attr_put_u32(nlh, NFTA_SET_ELEM_LIST_SET_ID,
+ htonl(nftnl_set_get_u32(nls, NFTNL_SET_ID)));
+ }
+
+ if (!set || list_empty(&set->expressions))
+ return 0;
- nftnl_set_elems_iter_destroy(iter);
+ assert(expr);
+ nest1 = mnl_attr_nest_start(nlh, NFTA_SET_ELEM_LIST_ELEMENTS);
+ list_for_each_entry_from(expr, &set->expressions, list) {
+ nlse = alloc_nftnl_setelem(set, expr);
+ nest2 = nftnl_set_elem_nlmsg_build(nlh, nlse, ++i);
+ netlink_dump_setelem(nlse, ctx);
+ nftnl_set_elem_free(nlse);
+ if (mnl_nft_attr_nest_overflow(nlh, nest1, nest2)) {
+ mnl_attr_nest_end(nlh, nest1);
+ mnl_nft_batch_continue(batch);
+ goto next;
+ }
+ }
+ mnl_attr_nest_end(nlh, nest1);
+ mnl_nft_batch_continue(batch);
+ netlink_dump_setelem_done(ctx);
return 0;
}
@@ -1569,11 +1638,10 @@ int mnl_nft_setelem_add(struct netlink_ctx *ctx, const struct set *set,
nftnl_set_set_u32(nls, NFTNL_SET_DATA_TYPE,
dtype_map_to_kernel(set->data->dtype));
- alloc_setelem_cache(expr, nls);
netlink_dump_set(nls, ctx);
- err = mnl_nft_setelem_batch(nls, ctx->batch, NFT_MSG_NEWSETELEM, flags,
- ctx->seqnum);
+ err = mnl_nft_setelem_batch(nls, ctx->batch, NFT_MSG_NEWSETELEM,
+ flags, ctx->seqnum, expr, ctx);
nftnl_set_free(nls);
return err;
@@ -1626,12 +1694,10 @@ int mnl_nft_setelem_del(struct netlink_ctx *ctx, const struct cmd *cmd)
else if (h->handle.id)
nftnl_set_set_u64(nls, NFTNL_SET_HANDLE, h->handle.id);
- if (cmd->expr)
- alloc_setelem_cache(cmd->expr, nls);
netlink_dump_set(nls, ctx);
err = mnl_nft_setelem_batch(nls, ctx->batch, NFT_MSG_DELSETELEM, 0,
- ctx->seqnum);
+ ctx->seqnum, cmd->expr, ctx);
nftnl_set_free(nls);
return err;
diff --git a/src/netlink.c b/src/netlink.c
index 28a5514ad873..f63f2bd1bd60 100644
--- a/src/netlink.c
+++ b/src/netlink.c
@@ -100,8 +100,8 @@ struct nftnl_expr *alloc_nft_expr(const char *name)
void __netlink_gen_data(const struct expr *expr,
struct nft_data_linearize *data, bool expand);
-static struct nftnl_set_elem *alloc_nftnl_setelem(const struct expr *set,
- const struct expr *expr)
+struct nftnl_set_elem *alloc_nftnl_setelem(const struct expr *set,
+ const struct expr *expr)
{
const struct expr *elem, *data;
struct nftnl_set_elem *nlse;
--
2.30.2
^ permalink raw reply related [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-11-04 21:40 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-04 21:40 [PATCH nft 1/2] tests: py: remove verdict from closing end interval Pablo Neira Ayuso
2021-11-04 21:40 ` [PATCH nft 2/2] mnl: do not build nftnl_set element list Pablo Neira Ayuso
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.