"Sets" element cannot update "struct proto_ctx"

"Sets" element cannot update "struct proto_ctx"

[Ttttabcd]

Recently I was trying to set up tproxy, I need to transparently proxy TCP and UDP at the same time, naturally I would think of using the following command.

nft add rule inet myproxy prerouting iif wlan0 meta pkttype unicast meta l4proto {tcp,udp} tproxy to :5555 meta mark set 1 accept

Error: Transparent proxy support requires transport protocol match
add rule inet myproxy prerouting iif wlan0 meta pkttype unicast meta l4proto {tcp,udp} tproxy to :5555 meta mark set 1 accept
                                                                                    ^^^^^^^^^^^^^^^
This command is wrong.

But the strange thing is if I use the following command

nft add rule inet myproxy prerouting iif wlan0 meta pkttype unicast meta l4proto tcp tproxy to :5555 meta mark set 1 accept

There is no problem, "tproxy" seems to conflict with "Sets".

In order to find the root cause, I can only start analyzing the source code of nft.

static int stmt_evaluate_tproxy(struct eval_ctx *ctx, struct stmt *stmt)
{
...
	if (ctx->pctx.protocol[PROTO_BASE_TRANSPORT_HDR].desc == NULL)
		return stmt_error(ctx, stmt, "Transparent proxy support requires"
					     " transport protocol match");
...
}

Eventually I found the above code, the error condition is that protocol[PROTO_BASE_TRANSPORT_HDR].desc is NULL.

Then I used the debugging function in nft and executed the following two commands respectively.

nft --debug all add rule inet myproxy prerouting iif wlan0 meta pkttype unicast meta l4proto {tcp,udp} tproxy to :5555 meta mark set 1 accept

nft --debug all add rule inet myproxy prerouting iif wlan0 meta pkttype unicast meta l4proto tcp tproxy to :5555 meta mark set 1 accept


Evaluate value
add rule inet mitm prerouting iif wlan0 meta pkttype unicast meta l4proto tcp tproxy to :5555 meta mark set 1 accept
                                                                          ^^^
tcp

update transport layer protocol context:
 link layer          : inet
 network layer       : none
 transport layer     : tcp <-

I found the difference. If it is a protocol value, it will update the transport layer protocol context. If it is "Sets", it will not be updated.

In order to figure out which line of code is causing, I continue to analyze the source code.

static void meta_expr_pctx_update(struct proto_ctx *ctx,
				  const struct expr *expr)
{
...
	case NFT_META_L4PROTO:
		desc = proto_find_upper(&proto_inet_service,
					mpz_get_uint8(right->value));
		if (desc == NULL)
			desc = &proto_unknown;

		proto_ctx_update(ctx, PROTO_BASE_TRANSPORT_HDR, &expr->location, desc);
		break;
...
}

Update the transport layer protocol context is executed in the meta_expr_pctx_update function

const struct expr_ops meta_expr_ops = {
	.type		= EXPR_META,
	.name		= "meta",
	.print		= meta_expr_print,
	.json		= meta_expr_json,
	.cmp		= meta_expr_cmp,
	.clone		= meta_expr_clone,
	.pctx_update	= meta_expr_pctx_update,
};

Meta_expr_pctx_update is a function in meta_expr_ops

void relational_expr_pctx_update(struct proto_ctx *ctx,
				 const struct expr *expr)
{
...
	ops = expr_ops(left);
	if (ops->pctx_update &&
	    (left->flags & EXPR_F_PROTOCOL))
		ops->pctx_update(ctx, expr);
}

Relational_expr_pctx_update will call ops->pctx_update

static void ct_meta_common_postprocess(struct rule_pp_ctx *ctx,
				       const struct expr *expr,
				       enum proto_bases base)
{
...
	switch (expr->op) {
	case OP_EQ:
		if (expr->right->etype == EXPR_RANGE ||
		    expr->right->etype == EXPR_SET ||
		    expr->right->etype == EXPR_SET_REF)
			break;

		relational_expr_pctx_update(&ctx->pctx, expr);
...
}

Eventually I found the problem in the ct_meta_common_postprocess function, which specifies that if it is a "Sets" or "Range" or a "Sets" reference, it will not call relational_expr_pctx_update

static void meta_match_postprocess(struct rule_pp_ctx *ctx,
				   const struct expr *expr)
{
	const struct expr *left = expr->left;

	ct_meta_common_postprocess(ctx, expr, left->meta.base);
}

The ct_meta_common_postprocess function is called by meta_match_postprocess, which looks like a handler for a "meta" expression.

The above analysis is only a personal guess. I don't know much about nft's source structure. I don't know if my analysis is correct.

But the above problem does exist. This may not be a "BUG", but it must be a "TODO".

struct proto_ctx {
	unsigned int			debug_mask;
	unsigned int			family;
	struct {
		struct location			location;
		const struct proto_desc		*desc;
		unsigned int			offset;
	} protocol[PROTO_BASE_MAX + 1];
};

The design of struct proto_ctx is flawed. Because users are likely to use more than one network protocol, whether it is the network layer or the transport layer.

Only support setting a single protocol will lose a lot of flexibility

The "inet" protocol family, although it can support ipv4 and ipv6, may not be a good design, maybe ipv8 will appear in the future? A better design is that each layer of protocol can support "Sets" and support multiple protocols, so the scalability is much higher, at least the user can manipulate everything.

We can add a list_head to the struct protocol, use a linked list to concatenate multiple protocols, and only need to traverse the linked list when using it.

Of course, this may have to modify a lot of code, but also a big project, but the above is my personal suggestion, I also hope that nftables will develop better.