[PATCH nft 0/8] Enableruntime queue selection via jhash, numgen and map statement

[PATCH nft 0/8] Enableruntime queue selection via jhash, numgen and map statement

[Florian Westphal]

Back in 2016 Liping Zhang added support to kernel and libnftnl to
specify a source register containing the queue number to use.

This was never added to nft itself, so allow this.

On linearization side, check if attached expression is a range.
If its not, allocate a new register and set NFTNL_EXPR_QUEUE_SREG_QNUM
attribute after generating the lowlevel expressions for the kernel.

On delinarization we need to check for presence of
NFTNL_EXPR_QUEUE_SREG_QNUM and decode the expression(s) when present.

Also need to do postprocessing for STMT_QUEUE so that the protocol
context is set correctly, without this only raw payload expressions
will be shown (@nh,32,...) instead of 'ip ...'.

Unfortunately, it turned out that just removing the eval checks
to allow arbitrary statements as 'num' argument results in parser
problems.

One example is this:
   queue num jhash ip saddr mod 4 bypass

This fails because scanner is still in 'ip' state, not 'queue', when
"bypass" is read, so this will not be recognized as belonging to the
queue statement.

This series solves this in the following way:
1. On output, nft will now always prepend the flags, i.e.
  queue flags bypass num 42

2. On input, 'queue num' is restricted to numbers and ranges.
This is backwards compatible because range and value were the
only permitted inputs (eval step rejects non-constant expressions).

3. To use numgen or jhash, new grammar is added:
 queue flags bypass to jhash ip saddr mod 4

I've restricted the 'to' expressions to numgen, (sym)hash and map
for now.

This can be relaxed later on if other usecases become available.

Florian Westphal (8):
  evaluate: fix hash expression maxval
  parser: restrict queue num expressiveness
  src: add queue expr and flags to queue_stmt_alloc
  parser: add queue_stmt_compat
  parser: new queue flag input format
  src: queue: allow use of arbitrary queue expressions
  tests: extend queue testcases for new sreg support
  src: queue: allow use of MAP statement for queue number retrieval

 doc/statements.txt           | 10 +++-
 include/statement.h          |  3 +-
 src/evaluate.c               | 21 +++++----
 src/netlink_delinearize.c    | 56 +++++++++++++++-------
 src/netlink_linearize.c      | 28 +++++++++--
 src/parser_bison.y           | 38 ++++++++++++---
 src/parser_json.c            | 22 ++++-----
 src/statement.c              | 30 +++++++++---
 tests/py/any/queue.t         | 18 ++++++--
 tests/py/any/queue.t.json    | 90 ++++++++++++++++++++++++++++++++++++
 tests/py/any/queue.t.payload | 25 ++++++++++
 11 files changed, 281 insertions(+), 60 deletions(-)

-- 
2.31.1

[PATCH nft 1/8] evaluate: fix hash expression maxval

[Florian Westphal]

It needs to account for the offset too.

Fixes: 9bee0c86f179 ("src: add offset attribute for hash expression")
Fixes: d4f9a8fb9e9a ("src: add offset attribute for numgen expression")
Signed-off-by: Florian Westphal <fw@strlen.de>
---
 src/evaluate.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/src/evaluate.c b/src/evaluate.c
index aa7ec9bee4ae..bebdb3f827e9 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -1657,17 +1657,20 @@ static void expr_dtype_integer_compatible(struct eval_ctx *ctx,
 static int expr_evaluate_numgen(struct eval_ctx *ctx, struct expr **exprp)
 {
 	struct expr *expr = *exprp;
+	unsigned int maxval;
 
 	expr_dtype_integer_compatible(ctx, expr);
 
+	maxval = expr->numgen.mod + expr->numgen.offset - 1;
 	__expr_set_context(&ctx->ectx, expr->dtype, expr->byteorder, expr->len,
-			   expr->numgen.mod - 1);
+			   maxval);
 	return 0;
 }
 
 static int expr_evaluate_hash(struct eval_ctx *ctx, struct expr **exprp)
 {
 	struct expr *expr = *exprp;
+	unsigned int maxval;
 
 	expr_dtype_integer_compatible(ctx, expr);
 
@@ -1680,8 +1683,9 @@ static int expr_evaluate_hash(struct eval_ctx *ctx, struct expr **exprp)
          * expression to be hashed. Since this input is transformed to a 4 bytes
 	 * integer, restore context to the datatype that results from hashing.
 	 */
+	maxval = expr->hash.mod + expr->hash.offset - 1;
 	__expr_set_context(&ctx->ectx, expr->dtype, expr->byteorder, expr->len,
-			   expr->hash.mod - 1);
+			   maxval);
 
 	return 0;
 }
-- 
2.31.1

[PATCH nft 2/8] parser: restrict queue num expressiveness

[Florian Westphal]

Else we run into trouble once we allow
queue num symhash mod 4 and 1

and so on.  Example problem:

queue num jhash ip saddr mod 4 and 1 bypass

This will fail to parse because the scanner is in the wrong state
(ip, not queue), so 'bypass' is parsed as a string.

Currently, while nft will eat the above just fine (minus 'bypass'),
nft rejects this from the evaluation phase with
   Error: queue number is not constant

So seems we are lucky and can restrict the supported expressions
to integer and range.

Furthermore, the line looks wrong because this statement:

   queue num jhash ip saddr mod 4 and 1 bypass

doesn't specifiy a number, "queue num 4" does, or "queue num 1-2" do.

For arbitrary expr support it seems sensible to enforce stricter
ordering to avoid any problems with the flags, for example:

queue bypass,futurekeyword to jhash ip saddr mod 42

Signed-off-by: Florian Westphal <fw@strlen.de>
---
 src/parser_bison.y | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/src/parser_bison.y b/src/parser_bison.y
index bd2232a3de27..2ab47ed55166 100644
--- a/src/parser_bison.y
+++ b/src/parser_bison.y
@@ -705,6 +705,8 @@ int nft_lex(void *, void *, void *);
 
 %type <stmt>			queue_stmt queue_stmt_alloc
 %destructor { stmt_free($$); }	queue_stmt queue_stmt_alloc
+%type <expr>			queue_stmt_expr
+%destructor { expr_free($$); }	queue_stmt_expr
 %type <val>			queue_stmt_flags queue_stmt_flag
 %type <stmt>			dup_stmt
 %destructor { stmt_free($$); }	dup_stmt
@@ -3753,7 +3755,7 @@ queue_stmt_args		:	queue_stmt_arg
 			|	queue_stmt_args	queue_stmt_arg
 			;
 
-queue_stmt_arg		:	QUEUENUM	stmt_expr
+queue_stmt_arg		:	QUEUENUM	queue_stmt_expr
 			{
 				$<stmt>0->queue.queue = $2;
 				$<stmt>0->queue.queue->location = @$;
@@ -3764,6 +3766,10 @@ queue_stmt_arg		:	QUEUENUM	stmt_expr
 			}
 			;
 
+queue_stmt_expr		:	integer_expr
+			|	range_rhs_expr
+			;
+
 queue_stmt_flags	:	queue_stmt_flag
 			|	queue_stmt_flags	COMMA	queue_stmt_flag
 			{
-- 
2.31.1

[PATCH nft 3/8] src: add queue expr and flags to queue_stmt_alloc

[Florian Westphal]

Preparation patch to avoid too much $<stmt>$ references in the parser.

Signed-off-by: Florian Westphal <fw@strlen.de>
---
 include/statement.h       |  3 ++-
 src/netlink_delinearize.c | 10 +++-------
 src/parser_bison.y        |  2 +-
 src/parser_json.c         | 22 +++++++++++-----------
 src/statement.c           | 10 ++++++++--
 5 files changed, 25 insertions(+), 22 deletions(-)

diff --git a/include/statement.h b/include/statement.h
index 7637a82e4e00..06221040fa0c 100644
--- a/include/statement.h
+++ b/include/statement.h
@@ -159,7 +159,8 @@ struct queue_stmt {
 	uint16_t		flags;
 };
 
-extern struct stmt *queue_stmt_alloc(const struct location *loc);
+extern struct stmt *queue_stmt_alloc(const struct location *loc,
+				     struct expr *e, uint16_t flags);
 
 struct quota_stmt {
 	uint64_t		bytes;
diff --git a/src/netlink_delinearize.c b/src/netlink_delinearize.c
index 5c80397db26c..7ea31e6a2639 100644
--- a/src/netlink_delinearize.c
+++ b/src/netlink_delinearize.c
@@ -1467,9 +1467,8 @@ static void netlink_parse_queue(struct netlink_parse_ctx *ctx,
 			      const struct location *loc,
 			      const struct nftnl_expr *nle)
 {
+	uint16_t num, total, flags;
 	struct expr *expr, *high;
-	struct stmt *stmt;
-	uint16_t num, total;
 
 	num   = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_NUM);
 	total = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_TOTAL);
@@ -1483,11 +1482,8 @@ static void netlink_parse_queue(struct netlink_parse_ctx *ctx,
 		expr = range_expr_alloc(loc, expr, high);
 	}
 
-	stmt = queue_stmt_alloc(loc);
-	stmt->queue.queue = expr;
-	stmt->queue.flags = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_FLAGS);
-
-	ctx->stmt = stmt;
+	flags = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_FLAGS);
+	ctx->stmt = queue_stmt_alloc(loc, expr, flags);
 }
 
 struct dynset_parse_ctx {
diff --git a/src/parser_bison.y b/src/parser_bison.y
index 2ab47ed55166..96676aed2e38 100644
--- a/src/parser_bison.y
+++ b/src/parser_bison.y
@@ -3744,7 +3744,7 @@ queue_stmt		:	queue_stmt_alloc	close_scope_queue
 
 queue_stmt_alloc	:	QUEUE
 			{
-				$$ = queue_stmt_alloc(&@$);
+				$$ = queue_stmt_alloc(&@$, NULL, 0);
 			}
 			;
 
diff --git a/src/parser_json.c b/src/parser_json.c
index bb0e4169b477..e03b51697cb7 100644
--- a/src/parser_json.c
+++ b/src/parser_json.c
@@ -2559,14 +2559,14 @@ static int queue_flag_parse(const char *name, uint16_t *flags)
 static struct stmt *json_parse_queue_stmt(struct json_ctx *ctx,
 					  const char *key, json_t *value)
 {
-	struct stmt *stmt = queue_stmt_alloc(int_loc);
+	struct expr *qexpr = NULL;
+	uint16_t flags = 0;
 	json_t *tmp;
 
 	if (!json_unpack(value, "{s:o}", "num", &tmp)) {
-		stmt->queue.queue = json_parse_stmt_expr(ctx, tmp);
-		if (!stmt->queue.queue) {
+		qexpr = json_parse_stmt_expr(ctx, tmp);
+		if (!qexpr) {
 			json_error(ctx, "Invalid queue num.");
-			stmt_free(stmt);
 			return NULL;
 		}
 	}
@@ -2578,15 +2578,15 @@ static struct stmt *json_parse_queue_stmt(struct json_ctx *ctx,
 		if (json_is_string(tmp)) {
 			flag = json_string_value(tmp);
 
-			if (queue_flag_parse(flag, &stmt->queue.flags)) {
+			if (queue_flag_parse(flag, &flags)) {
 				json_error(ctx, "Invalid queue flag '%s'.",
 					   flag);
-				stmt_free(stmt);
+				expr_free(qexpr);
 				return NULL;
 			}
 		} else if (!json_is_array(tmp)) {
 			json_error(ctx, "Unexpected object type in queue flags.");
-			stmt_free(stmt);
+			expr_free(qexpr);
 			return NULL;
 		}
 
@@ -2594,20 +2594,20 @@ static struct stmt *json_parse_queue_stmt(struct json_ctx *ctx,
 			if (!json_is_string(val)) {
 				json_error(ctx, "Invalid object in queue flag array at index %zu.",
 					   index);
-				stmt_free(stmt);
+				expr_free(qexpr);
 				return NULL;
 			}
 			flag = json_string_value(val);
 
-			if (queue_flag_parse(flag, &stmt->queue.flags)) {
+			if (queue_flag_parse(flag, &flags)) {
 				json_error(ctx, "Invalid queue flag '%s'.",
 					   flag);
-				stmt_free(stmt);
+				expr_free(qexpr);
 				return NULL;
 			}
 		}
 	}
-	return stmt;
+	return queue_stmt_alloc(int_loc, qexpr, flags);
 }
 
 static struct stmt *json_parse_connlimit_stmt(struct json_ctx *ctx,
diff --git a/src/statement.c b/src/statement.c
index 7537c07f495c..a713952c0af7 100644
--- a/src/statement.c
+++ b/src/statement.c
@@ -522,9 +522,15 @@ static const struct stmt_ops queue_stmt_ops = {
 	.destroy	= queue_stmt_destroy,
 };
 
-struct stmt *queue_stmt_alloc(const struct location *loc)
+struct stmt *queue_stmt_alloc(const struct location *loc, struct expr *e, uint16_t flags)
 {
-	return stmt_alloc(loc, &queue_stmt_ops);
+	struct stmt *stmt;
+
+	stmt = stmt_alloc(loc, &queue_stmt_ops);
+	stmt->queue.queue = e;
+	stmt->queue.flags = flags;
+
+	return stmt;
 }
 
 static void quota_stmt_print(const struct stmt *stmt, struct output_ctx *octx)
-- 
2.31.1

[PATCH nft 4/8] parser: add queue_stmt_compat

[Florian Westphal]

Rename existing rules to _compat to make sure old rules using 'queue'
statement will work.

Next patch adds distinct input format where flags are explicitly
provided:

 queue flags name,<nextflag> num 1

Without this, extension of queue expression to handle arbitrary
expression instead of queue number or range results in parser errors.

Example:
   queue num jhash ip saddr mod 4 and 1 bypass

will fail because scanner is still in 'ip' state, not 'queue', when
"bypass" is read.

Signed-off-by: Florian Westphal <fw@strlen.de>
---
 src/parser_bison.y | 19 +++++++++++--------
 1 file changed, 11 insertions(+), 8 deletions(-)

diff --git a/src/parser_bison.y b/src/parser_bison.y
index 96676aed2e38..9e45a5da1716 100644
--- a/src/parser_bison.y
+++ b/src/parser_bison.y
@@ -703,10 +703,10 @@ int nft_lex(void *, void *, void *);
 %destructor { stmt_free($$); }	chain_stmt
 %type <val>			chain_stmt_type
 
-%type <stmt>			queue_stmt queue_stmt_alloc
-%destructor { stmt_free($$); }	queue_stmt queue_stmt_alloc
-%type <expr>			queue_stmt_expr
-%destructor { expr_free($$); }	queue_stmt_expr
+%type <stmt>			queue_stmt queue_stmt_alloc	queue_stmt_compat
+%destructor { stmt_free($$); }	queue_stmt queue_stmt_alloc	queue_stmt_compat
+%type <expr>			queue_stmt_expr_simple
+%destructor { expr_free($$); }	queue_stmt_expr_simple
 %type <val>			queue_stmt_flags queue_stmt_flag
 %type <stmt>			dup_stmt
 %destructor { stmt_free($$); }	dup_stmt
@@ -3738,8 +3738,11 @@ nf_nat_flag		:	RANDOM		{ $$ = NF_NAT_RANGE_PROTO_RANDOM; }
 			|	PERSISTENT 	{ $$ = NF_NAT_RANGE_PERSISTENT; }
 			;
 
-queue_stmt		:	queue_stmt_alloc	close_scope_queue
-			|	queue_stmt_alloc	queue_stmt_args	close_scope_queue
+queue_stmt		:	queue_stmt_compat	close_scope_queue
+			;
+
+queue_stmt_compat	:	queue_stmt_alloc
+			|	queue_stmt_alloc	queue_stmt_args
 			;
 
 queue_stmt_alloc	:	QUEUE
@@ -3755,7 +3758,7 @@ queue_stmt_args		:	queue_stmt_arg
 			|	queue_stmt_args	queue_stmt_arg
 			;
 
-queue_stmt_arg		:	QUEUENUM	queue_stmt_expr
+queue_stmt_arg		:	QUEUENUM	queue_stmt_expr_simple
 			{
 				$<stmt>0->queue.queue = $2;
 				$<stmt>0->queue.queue->location = @$;
@@ -3766,7 +3769,7 @@ queue_stmt_arg		:	QUEUENUM	queue_stmt_expr
 			}
 			;
 
-queue_stmt_expr		:	integer_expr
+queue_stmt_expr_simple	:	integer_expr
 			|	range_rhs_expr
 			;
 
-- 
2.31.1

[PATCH nft 5/8] parser: new queue flag input format

[Florian Westphal]

This changes output to first list the queue flags (with prepended
"flags" keyword), then the queue number.

This is to avoid parser problems when a flag is used after the
queue number, e.g.

"queue num 42 bypass".

While this works fine, this does not:

"queue num tcp dport bypass"

... because scanner state has been switched, "bypass" is parsed
as symbol expression.

Input parser is changed to recognize the stricter flag usage.

Signed-off-by: Florian Westphal <fw@strlen.de>
---
 doc/statements.txt   |  4 ++--
 src/parser_bison.y   |  4 ++++
 src/statement.c      | 15 ++++++++++-----
 tests/py/any/queue.t |  7 +++----
 4 files changed, 19 insertions(+), 11 deletions(-)

diff --git a/doc/statements.txt b/doc/statements.txt
index 7c7240c82fab..602a5b2011a7 100644
--- a/doc/statements.txt
+++ b/doc/statements.txt
@@ -589,8 +589,8 @@ for details.
 
 [verse]
 ____
-*queue* [*num* 'queue_number'] [*bypass*]
-*queue* [*num* 'queue_number_from' - 'queue_number_to'] ['QUEUE_FLAGS']
+*queue* [*flags* 'QUEUE_FLAGS'] [*num* 'queue_number']
+*queue* [*flags* 'QUEUE_FLAGS'] [*num* 'queue_number_from' - 'queue_number_to']
 
 'QUEUE_FLAGS' := 'QUEUE_FLAG' [*,* 'QUEUE_FLAGS']
 'QUEUE_FLAG'  := *bypass* | *fanout*
diff --git a/src/parser_bison.y b/src/parser_bison.y
index 9e45a5da1716..cf90d5ce5672 100644
--- a/src/parser_bison.y
+++ b/src/parser_bison.y
@@ -3739,6 +3739,10 @@ nf_nat_flag		:	RANDOM		{ $$ = NF_NAT_RANGE_PROTO_RANDOM; }
 			;
 
 queue_stmt		:	queue_stmt_compat	close_scope_queue
+			|	QUEUE	FLAGS	queue_stmt_flags QUEUENUM queue_stmt_expr_simple close_scope_queue
+			{
+				$$ = queue_stmt_alloc(&@$, $5, $3);
+			}
 			;
 
 queue_stmt_compat	:	queue_stmt_alloc
diff --git a/src/statement.c b/src/statement.c
index a713952c0af7..9eb49339555b 100644
--- a/src/statement.c
+++ b/src/statement.c
@@ -493,20 +493,25 @@ struct stmt *limit_stmt_alloc(const struct location *loc)
 
 static void queue_stmt_print(const struct stmt *stmt, struct output_ctx *octx)
 {
-	const char *delim = " ";
+	struct expr *e = stmt->queue.queue;
+	const char *delim = " flags ";
 
 	nft_print(octx, "queue");
-	if (stmt->queue.queue != NULL) {
-		nft_print(octx, " num ");
-		expr_print(stmt->queue.queue, octx);
-	}
+
 	if (stmt->queue.flags & NFT_QUEUE_FLAG_BYPASS) {
 		nft_print(octx, "%sbypass", delim);
 		delim = ",";
 	}
+
 	if (stmt->queue.flags & NFT_QUEUE_FLAG_CPU_FANOUT)
 		nft_print(octx, "%sfanout", delim);
 
+	if (e) {
+		nft_print(octx, " num ");
+		expr_print(stmt->queue.queue, octx);
+	} else {
+		nft_print(octx, " num 0");
+	}
 }
 
 static void queue_stmt_destroy(struct stmt *stmt)
diff --git a/tests/py/any/queue.t b/tests/py/any/queue.t
index 75c071dde44b..af844aa7c835 100644
--- a/tests/py/any/queue.t
+++ b/tests/py/any/queue.t
@@ -12,7 +12,6 @@ queue num 65535;ok
 queue num 65536;fail
 queue num 2-3;ok
 queue num 1-65535;ok
-- queue num {3, 4, 6};ok
-queue num 4-5 fanout bypass;ok;queue num 4-5 bypass,fanout
-queue num 4-5 fanout;ok
-queue num 4-5 bypass;ok
+queue num 4-5 fanout bypass;ok;queue flags bypass,fanout num 4-5
+queue num 4-5 fanout;ok;queue flags fanout num 4-5
+queue num 4-5 bypass;ok;queue flags bypass num 4-5
-- 
2.31.1

[PATCH nft 6/8] src: queue: allow use of arbitrary queue expressions

[Florian Westphal]

back in 2016 Liping Zhang added support to kernel and libnftnl to
specify a source register containing the queue number to use.

This was never added to nft itself, so allow this.
On linearization side, check if attached expression is a range.
If its not, allocate a new register and set NFTNL_EXPR_QUEUE_SREG_QNUM
attribute after generating the lowlevel expressions for the kernel.

On delinarization we need to check for presence of
NFTNL_EXPR_QUEUE_SREG_QNUM and decode the expression(s) when present.

Also need to do postprocessing for STMT_QUEUE so that the protocol
context is set correctly, without this only raw payload expressions
will be shown (@nh,32,...) instead of 'ip ...'.

Next patch adds test cases.

Signed-off-by: Florian Westphal <fw@strlen.de>
---
 doc/statements.txt        |  4 ++++
 src/evaluate.c            | 13 ++++++-----
 src/netlink_delinearize.c | 48 +++++++++++++++++++++++++++++++--------
 src/netlink_linearize.c   | 28 +++++++++++++++++++----
 src/parser_bison.y        | 16 +++++++++++--
 src/statement.c           |  9 ++++++--
 6 files changed, 93 insertions(+), 25 deletions(-)

diff --git a/doc/statements.txt b/doc/statements.txt
index 602a5b2011a7..c2a616594fce 100644
--- a/doc/statements.txt
+++ b/doc/statements.txt
@@ -591,11 +591,15 @@ for details.
 ____
 *queue* [*flags* 'QUEUE_FLAGS'] [*num* 'queue_number']
 *queue* [*flags* 'QUEUE_FLAGS'] [*num* 'queue_number_from' - 'queue_number_to']
+*queue* [*flags* 'QUEUE_FLAGS'] [*to* 'QUEUE_EXPRESSION' ]
 
 'QUEUE_FLAGS' := 'QUEUE_FLAG' [*,* 'QUEUE_FLAGS']
 'QUEUE_FLAG'  := *bypass* | *fanout*
+'QUEUE_EXPRESSION' := *numgen* | *hash* | *symhash*
 ____
 
+QUEUE_EXPRESSION can be used to compute a queue number
+at run-time with the hash or numgen expressions.
 
 .queue statement values
 [options="header"]
diff --git a/src/evaluate.c b/src/evaluate.c
index bebdb3f827e9..4a7ec95cd961 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -1019,7 +1019,6 @@ static int expr_evaluate_range(struct eval_ctx *ctx, struct expr **expr)
 	if (mpz_cmp(left->value, right->value) >= 0)
 		return expr_error(ctx->msgs, range,
 				  "Range has zero or negative size");
-
 	datatype_set(range, left->dtype);
 	range->flags |= EXPR_F_CONSTANT;
 	return 0;
@@ -3428,14 +3427,16 @@ static int stmt_evaluate_queue(struct eval_ctx *ctx, struct stmt *stmt)
 				      BYTEORDER_HOST_ENDIAN,
 				      &stmt->queue.queue) < 0)
 			return -1;
-		if (!expr_is_constant(stmt->queue.queue))
-			return expr_error(ctx->msgs, stmt->queue.queue,
-					  "queue number is not constant");
-		if (stmt->queue.queue->etype != EXPR_RANGE &&
-		    (stmt->queue.flags & NFT_QUEUE_FLAG_CPU_FANOUT))
+
+		if ((stmt->queue.flags & NFT_QUEUE_FLAG_CPU_FANOUT) &&
+		    stmt->queue.queue->etype != EXPR_RANGE)
 			return expr_error(ctx->msgs, stmt->queue.queue,
 					  "fanout requires a range to be "
 					  "specified");
+
+		if (ctx->ectx.maxval > USHRT_MAX)
+			return expr_error(ctx->msgs, stmt->queue.queue,
+					  "queue expression max value exceeds %u", USHRT_MAX);
 	}
 	stmt->flags |= STMT_F_TERMINAL;
 	return 0;
diff --git a/src/netlink_delinearize.c b/src/netlink_delinearize.c
index 7ea31e6a2639..07a6d06876e2 100644
--- a/src/netlink_delinearize.c
+++ b/src/netlink_delinearize.c
@@ -1467,19 +1467,32 @@ static void netlink_parse_queue(struct netlink_parse_ctx *ctx,
 			      const struct location *loc,
 			      const struct nftnl_expr *nle)
 {
-	uint16_t num, total, flags;
-	struct expr *expr, *high;
+	struct expr *expr;
+	uint16_t flags;
 
-	num   = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_NUM);
-	total = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_TOTAL);
+	if (nftnl_expr_is_set(nle, NFTNL_EXPR_QUEUE_SREG_QNUM)) {
+		enum nft_registers reg = netlink_parse_register(nle, NFTNL_EXPR_QUEUE_SREG_QNUM);
 
-	expr = constant_expr_alloc(loc, &integer_type,
-				   BYTEORDER_HOST_ENDIAN, 16, &num);
-	if (total > 1) {
-		total += num - 1;
-		high = constant_expr_alloc(loc, &integer_type,
+		expr = netlink_get_register(ctx, loc, reg);
+		if (!expr) {
+			netlink_error(ctx, loc, "queue statement has no sreg expression");
+			return;
+		}
+	} else {
+		uint16_t total = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_TOTAL);
+		uint16_t num = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_NUM);
+
+		expr = constant_expr_alloc(loc, &integer_type,
+					   BYTEORDER_HOST_ENDIAN, 16, &num);
+
+		if (total > 1) {
+			struct expr *high;
+
+			total += num - 1;
+			high = constant_expr_alloc(loc, &integer_type,
 					   BYTEORDER_HOST_ENDIAN, 16, &total);
-		expr = range_expr_alloc(loc, expr, high);
+			expr = range_expr_alloc(loc, expr, high);
+		}
 	}
 
 	flags = nftnl_expr_get_u16(nle, NFTNL_EXPR_QUEUE_FLAGS);
@@ -2788,6 +2801,18 @@ static void stmt_payload_postprocess(struct rule_pp_ctx *ctx)
 	expr_postprocess(ctx, &stmt->payload.val);
 }
 
+static void stmt_queue_postprocess(struct rule_pp_ctx *ctx)
+{
+	struct stmt *stmt = ctx->stmt;
+	struct expr *e = stmt->queue.queue;
+
+	if (e == NULL || e->etype == EXPR_VALUE ||
+	    e->etype == EXPR_RANGE)
+		return;
+
+	expr_postprocess(ctx, &stmt->queue.queue);
+}
+
 /*
  * We can only remove payload dependencies if they occur without
  * a statement with side effects in between.
@@ -2889,6 +2914,9 @@ static void rule_parse_postprocess(struct netlink_parse_ctx *ctx, struct rule *r
 		case STMT_OBJREF:
 			expr_postprocess(&rctx, &stmt->objref.expr);
 			break;
+		case STMT_QUEUE:
+			stmt_queue_postprocess(&rctx);
+			break;
 		default:
 			break;
 		}
diff --git a/src/netlink_linearize.c b/src/netlink_linearize.c
index 7b35aae1f913..b1f3feeeb4b7 100644
--- a/src/netlink_linearize.c
+++ b/src/netlink_linearize.c
@@ -1334,21 +1334,39 @@ static void netlink_gen_fwd_stmt(struct netlink_linearize_ctx *ctx,
 static void netlink_gen_queue_stmt(struct netlink_linearize_ctx *ctx,
 				 const struct stmt *stmt)
 {
+	enum nft_registers sreg = 0;
 	struct nftnl_expr *nle;
 	uint16_t total_queues;
+	struct expr *expr;
 	mpz_t low, high;
 
 	mpz_init2(low, 16);
 	mpz_init2(high, 16);
-	if (stmt->queue.queue != NULL) {
-		range_expr_value_low(low, stmt->queue.queue);
-		range_expr_value_high(high, stmt->queue.queue);
+
+	expr = stmt->queue.queue;
+
+	if (expr) {
+		if (expr->etype == EXPR_RANGE || expr->etype == EXPR_VALUE) {
+			range_expr_value_low(low, stmt->queue.queue);
+			range_expr_value_high(high, stmt->queue.queue);
+		} else {
+			sreg = get_register(ctx, expr);
+			netlink_gen_expr(ctx, expr, sreg);
+			release_register(ctx, expr);
+		}
 	}
+
 	total_queues = mpz_get_uint16(high) - mpz_get_uint16(low) + 1;
 
 	nle = alloc_nft_expr("queue");
-	nftnl_expr_set_u16(nle, NFTNL_EXPR_QUEUE_NUM, mpz_get_uint16(low));
-	nftnl_expr_set_u16(nle, NFTNL_EXPR_QUEUE_TOTAL, total_queues);
+
+	if (sreg) {
+		netlink_put_register(nle, NFTNL_EXPR_QUEUE_SREG_QNUM, sreg);
+	} else {
+		nftnl_expr_set_u16(nle, NFTNL_EXPR_QUEUE_NUM, mpz_get_uint16(low));
+		nftnl_expr_set_u16(nle, NFTNL_EXPR_QUEUE_TOTAL, total_queues);
+	}
+
 	if (stmt->queue.flags)
 		nftnl_expr_set_u16(nle, NFTNL_EXPR_QUEUE_FLAGS,
 				   stmt->queue.flags);
diff --git a/src/parser_bison.y b/src/parser_bison.y
index cf90d5ce5672..d75960715a90 100644
--- a/src/parser_bison.y
+++ b/src/parser_bison.y
@@ -705,8 +705,8 @@ int nft_lex(void *, void *, void *);
 
 %type <stmt>			queue_stmt queue_stmt_alloc	queue_stmt_compat
 %destructor { stmt_free($$); }	queue_stmt queue_stmt_alloc	queue_stmt_compat
-%type <expr>			queue_stmt_expr_simple
-%destructor { expr_free($$); }	queue_stmt_expr_simple
+%type <expr>			queue_stmt_expr_simple queue_stmt_expr
+%destructor { expr_free($$); }	queue_stmt_expr_simple queue_stmt_expr
 %type <val>			queue_stmt_flags queue_stmt_flag
 %type <stmt>			dup_stmt
 %destructor { stmt_free($$); }	dup_stmt
@@ -3739,6 +3739,14 @@ nf_nat_flag		:	RANDOM		{ $$ = NF_NAT_RANGE_PROTO_RANDOM; }
 			;
 
 queue_stmt		:	queue_stmt_compat	close_scope_queue
+			|	QUEUE TO queue_stmt_expr	close_scope_queue
+			{
+				$$ = queue_stmt_alloc(&@$, $3, 0);
+			}
+			|	QUEUE FLAGS	queue_stmt_flags TO queue_stmt_expr close_scope_queue
+			{
+				$$ = queue_stmt_alloc(&@$, $5, $3);
+			}
 			|	QUEUE	FLAGS	queue_stmt_flags QUEUENUM queue_stmt_expr_simple close_scope_queue
 			{
 				$$ = queue_stmt_alloc(&@$, $5, $3);
@@ -3777,6 +3785,10 @@ queue_stmt_expr_simple	:	integer_expr
 			|	range_rhs_expr
 			;
 
+queue_stmt_expr		:	numgen_expr
+			|	hash_expr
+			;
+
 queue_stmt_flags	:	queue_stmt_flag
 			|	queue_stmt_flags	COMMA	queue_stmt_flag
 			{
diff --git a/src/statement.c b/src/statement.c
index 9eb49339555b..dfd275104c59 100644
--- a/src/statement.c
+++ b/src/statement.c
@@ -507,8 +507,13 @@ static void queue_stmt_print(const struct stmt *stmt, struct output_ctx *octx)
 		nft_print(octx, "%sfanout", delim);
 
 	if (e) {
-		nft_print(octx, " num ");
-		expr_print(stmt->queue.queue, octx);
+		if (e->etype == EXPR_VALUE || e->etype == EXPR_RANGE) {
+			nft_print(octx, " num ");
+			expr_print(stmt->queue.queue, octx);
+		} else {
+			nft_print(octx, " to ");
+			expr_print(stmt->queue.queue, octx);
+		}
 	} else {
 		nft_print(octx, " num 0");
 	}
-- 
2.31.1

[PATCH nft 7/8] tests: extend queue testcases for new sreg support

[Florian Westphal]

Signed-off-by: Florian Westphal <fw@strlen.de>
---
 tests/py/any/queue.t         | 10 +++++++
 tests/py/any/queue.t.json    | 56 ++++++++++++++++++++++++++++++++++++
 tests/py/any/queue.t.payload | 16 +++++++++++
 3 files changed, 82 insertions(+)

diff --git a/tests/py/any/queue.t b/tests/py/any/queue.t
index af844aa7c835..670dfd92d5b0 100644
--- a/tests/py/any/queue.t
+++ b/tests/py/any/queue.t
@@ -15,3 +15,13 @@ queue num 1-65535;ok
 queue num 4-5 fanout bypass;ok;queue flags bypass,fanout num 4-5
 queue num 4-5 fanout;ok;queue flags fanout num 4-5
 queue num 4-5 bypass;ok;queue flags bypass num 4-5
+
+queue to symhash mod 2 offset 65536;fail
+queue num symhash mod 65536;fail
+queue to symhash mod 65536;ok
+queue flags fanout to symhash mod 65536;fail
+queue flags bypass,fanout to symhash mod 65536;fail
+queue flags bypass to numgen inc mod 65536;ok
+queue to jhash oif . meta mark mod 32;ok
+queue to oif;fail
+queue num oif;fail
diff --git a/tests/py/any/queue.t.json b/tests/py/any/queue.t.json
index 48e86727a2ff..18ed3c817ac9 100644
--- a/tests/py/any/queue.t.json
+++ b/tests/py/any/queue.t.json
@@ -84,3 +84,59 @@
     }
 ]
 
+# queue to symhash mod 65536
+[
+    {
+        "queue": {
+            "num": {
+                "symhash": {
+                    "mod": 65536
+                }
+            }
+        }
+    }
+]
+
+# queue flags bypass to numgen inc mod 65536
+[
+    {
+        "queue": {
+            "flags": "bypass",
+            "num": {
+                "numgen": {
+                    "mod": 65536,
+                    "mode": "inc",
+                    "offset": 0
+                }
+            }
+        }
+    }
+]
+
+# queue to jhash oif . meta mark mod 32
+[
+    {
+        "queue": {
+            "num": {
+                "jhash": {
+                    "expr": {
+                        "concat": [
+                            {
+                                "meta": {
+                                    "key": "oif"
+                                }
+                            },
+                            {
+                                "meta": {
+                                    "key": "mark"
+                                }
+                            }
+                        ]
+                    },
+                    "mod": 32
+                }
+            }
+        }
+    }
+]
+
diff --git a/tests/py/any/queue.t.payload b/tests/py/any/queue.t.payload
index 78d939c692e9..35e757ee5cf0 100644
--- a/tests/py/any/queue.t.payload
+++ b/tests/py/any/queue.t.payload
@@ -30,3 +30,19 @@ ip test-ip4 output
 ip test-ip4 output
   [ queue num 4-5 bypass ]
 
+# queue to symhash mod 65536
+ip
+  [ hash reg 1 = symhash() % mod 65536 ]
+  [ queue sreg_qnum 1 ]
+
+# queue to jhash oif . meta mark mod 32
+ip
+  [ meta load oif => reg 2 ]
+  [ meta load mark => reg 13 ]
+  [ hash reg 1 = jhash(reg 2, 8, 0x0) % mod 32 ]
+  [ queue sreg_qnum 1 ]
+
+# queue flags bypass to numgen inc mod 65536
+ip
+  [ numgen reg 1 = inc mod 65536 ]
+  [ queue sreg_qnum 1 bypass ]
-- 
2.31.1

[PATCH nft 8/8] src: queue: allow use of MAP statement for queue number retrieval

[Florian Westphal]

This allows to chose a queue number at run time using map statements,
e.g.:

queue flags bypass to ip saddr map { 192.168.7/24 : 0, 192.168.0/24 : 1 }

Signed-off-by: Florian Westphal <fw@strlen.de>
---
 doc/statements.txt           |  6 ++++--
 src/parser_bison.y           |  1 +
 tests/py/any/queue.t         |  1 +
 tests/py/any/queue.t.json    | 34 ++++++++++++++++++++++++++++++++++
 tests/py/any/queue.t.payload |  9 +++++++++
 5 files changed, 49 insertions(+), 2 deletions(-)

diff --git a/doc/statements.txt b/doc/statements.txt
index c2a616594fce..097cf2e07eeb 100644
--- a/doc/statements.txt
+++ b/doc/statements.txt
@@ -595,11 +595,13 @@ ____
 
 'QUEUE_FLAGS' := 'QUEUE_FLAG' [*,* 'QUEUE_FLAGS']
 'QUEUE_FLAG'  := *bypass* | *fanout*
-'QUEUE_EXPRESSION' := *numgen* | *hash* | *symhash*
+'QUEUE_EXPRESSION' := *numgen* | *hash* | *symhash* | *MAP STATEMENT*
 ____
 
 QUEUE_EXPRESSION can be used to compute a queue number
-at run-time with the hash or numgen expressions.
+at run-time with the hash or numgen expressions. It also
+allows to use the map statement to assign fixed queue numbers
+based on external inputs such as the source ip address or interface names.
 
 .queue statement values
 [options="header"]
diff --git a/src/parser_bison.y b/src/parser_bison.y
index d75960715a90..2615db1ba14b 100644
--- a/src/parser_bison.y
+++ b/src/parser_bison.y
@@ -3787,6 +3787,7 @@ queue_stmt_expr_simple	:	integer_expr
 
 queue_stmt_expr		:	numgen_expr
 			|	hash_expr
+			|	map_expr
 			;
 
 queue_stmt_flags	:	queue_stmt_flag
diff --git a/tests/py/any/queue.t b/tests/py/any/queue.t
index 670dfd92d5b0..446b8b1806f2 100644
--- a/tests/py/any/queue.t
+++ b/tests/py/any/queue.t
@@ -25,3 +25,4 @@ queue flags bypass to numgen inc mod 65536;ok
 queue to jhash oif . meta mark mod 32;ok
 queue to oif;fail
 queue num oif;fail
+queue flags bypass to oifname map { "eth0" : 0, "ppp0" : 2, "eth1" : 2 };ok
diff --git a/tests/py/any/queue.t.json b/tests/py/any/queue.t.json
index 18ed3c817ac9..162bdff875d6 100644
--- a/tests/py/any/queue.t.json
+++ b/tests/py/any/queue.t.json
@@ -140,3 +140,37 @@
     }
 ]
 
+# queue flags bypass to oifname map { "eth0" : 0, "ppp0" : 2, "eth1" : 2 }
+[
+    {
+        "queue": {
+            "flags": "bypass",
+            "num": {
+                "map": {
+                    "data": {
+                        "set": [
+                            [
+                                "eth0",
+                                0
+                            ],
+                            [
+                                "ppp0",
+                                2
+                            ],
+                            [
+                                "eth1",
+                                2
+                            ]
+                        ]
+                    },
+                    "key": {
+                        "meta": {
+                            "key": "oifname"
+                        }
+                    }
+                }
+            }
+        }
+    }
+]
+
diff --git a/tests/py/any/queue.t.payload b/tests/py/any/queue.t.payload
index 35e757ee5cf0..02660afa8d30 100644
--- a/tests/py/any/queue.t.payload
+++ b/tests/py/any/queue.t.payload
@@ -46,3 +46,12 @@ ip
 ip
   [ numgen reg 1 = inc mod 65536 ]
   [ queue sreg_qnum 1 bypass ]
+
+# queue flags bypass to oifname map { "eth0" : 0, "ppp0" : 2, "eth1" : 2 }
+__map%d test-ip4 b size 3
+__map%d test-ip4 0
+	element 30687465 00000000 00000000 00000000  : 00000000 0 [end]	element 30707070 00000000 00000000 00000000  : 00000002 0 [end]	element 31687465 00000000 00000000 00000000  : 00000002 0 [end]
+ip
+  [ meta load oifname => reg 1 ]
+  [ lookup reg 1 set __map%d dreg 1 ]
+  [ queue sreg_qnum 1 bypass ]
-- 
2.31.1