netfilter-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH nft v5 00/14] cache consolidation
@ 2015-08-10 11:50 Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 01/14] src: add cache infrastructure and use it for table objects Pablo Neira Ayuso
                   ` (13 more replies)
  0 siblings, 14 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

Hi,

This is another round of the patchset to consolidate the nft cache:

http://marc.info/?l=netfilter-devel&m=143620630223923&w=2

The idea consists of creating a cache of tables that is populated with chains,
rules, sets and elements.

The major change in this round is the fact that the cache is built later on
from the evaluation step where we have more context on what is going on. At
that stage, we know if this is a listing, in that case nft populates a full
blown cache. Otherwise, it only retrieves the table and set objects which is
the bare minimum that we require at this stage (at least by now). With the
previous patchset, with inconditional full blown cache retrieval, I noticed a
slowdown when working with large rulesets, so this new round patchset round
addresses this.

In the interactive mode, this cache is refreshed for every command to make sure
that we work with a ruleset that is current. We can avoid this by checking for
the generation counter, but will be looking into how to speed up this with
follow up patches when this is in master.

Comments welcome. Thanks.

Pablo Neira Ayuso (14):
  src: add cache infrastructure and use it for table objects
  src: add cmd_evaluate_list()
  rule: add reference counter to the table object
  src: add table declaration to cache
  src: use cache infrastructure for set objects
  src: add set declaration to cache
  src: early allocation of the set ID
  rule: add chain reference counter
  src: use cache infrastructure for chain objects
  evaluate: add cmd_evaluate_rename()
  src: add chain declarations to cache
  src: use cache infrastructure for rule objects
  src: use cache infrastructure for set element objects
  src: get rid of EINTR handling for nft_netlink()

 include/rule.h |    9 ++
 src/cli.c      |    1 +
 src/evaluate.c |  146 +++++++++++++++++-------
 src/main.c     |    7 +-
 src/netlink.c  |    4 -
 src/rule.c     |  337 ++++++++++++++++++++++++++++++++------------------------
 6 files changed, 310 insertions(+), 194 deletions(-)

-- 
1.7.10.4


^ permalink raw reply	[flat|nested] 15+ messages in thread

* [PATCH nft v5 01/14] src: add cache infrastructure and use it for table objects
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 02/14] src: add cmd_evaluate_list() Pablo Neira Ayuso
                   ` (12 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

This patch introduces the generic object cache that is populated during the
evaluation phase.

The first client of this infrastructure are table objects. As a result, there
is a single call to netlink_list_tables().

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 include/rule.h |    3 ++
 src/cli.c      |    1 +
 src/evaluate.c |    6 ++++
 src/main.c     |    2 +-
 src/rule.c     |   98 +++++++++++++++++++++++++++++++++++++++++++-------------
 5 files changed, 86 insertions(+), 24 deletions(-)

diff --git a/include/rule.h b/include/rule.h
index 8ec7f91..1520222 100644
--- a/include/rule.h
+++ b/include/rule.h
@@ -381,4 +381,7 @@ extern struct error_record *rule_postprocess(struct rule *rule);
 struct netlink_ctx;
 extern int do_command(struct netlink_ctx *ctx, struct cmd *cmd);
 
+extern int cache_update(enum cmd_ops cmd, struct list_head *msgs);
+extern void cache_release(void);
+
 #endif /* NFTABLES_RULE_H */
diff --git a/src/cli.c b/src/cli.c
index fbae0ef..adffd6b 100644
--- a/src/cli.c
+++ b/src/cli.c
@@ -124,6 +124,7 @@ static void cli_complete(char *line)
 	nft_run(scanner, state, &msgs);
 	erec_print_list(stdout, &msgs);
 	xfree(line);
+	cache_release();
 	iface_cache_release();
 }
 
diff --git a/src/evaluate.c b/src/evaluate.c
index d99b38f..f8c7372 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -2001,6 +2001,12 @@ static int cmd_evaluate_monitor(struct eval_ctx *ctx, struct cmd *cmd)
 
 int cmd_evaluate(struct eval_ctx *ctx, struct cmd *cmd)
 {
+	int ret;
+
+	ret = cache_update(cmd->op, ctx->msgs);
+	if (ret < 0)
+		return ret;
+
 #ifdef DEBUG
 	if (debug_level & DEBUG_EVALUATION) {
 		struct error_record *erec;
diff --git a/src/main.c b/src/main.c
index a2c4f87..d6c9ccc 100644
--- a/src/main.c
+++ b/src/main.c
@@ -182,7 +182,6 @@ static int nft_netlink(struct parser_state *state, struct list_head *msgs)
 	bool batch_supported = netlink_batch_supported();
 	int ret = 0;
 
-	netlink_genid_get();
 	mnl_batch_init();
 
 	batch_seqnum = mnl_batch_begin();
@@ -366,6 +365,7 @@ out:
 	scanner_destroy(scanner);
 	erec_print_list(stderr, &msgs);
 	xfree(buf);
+	cache_release();
 	iface_cache_release();
 
 	return rc;
diff --git a/src/rule.c b/src/rule.c
index 09d1ffd..60c5d72 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -53,6 +53,73 @@ void handle_merge(struct handle *dst, const struct handle *src)
 		dst->comment = xstrdup(src->comment);
 }
 
+static LIST_HEAD(table_list);
+
+static int cache_init_tables(struct netlink_ctx *ctx, struct handle *h)
+{
+	int ret;
+
+	ret = netlink_list_tables(ctx, h, &internal_location);
+	if (ret < 0)
+		return -1;
+
+	list_splice_tail_init(&ctx->list, &table_list);
+	return 0;
+}
+
+static int cache_init(enum cmd_ops cmd, struct list_head *msgs)
+{
+	struct handle handle = {
+		.family = NFPROTO_UNSPEC,
+	};
+	struct netlink_ctx ctx;
+	int ret;
+
+	memset(&ctx, 0, sizeof(ctx));
+	init_list_head(&ctx.list);
+	ctx.msgs = msgs;
+
+	ret = cache_init_tables(&ctx, &handle);
+	if (ret < 0)
+		return ret;
+
+	return 0;
+}
+
+static bool cache_initialized;
+
+int cache_update(enum cmd_ops cmd, struct list_head *msgs)
+{
+	int ret;
+
+	if (cache_initialized)
+		return 0;
+replay:
+	netlink_genid_get();
+	ret = cache_init(cmd, msgs);
+	if (ret < 0) {
+		if (errno == EINTR) {
+			netlink_restart();
+			goto replay;
+		}
+		cache_release();
+		return -1;
+	}
+	cache_initialized = true;
+	return 0;
+}
+
+void cache_release(void)
+{
+	struct table *table, *next;
+
+	list_for_each_entry_safe(table, next, &table_list, list) {
+		list_del(&table->list);
+		table_free(table);
+	}
+	cache_initialized = false;
+}
+
 struct set *set_alloc(const struct location *loc)
 {
 	struct set *set;
@@ -516,8 +583,6 @@ void table_free(struct table *table)
 	xfree(table);
 }
 
-static LIST_HEAD(table_list);
-
 void table_add_hash(struct table *table)
 {
 	list_add_tail(&table->list, &table_list);
@@ -845,8 +910,6 @@ static int do_list_table(struct netlink_ctx *ctx, struct cmd *cmd,
 	struct rule *rule, *nrule;
 	struct chain *chain;
 
-	if (netlink_get_table(ctx, &cmd->handle, &cmd->location, table) < 0)
-		goto err;
 	if (do_list_sets(ctx, &cmd->location, table) < 0)
 		goto err;
 	if (netlink_list_chains(ctx, &cmd->handle, &cmd->location) < 0)
@@ -876,25 +939,19 @@ err:
 
 static int do_list_ruleset(struct netlink_ctx *ctx, struct cmd *cmd)
 {
-	struct table *table, *next;
-	LIST_HEAD(tables);
-
-	if (netlink_list_tables(ctx, &cmd->handle, &cmd->location) < 0)
-		return -1;
-
-	list_splice_tail_init(&ctx->list, &tables);
+	unsigned int family = cmd->handle.family;
+	struct table *table;
 
-	list_for_each_entry_safe(table, next, &tables, list) {
-		table_add_hash(table);
+	list_for_each_entry(table, &table_list, list) {
+		if (family != NFPROTO_UNSPEC &&
+		    table->handle.family != family)
+			continue;
 
 		cmd->handle.family = table->handle.family;
 		cmd->handle.table = xstrdup(table->handle.table);
 
 		if (do_list_table(ctx, cmd, table) < 0)
 			return -1;
-
-		list_del(&table->list);
-		table_free(table);
 	}
 
 	return 0;
@@ -1015,7 +1072,7 @@ static int do_command_rename(struct netlink_ctx *ctx, struct cmd *cmd)
 
 static int do_command_monitor(struct netlink_ctx *ctx, struct cmd *cmd)
 {
-	struct table *t, *nt;
+	struct table *t;
 	struct set *s, *ns;
 	struct netlink_ctx set_ctx;
 	LIST_HEAD(msgs);
@@ -1038,10 +1095,7 @@ static int do_command_monitor(struct netlink_ctx *ctx, struct cmd *cmd)
 		init_list_head(&msgs);
 		set_ctx.msgs = &msgs;
 
-		if (netlink_list_tables(ctx, &cmd->handle, &cmd->location) < 0)
-			return -1;
-
-		list_for_each_entry_safe(t, nt, &ctx->list, list) {
+		list_for_each_entry(t, &table_list, list) {
 			set_handle.family = t->handle.family;
 			set_handle.table = t->handle.table;
 
@@ -1055,8 +1109,6 @@ static int do_command_monitor(struct netlink_ctx *ctx, struct cmd *cmd)
 				s->init = set_expr_alloc(&cmd->location);
 				set_add_hash(s, t);
 			}
-
-			table_add_hash(t);
 		}
 	}
 
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 02/14] src: add cmd_evaluate_list()
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 01/14] src: add cache infrastructure and use it for table objects Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 03/14] rule: add reference counter to the table object Pablo Neira Ayuso
                   ` (11 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

This function validates that the table that we want to list already exists by
looking it up from the cache.

This also adds cmd_error() to display an error from the evaluation step, when
the objects that the rule indicates do not exist.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/evaluate.c |   23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/src/evaluate.c b/src/evaluate.c
index f8c7372..8598eef 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -62,6 +62,8 @@ static int __fmtstring(4, 5) __stmt_binary_error(struct eval_ctx *ctx,
 	__stmt_binary_error(ctx, &(s1)->location, NULL, fmt, ## args)
 #define monitor_error(ctx, s1, fmt, args...) \
 	__stmt_binary_error(ctx, &(s1)->location, NULL, fmt, ## args)
+#define cmd_error(ctx, fmt, args...) \
+	__stmt_binary_error(ctx, &(ctx->cmd)->location, NULL, fmt, ## args)
 
 static int __fmtstring(3, 4) set_error(struct eval_ctx *ctx,
 				       const struct set *set,
@@ -1933,6 +1935,26 @@ static int cmd_evaluate_delete(struct eval_ctx *ctx, struct cmd *cmd)
 	}
 }
 
+static int cmd_evaluate_list(struct eval_ctx *ctx, struct cmd *cmd)
+{
+	switch (cmd->obj) {
+	case CMD_OBJ_TABLE:
+		if (cmd->handle.table == NULL)
+			return 0;
+	case CMD_OBJ_CHAIN:
+	case CMD_OBJ_SETS:
+	case CMD_OBJ_SET:
+		if (table_lookup(&cmd->handle) == NULL)
+			return cmd_error(ctx, "Could not process rule: Table '%s' does not exist",
+					 cmd->handle.table);
+		return 0;
+	case CMD_OBJ_RULESET:
+		return 0;
+	default:
+		BUG("invalid command object type %u\n", cmd->obj);
+	}
+}
+
 enum {
 	CMD_MONITOR_EVENT_ANY,
 	CMD_MONITOR_EVENT_NEW,
@@ -2024,6 +2046,7 @@ int cmd_evaluate(struct eval_ctx *ctx, struct cmd *cmd)
 	case CMD_DELETE:
 		return cmd_evaluate_delete(ctx, cmd);
 	case CMD_LIST:
+		return cmd_evaluate_list(ctx, cmd);
 	case CMD_FLUSH:
 	case CMD_RENAME:
 	case CMD_EXPORT:
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 03/14] rule: add reference counter to the table object
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 01/14] src: add cache infrastructure and use it for table objects Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 02/14] src: add cmd_evaluate_list() Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 04/14] src: add table declaration to cache Pablo Neira Ayuso
                   ` (10 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

We may hold multiple references to table objects in follow up patches when
adding object declarations to the cache.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 include/rule.h |    3 +++
 src/rule.c     |   10 ++++++++++
 2 files changed, 13 insertions(+)

diff --git a/include/rule.h b/include/rule.h
index 1520222..cc4e5c2 100644
--- a/include/rule.h
+++ b/include/rule.h
@@ -76,6 +76,7 @@ enum table_flags {
  * @chains:	chains contained in the table
  * @sets:	sets contained in the table
  * @flags:	table flags
+ * @refcnt:	table reference counter
  */
 struct table {
 	struct list_head	list;
@@ -85,9 +86,11 @@ struct table {
 	struct list_head	chains;
 	struct list_head	sets;
 	enum table_flags 	flags;
+	unsigned int		refcnt;
 };
 
 extern struct table *table_alloc(void);
+extern struct table *table_get(struct table *table);
 extern void table_free(struct table *table);
 extern void table_add_hash(struct table *table);
 extern struct table *table_lookup(const struct handle *h);
diff --git a/src/rule.c b/src/rule.c
index 60c5d72..cb69369 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -569,6 +569,8 @@ struct table *table_alloc(void)
 	init_list_head(&table->chains);
 	init_list_head(&table->sets);
 	init_list_head(&table->scope.symbols);
+	table->refcnt = 1;
+
 	return table;
 }
 
@@ -576,6 +578,8 @@ void table_free(struct table *table)
 {
 	struct chain *chain, *next;
 
+	if (--table->refcnt > 0)
+		return;
 	list_for_each_entry_safe(chain, next, &table->chains, list)
 		chain_free(chain);
 	handle_free(&table->handle);
@@ -583,6 +587,12 @@ void table_free(struct table *table)
 	xfree(table);
 }
 
+struct table *table_get(struct table *table)
+{
+	table->refcnt++;
+	return table;
+}
+
 void table_add_hash(struct table *table)
 {
 	list_add_tail(&table->list, &table_list);
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 04/14] src: add table declaration to cache
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (2 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 03/14] rule: add reference counter to the table object Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 05/14] src: use cache infrastructure for set objects Pablo Neira Ayuso
                   ` (9 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

Add declared table objects to the cache, thus we can refer to objects that
come in this batch but that are not yet available in the kernel. This happens
from the evaluation step.

Get rid of code that is doing this from the later do_command_*() stage.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/evaluate.c |   15 +++++++++++++--
 src/rule.c     |   15 ++-------------
 2 files changed, 15 insertions(+), 15 deletions(-)

diff --git a/src/evaluate.c b/src/evaluate.c
index 8598eef..865b707 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -1881,6 +1881,19 @@ static int table_evaluate(struct eval_ctx *ctx, struct table *table)
 	struct chain *chain;
 	struct set *set;
 
+	if (table_lookup(&ctx->cmd->handle) == NULL) {
+		if (table == NULL) {
+			table = table_alloc();
+			handle_merge(&table->handle, &ctx->cmd->handle);
+			table_add_hash(table);
+		} else {
+			table_add_hash(table_get(table));
+		}
+	}
+
+	if (ctx->cmd->table == NULL)
+		return 0;
+
 	ctx->table = table;
 	list_for_each_entry(set, &table->sets, list) {
 		handle_merge(&set->handle, &table->handle);
@@ -1912,8 +1925,6 @@ static int cmd_evaluate_add(struct eval_ctx *ctx, struct cmd *cmd)
 			return 0;
 		return chain_evaluate(ctx, cmd->chain);
 	case CMD_OBJ_TABLE:
-		if (cmd->data == NULL)
-			return 0;
 		return table_evaluate(ctx, cmd->table);
 	default:
 		BUG("invalid command object type %u\n", cmd->obj);
diff --git a/src/rule.c b/src/rule.c
index cb69369..1e5b1a8 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -987,15 +987,8 @@ static int do_command_list(struct netlink_ctx *ctx, struct cmd *cmd)
 	struct table *table = NULL;
 	struct set *set;
 
-	/* No need to allocate the table object when listing all tables */
-	if (cmd->handle.table != NULL) {
+	if (cmd->handle.table != NULL)
 		table = table_lookup(&cmd->handle);
-		if (table == NULL) {
-			table = table_alloc();
-			handle_merge(&table->handle, &cmd->handle);
-			table_add_hash(table);
-		}
-	}
 
 	switch (cmd->obj) {
 	case CMD_OBJ_TABLE:
@@ -1056,14 +1049,10 @@ static int do_command_flush(struct netlink_ctx *ctx, struct cmd *cmd)
 
 static int do_command_rename(struct netlink_ctx *ctx, struct cmd *cmd)
 {
-	struct table *table;
+	struct table *table = table_lookup(&cmd->handle);
 	struct chain *chain;
 	int err;
 
-	table = table_alloc();
-	handle_merge(&table->handle, &cmd->handle);
-	table_add_hash(table);
-
 	switch (cmd->obj) {
 	case CMD_OBJ_CHAIN:
 		err = netlink_get_chain(ctx, &cmd->handle, &cmd->location);
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 05/14] src: use cache infrastructure for set objects
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (3 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 04/14] src: add table declaration to cache Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 06/14] src: add set declaration to cache Pablo Neira Ayuso
                   ` (8 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

This patch populates the cache only once through netlink_list_sets() during
evaluation. As a result, there is a single call to netlink_list_sets().

After this change, we can rid of get_set(). This function was fine by the time
we had no transaction support, but this doesn't work for set objects that are
declared in this batch, so inquiring the kernel doesn't help since they are not
yet available.

As a result from this update, the monitor code gets simplified quite a lot
since it can rely of the set cache. Moreover, we can now validate that the
table and set exists from evaluation path.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/evaluate.c |   53 +++++++++------------------
 src/rule.c     |  112 ++++++++++++++++++++++++++++++--------------------------
 2 files changed, 79 insertions(+), 86 deletions(-)

diff --git a/src/evaluate.c b/src/evaluate.c
index 865b707..31aaecc 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -109,37 +109,6 @@ static struct expr *implicit_set_declaration(struct eval_ctx *ctx,
 	return set_ref_expr_alloc(&expr->location, set);
 }
 
-// FIXME
-#include <netlink.h>
-static struct set *get_set(struct eval_ctx *ctx, const struct handle *h,
-			   const char *identifier)
-{
-	struct netlink_ctx nctx = {
-		.msgs = ctx->msgs,
-	};
-	struct handle handle;
-	struct set *set;
-	int err;
-
-	if (ctx->table != NULL) {
-		set = set_lookup(ctx->table, identifier);
-		if (set != NULL)
-			return set;
-	}
-
-	init_list_head(&nctx.list);
-
-	memset(&handle, 0, sizeof(handle));
-	handle_merge(&handle, h);
-	handle.set = xstrdup(identifier);
-	err = netlink_get_set(&nctx, &handle, &internal_location);
-	handle_free(&handle);
-
-	if (err < 0)
-		return NULL;
-	return list_first_entry(&nctx.list, struct set, list);
-}
-
 static enum ops byteorder_conversion_op(struct expr *expr,
 					enum byteorder byteorder)
 {
@@ -192,6 +161,7 @@ static int expr_evaluate_symbol(struct eval_ctx *ctx, struct expr **expr)
 {
 	struct error_record *erec;
 	struct symbol *sym;
+	struct table *table;
 	struct set *set;
 	struct expr *new;
 
@@ -213,9 +183,15 @@ static int expr_evaluate_symbol(struct eval_ctx *ctx, struct expr **expr)
 		new = expr_clone(sym->expr);
 		break;
 	case SYMBOL_SET:
-		set = get_set(ctx, &ctx->cmd->handle, (*expr)->identifier);
+		table = table_lookup(&ctx->cmd->handle);
+		if (table == NULL)
+			return cmd_error(ctx, "Could not process rule: Table '%s' does not exist",
+					 ctx->cmd->handle.table);
+
+		set = set_lookup(table, (*expr)->identifier);
 		if (set == NULL)
-			return -1;
+			return cmd_error(ctx, "Could not process rule: Set '%s' does not exist",
+					 (*expr)->identifier);
 		new = set_ref_expr_alloc(&(*expr)->location, set);
 		break;
 	}
@@ -1737,11 +1713,18 @@ int stmt_evaluate(struct eval_ctx *ctx, struct stmt *stmt)
 
 static int setelem_evaluate(struct eval_ctx *ctx, struct expr **expr)
 {
+	struct table *table;
 	struct set *set;
 
-	set = get_set(ctx, &ctx->cmd->handle, ctx->cmd->handle.set);
+	table = table_lookup(&ctx->cmd->handle);
+	if (table == NULL)
+		return cmd_error(ctx, "Could not process rule: Table '%s' does not exist",
+				 ctx->cmd->handle.table);
+
+	set = set_lookup(table, ctx->cmd->handle.set);
 	if (set == NULL)
-		return -1;
+		return cmd_error(ctx, "Could not process rule: Set '%s' does not exist",
+				 ctx->cmd->handle.set);
 
 	ctx->set = set;
 	expr_set_context(&ctx->ectx, set->keytype, set->keylen);
diff --git a/src/rule.c b/src/rule.c
index 1e5b1a8..32c4a15 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -67,6 +67,22 @@ static int cache_init_tables(struct netlink_ctx *ctx, struct handle *h)
 	return 0;
 }
 
+static int cache_init_objects(struct netlink_ctx *ctx)
+{
+	struct table *table;
+	int ret;
+
+	list_for_each_entry(table, &table_list, list) {
+		ret = netlink_list_sets(ctx, &table->handle,
+					&internal_location);
+		list_splice_tail_init(&ctx->list, &table->sets);
+
+		if (ret < 0)
+			return -1;
+	}
+	return 0;
+}
+
 static int cache_init(enum cmd_ops cmd, struct list_head *msgs)
 {
 	struct handle handle = {
@@ -82,6 +98,9 @@ static int cache_init(enum cmd_ops cmd, struct list_head *msgs)
 	ret = cache_init_tables(&ctx, &handle);
 	if (ret < 0)
 		return ret;
+	ret = cache_init_objects(&ctx);
+	if (ret < 0)
+		return ret;
 
 	return 0;
 }
@@ -577,11 +596,14 @@ struct table *table_alloc(void)
 void table_free(struct table *table)
 {
 	struct chain *chain, *next;
+	struct set *set, *nset;
 
 	if (--table->refcnt > 0)
 		return;
 	list_for_each_entry_safe(chain, next, &table->chains, list)
 		chain_free(chain);
+	list_for_each_entry_safe(set, nset, &table->sets, list)
+		set_free(set);
 	handle_free(&table->handle);
 	scope_release(&table->scope);
 	xfree(table);
@@ -870,15 +892,11 @@ static int do_command_delete(struct netlink_ctx *ctx, struct cmd *cmd)
 static int do_list_sets(struct netlink_ctx *ctx, const struct location *loc,
 			struct table *table)
 {
-	struct set *set, *nset;
-
-	if (netlink_list_sets(ctx, &table->handle, loc) < 0)
-		return -1;
+	struct set *set;
 
-	list_for_each_entry_safe(set, nset, &ctx->list, list) {
+	list_for_each_entry(set, &table->sets, list) {
 		if (netlink_get_setelems(ctx, &set->handle, loc, set) < 0)
 			return -1;
-		list_move_tail(&set->list, &table->sets);
 	}
 	return 0;
 }
@@ -947,6 +965,23 @@ err:
 	return -1;
 }
 
+static int do_list_sets_global(struct netlink_ctx *ctx, struct cmd *cmd)
+{
+	struct table *table;
+	struct set *set;
+
+	list_for_each_entry(table, &table_list, list) {
+		list_for_each_entry(set, &table->sets, list) {
+			if (netlink_get_setelems(ctx, &set->handle,
+						 &cmd->location, set) < 0)
+				return -1;
+
+			set_print(set);
+		}
+	}
+	return 0;
+}
+
 static int do_list_ruleset(struct netlink_ctx *ctx, struct cmd *cmd)
 {
 	unsigned int family = cmd->handle.family;
@@ -982,10 +1017,25 @@ static int do_list_tables(struct netlink_ctx *ctx, struct cmd *cmd)
 	return 0;
 }
 
+static int do_list_set(struct netlink_ctx *ctx, struct cmd *cmd,
+		       struct table *table)
+{
+	struct set *set;
+
+	set = set_lookup(table, cmd->handle.set);
+	if (set == NULL)
+		return -1;
+
+	if (netlink_get_setelems(ctx, &cmd->handle, &cmd->location, set) < 0)
+		return -1;
+
+	set_print(set);
+	return 0;
+}
+
 static int do_command_list(struct netlink_ctx *ctx, struct cmd *cmd)
 {
 	struct table *table = NULL;
-	struct set *set;
 
 	if (cmd->handle.table != NULL)
 		table = table_lookup(&cmd->handle);
@@ -998,28 +1048,9 @@ static int do_command_list(struct netlink_ctx *ctx, struct cmd *cmd)
 	case CMD_OBJ_CHAIN:
 		return do_list_table(ctx, cmd, table);
 	case CMD_OBJ_SETS:
-		if (netlink_list_sets(ctx, &cmd->handle, &cmd->location) < 0)
-			return -1;
-
-		list_for_each_entry(set, &ctx->list, list){
-			if (netlink_get_setelems(ctx, &set->handle,
-						 &cmd->location, set) < 0) {
-				return -1;
-			}
-			set_print(set);
-		}
-		return 0;
+		return do_list_sets_global(ctx, cmd);
 	case CMD_OBJ_SET:
-		if (netlink_get_set(ctx, &cmd->handle, &cmd->location) < 0)
-			goto err;
-		list_for_each_entry(set, &ctx->list, list) {
-			if (netlink_get_setelems(ctx, &cmd->handle,
-						 &cmd->location, set) < 0) {
-				goto err;
-			}
-			set_print(set);
-		}
-		return 0;
+		return do_list_set(ctx, cmd, table);
 	case CMD_OBJ_RULESET:
 		return do_list_ruleset(ctx, cmd);
 	default:
@@ -1027,9 +1058,6 @@ static int do_command_list(struct netlink_ctx *ctx, struct cmd *cmd)
 	}
 
 	return 0;
-err:
-	table_cleanup(table);
-	return -1;
 }
 
 static int do_command_flush(struct netlink_ctx *ctx, struct cmd *cmd)
@@ -1072,10 +1100,7 @@ static int do_command_rename(struct netlink_ctx *ctx, struct cmd *cmd)
 static int do_command_monitor(struct netlink_ctx *ctx, struct cmd *cmd)
 {
 	struct table *t;
-	struct set *s, *ns;
-	struct netlink_ctx set_ctx;
-	LIST_HEAD(msgs);
-	struct handle set_handle;
+	struct set *s;
 	struct netlink_mon_handler monhandler;
 
 	/* cache only needed if monitoring:
@@ -1090,24 +1115,9 @@ static int do_command_monitor(struct netlink_ctx *ctx, struct cmd *cmd)
 		monhandler.cache_needed = false;
 
 	if (monhandler.cache_needed) {
-		memset(&set_ctx, 0, sizeof(set_ctx));
-		init_list_head(&msgs);
-		set_ctx.msgs = &msgs;
-
 		list_for_each_entry(t, &table_list, list) {
-			set_handle.family = t->handle.family;
-			set_handle.table = t->handle.table;
-
-			init_list_head(&set_ctx.list);
-
-			if (netlink_list_sets(&set_ctx, &set_handle,
-					      &cmd->location) < 0)
-				return -1;
-
-			list_for_each_entry_safe(s, ns, &set_ctx.list, list) {
+			list_for_each_entry(s, &t->sets, list)
 				s->init = set_expr_alloc(&cmd->location);
-				set_add_hash(s, t);
-			}
 		}
 	}
 
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 06/14] src: add set declaration to cache
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (4 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 05/14] src: use cache infrastructure for set objects Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 07/14] src: early allocation of the set ID Pablo Neira Ayuso
                   ` (7 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

This patch adds set objects to the cache if they don't exist in the kernel, so
they can be referenced from this batch. This occurs from the evaluation step.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/evaluate.c |    9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/src/evaluate.c b/src/evaluate.c
index 31aaecc..239dde7 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -1736,8 +1736,17 @@ static int setelem_evaluate(struct eval_ctx *ctx, struct expr **expr)
 
 static int set_evaluate(struct eval_ctx *ctx, struct set *set)
 {
+	struct table *table;
 	const char *type;
 
+	table = table_lookup(&ctx->cmd->handle);
+	if (table == NULL)
+		return cmd_error(ctx, "Could not process rule: Table '%s' does not exist",
+				 ctx->cmd->handle.table);
+
+	if (set_lookup(table, set->handle.set) == NULL)
+		set_add_hash(set_get(set), table);
+
 	type = set->flags & SET_F_MAP ? "map" : "set";
 
 	if (set->keytype == NULL)
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 07/14] src: early allocation of the set ID
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (5 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 06/14] src: add set declaration to cache Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 08/14] rule: add chain reference counter Pablo Neira Ayuso
                   ` (6 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

By when the set is created, so element in the batch use this set ID as
reference.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/netlink.c |    4 ----
 src/rule.c    |    4 ++++
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/src/netlink.c b/src/netlink.c
index 8ede8e6..0fb7b63 100644
--- a/src/netlink.c
+++ b/src/netlink.c
@@ -1121,9 +1121,6 @@ static int netlink_add_set_compat(struct netlink_ctx *ctx,
 	return err;
 }
 
-/* internal ID to uniquely identify a set in the batch */
-static uint32_t set_id;
-
 static int netlink_add_set_batch(struct netlink_ctx *ctx,
 				 const struct handle *h, struct set *set)
 {
@@ -1147,7 +1144,6 @@ static int netlink_add_set_batch(struct netlink_ctx *ctx,
 	if (set->gc_int)
 		nft_set_attr_set_u32(nls, NFT_SET_ATTR_GC_INTERVAL, set->gc_int);
 
-	set->handle.set_id = ++set_id;
 	nft_set_attr_set_u32(nls, NFT_SET_ATTR_ID, set->handle.set_id);
 
 	if (!(set->flags & (SET_F_CONSTANT))) {
diff --git a/src/rule.c b/src/rule.c
index 32c4a15..7f2a27f 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -139,12 +139,16 @@ void cache_release(void)
 	cache_initialized = false;
 }
 
+/* internal ID to uniquely identify a set in the batch */
+static uint32_t set_id;
+
 struct set *set_alloc(const struct location *loc)
 {
 	struct set *set;
 
 	set = xzalloc(sizeof(*set));
 	set->refcnt = 1;
+	set->handle.set_id = ++set_id;
 	if (loc != NULL)
 		set->location = *loc;
 	return set;
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 08/14] rule: add chain reference counter
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (6 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 07/14] src: early allocation of the set ID Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 09/14] src: use cache infrastructure for chain objects Pablo Neira Ayuso
                   ` (5 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

When adding declared chains to the cache, we may hold more than one single
reference from struct cmd and the cache.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 include/rule.h |    3 +++
 src/rule.c     |    9 +++++++++
 2 files changed, 12 insertions(+)

diff --git a/include/rule.h b/include/rule.h
index cc4e5c2..7e8be7c 100644
--- a/include/rule.h
+++ b/include/rule.h
@@ -110,6 +110,7 @@ enum chain_flags {
  * @list:	list node in table list
  * @handle:	chain handle
  * @location:	location the chain was defined at
+ * @refcnt:	reference counter
  * @flags:	chain flags
  * @hookstr:	unified and human readable hook name (base chains)
  * @hooknum:	hook number (base chains)
@@ -122,6 +123,7 @@ struct chain {
 	struct list_head	list;
 	struct handle		handle;
 	struct location		location;
+	unsigned int		refcnt;
 	uint32_t		flags;
 	const char		*hookstr;
 	unsigned int		hooknum;
@@ -135,6 +137,7 @@ struct chain {
 extern const char *chain_type_name_lookup(const char *name);
 extern const char *chain_hookname_lookup(const char *name);
 extern struct chain *chain_alloc(const char *name);
+extern struct chain *chain_get(struct chain *chain);
 extern void chain_free(struct chain *chain);
 extern void chain_add_hash(struct chain *chain, struct table *table);
 extern struct chain *chain_lookup(const struct table *table,
diff --git a/src/rule.c b/src/rule.c
index 7f2a27f..8461461 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -443,6 +443,7 @@ struct chain *chain_alloc(const char *name)
 	struct chain *chain;
 
 	chain = xzalloc(sizeof(*chain));
+	chain->refcnt = 1;
 	init_list_head(&chain->rules);
 	init_list_head(&chain->scope.symbols);
 	if (name != NULL)
@@ -452,10 +453,18 @@ struct chain *chain_alloc(const char *name)
 	return chain;
 }
 
+struct chain *chain_get(struct chain *chain)
+{
+	chain->refcnt++;
+	return chain;
+}
+
 void chain_free(struct chain *chain)
 {
 	struct rule *rule, *next;
 
+	if (--chain->refcnt > 0)
+		return;
 	list_for_each_entry_safe(rule, next, &chain->rules, list)
 		rule_free(rule);
 	handle_free(&chain->handle);
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 09/14] src: use cache infrastructure for chain objects
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (7 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 08/14] rule: add chain reference counter Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 10/14] evaluate: add cmd_evaluate_rename() Pablo Neira Ayuso
                   ` (4 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

The chain list is obtained if the user requests a listing.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/rule.c |   61 ++++++++++++++++++++++--------------------------------------
 1 file changed, 22 insertions(+), 39 deletions(-)

diff --git a/src/rule.c b/src/rule.c
index 8461461..b7c834f 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -67,7 +67,7 @@ static int cache_init_tables(struct netlink_ctx *ctx, struct handle *h)
 	return 0;
 }
 
-static int cache_init_objects(struct netlink_ctx *ctx)
+static int cache_init_objects(struct netlink_ctx *ctx, enum cmd_ops cmd)
 {
 	struct table *table;
 	int ret;
@@ -79,6 +79,24 @@ static int cache_init_objects(struct netlink_ctx *ctx)
 
 		if (ret < 0)
 			return -1;
+
+		/* Skip caching other objects to speed up things: We only need
+		 * a full cache when listing the existing ruleset or renaming
+		 * a chain.
+		 */
+		switch (cmd) {
+		case CMD_LIST:
+		case CMD_RENAME:
+			break;
+		default:
+			continue;
+		}
+
+		ret = netlink_list_chains(ctx, &table->handle,
+					  &internal_location);
+		if (ret < 0)
+			return -1;
+		list_splice_tail_init(&ctx->list, &table->chains);
 	}
 	return 0;
 }
@@ -98,7 +116,7 @@ static int cache_init(enum cmd_ops cmd, struct list_head *msgs)
 	ret = cache_init_tables(&ctx, &handle);
 	if (ret < 0)
 		return ret;
-	ret = cache_init_objects(&ctx);
+	ret = cache_init_objects(&ctx, cmd);
 	if (ret < 0)
 		return ret;
 
@@ -929,22 +947,6 @@ static int do_command_export(struct netlink_ctx *ctx, struct cmd *cmd)
 	return 0;
 }
 
-static void table_cleanup(struct table *table)
-{
-	struct chain *chain, *nchain;
-	struct set *set, *nset;
-
-	list_for_each_entry_safe(chain, nchain, &table->chains, list) {
-		list_del(&chain->list);
-		chain_free(chain);
-	}
-
-	list_for_each_entry_safe(set, nset, &table->sets, list) {
-		list_del(&set->list);
-		set_free(set);
-	}
-}
-
 static int do_list_table(struct netlink_ctx *ctx, struct cmd *cmd,
 			 struct table *table)
 {
@@ -952,30 +954,16 @@ static int do_list_table(struct netlink_ctx *ctx, struct cmd *cmd,
 	struct chain *chain;
 
 	if (do_list_sets(ctx, &cmd->location, table) < 0)
-		goto err;
-	if (netlink_list_chains(ctx, &cmd->handle, &cmd->location) < 0)
-		goto err;
-	list_splice_tail_init(&ctx->list, &table->chains);
+		return -1;
 	if (netlink_list_table(ctx, &cmd->handle, &cmd->location) < 0)
-		goto err;
+		return -1;
 
 	list_for_each_entry_safe(rule, nrule, &ctx->list, list) {
-		table = table_lookup(&rule->handle);
 		chain = chain_lookup(table, &rule->handle);
-		if (chain == NULL) {
-			chain = chain_alloc(rule->handle.chain);
-			chain_add_hash(chain, table);
-		}
-
 		list_move_tail(&rule->list, &chain->rules);
 	}
-
 	table_print(table);
-	table_cleanup(table);
 	return 0;
-err:
-	table_cleanup(table);
-	return -1;
 }
 
 static int do_list_sets_global(struct netlink_ctx *ctx, struct cmd *cmd)
@@ -1092,14 +1080,9 @@ static int do_command_rename(struct netlink_ctx *ctx, struct cmd *cmd)
 {
 	struct table *table = table_lookup(&cmd->handle);
 	struct chain *chain;
-	int err;
 
 	switch (cmd->obj) {
 	case CMD_OBJ_CHAIN:
-		err = netlink_get_chain(ctx, &cmd->handle, &cmd->location);
-		if (err < 0)
-			return err;
-		list_splice_tail_init(&ctx->list, &table->chains);
 		chain = chain_lookup(table, &cmd->handle);
 
 		return netlink_rename_chain(ctx, &chain->handle, &cmd->location,
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 10/14] evaluate: add cmd_evaluate_rename()
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (8 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 09/14] src: use cache infrastructure for chain objects Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 11/14] src: add chain declarations to cache Pablo Neira Ayuso
                   ` (3 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

Make sure the table that we want to rename already exist. This is required by
the follow up patch that that adds chains to the cache.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/evaluate.c |   20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/src/evaluate.c b/src/evaluate.c
index 239dde7..0ae07b6 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -1958,6 +1958,24 @@ static int cmd_evaluate_list(struct eval_ctx *ctx, struct cmd *cmd)
 	}
 }
 
+static int cmd_evaluate_rename(struct eval_ctx *ctx, struct cmd *cmd)
+{
+	struct table *table;
+
+	switch (cmd->obj) {
+	case CMD_OBJ_CHAIN:
+		table = table_lookup(&ctx->cmd->handle);
+		if (table == NULL) {
+			return cmd_error(ctx, "Could not process rule: Table '%s' does not exist",
+					 ctx->cmd->handle.table);
+		}
+		break;
+	default:
+		BUG("invalid command object type %u\n", cmd->obj);
+	}
+	return 0;
+}
+
 enum {
 	CMD_MONITOR_EVENT_ANY,
 	CMD_MONITOR_EVENT_NEW,
@@ -2051,7 +2069,9 @@ int cmd_evaluate(struct eval_ctx *ctx, struct cmd *cmd)
 	case CMD_LIST:
 		return cmd_evaluate_list(ctx, cmd);
 	case CMD_FLUSH:
+		return 0;
 	case CMD_RENAME:
+		return cmd_evaluate_rename(ctx, cmd);
 	case CMD_EXPORT:
 	case CMD_DESCRIBE:
 		return 0;
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 11/14] src: add chain declarations to cache
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (9 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 10/14] evaluate: add cmd_evaluate_rename() Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 12/14] src: use cache infrastructure for rule objects Pablo Neira Ayuso
                   ` (2 subsequent siblings)
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/evaluate.c |   20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/src/evaluate.c b/src/evaluate.c
index 0ae07b6..d2bab97 100644
--- a/src/evaluate.c
+++ b/src/evaluate.c
@@ -1850,8 +1850,26 @@ static uint32_t str2hooknum(uint32_t family, const char *hook)
 
 static int chain_evaluate(struct eval_ctx *ctx, struct chain *chain)
 {
+	struct table *table;
 	struct rule *rule;
 
+	table = table_lookup(&ctx->cmd->handle);
+	if (table == NULL)
+		return cmd_error(ctx, "Table '%s' does not exist",
+				 ctx->cmd->handle.table);
+
+	if (chain == NULL) {
+		if (chain_lookup(table, &ctx->cmd->handle) == NULL) {
+			chain = chain_alloc(NULL);
+			handle_merge(&chain->handle, &ctx->cmd->handle);
+			chain_add_hash(chain, table);
+		}
+		return 0;
+	} else {
+		if (chain_lookup(table, &chain->handle) == NULL)
+			chain_add_hash(chain_get(chain), table);
+	}
+
 	if (chain->flags & CHAIN_F_BASECHAIN) {
 		chain->hooknum = str2hooknum(chain->handle.family,
 					     chain->hookstr);
@@ -1913,8 +1931,6 @@ static int cmd_evaluate_add(struct eval_ctx *ctx, struct cmd *cmd)
 		handle_merge(&cmd->rule->handle, &cmd->handle);
 		return rule_evaluate(ctx, cmd->rule);
 	case CMD_OBJ_CHAIN:
-		if (cmd->data == NULL)
-			return 0;
 		return chain_evaluate(ctx, cmd->chain);
 	case CMD_OBJ_TABLE:
 		return table_evaluate(ctx, cmd->table);
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 12/14] src: use cache infrastructure for rule objects
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (10 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 11/14] src: add chain declarations to cache Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 13/14] src: use cache infrastructure for set element objects Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 14/14] src: get rid of EINTR handling for nft_netlink() Pablo Neira Ayuso
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

Populate the cache iff the user requests a ruleset listing.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/rule.c |   22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/src/rule.c b/src/rule.c
index b7c834f..cd18ca8 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -70,6 +70,8 @@ static int cache_init_tables(struct netlink_ctx *ctx, struct handle *h)
 static int cache_init_objects(struct netlink_ctx *ctx, enum cmd_ops cmd)
 {
 	struct table *table;
+	struct chain *chain;
+	struct rule *rule, *nrule;
 	int ret;
 
 	list_for_each_entry(table, &table_list, list) {
@@ -97,6 +99,16 @@ static int cache_init_objects(struct netlink_ctx *ctx, enum cmd_ops cmd)
 		if (ret < 0)
 			return -1;
 		list_splice_tail_init(&ctx->list, &table->chains);
+
+		ret = netlink_list_table(ctx, &table->handle,
+					 &internal_location);
+		list_for_each_entry_safe(rule, nrule, &ctx->list, list) {
+			chain = chain_lookup(table, &rule->handle);
+			list_move_tail(&rule->list, &chain->rules);
+		}
+
+		if (ret < 0)
+			return -1;
 	}
 	return 0;
 }
@@ -950,18 +962,8 @@ static int do_command_export(struct netlink_ctx *ctx, struct cmd *cmd)
 static int do_list_table(struct netlink_ctx *ctx, struct cmd *cmd,
 			 struct table *table)
 {
-	struct rule *rule, *nrule;
-	struct chain *chain;
-
 	if (do_list_sets(ctx, &cmd->location, table) < 0)
 		return -1;
-	if (netlink_list_table(ctx, &cmd->handle, &cmd->location) < 0)
-		return -1;
-
-	list_for_each_entry_safe(rule, nrule, &ctx->list, list) {
-		chain = chain_lookup(table, &rule->handle);
-		list_move_tail(&rule->list, &chain->rules);
-	}
 	table_print(table);
 	return 0;
 }
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 13/14] src: use cache infrastructure for set element objects
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (11 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 12/14] src: use cache infrastructure for rule objects Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  2015-08-10 11:50 ` [PATCH nft v5 14/14] src: get rid of EINTR handling for nft_netlink() Pablo Neira Ayuso
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

Populate the cache iff the user requests a ruleset listing.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/rule.c |   36 +++++++++++-------------------------
 1 file changed, 11 insertions(+), 25 deletions(-)

diff --git a/src/rule.c b/src/rule.c
index cd18ca8..310ff95 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -72,6 +72,7 @@ static int cache_init_objects(struct netlink_ctx *ctx, enum cmd_ops cmd)
 	struct table *table;
 	struct chain *chain;
 	struct rule *rule, *nrule;
+	struct set *set;
 	int ret;
 
 	list_for_each_entry(table, &table_list, list) {
@@ -94,6 +95,13 @@ static int cache_init_objects(struct netlink_ctx *ctx, enum cmd_ops cmd)
 			continue;
 		}
 
+		list_for_each_entry(set, &table->sets, list) {
+			ret = netlink_get_setelems(ctx, &set->handle,
+						   &internal_location, set);
+			if (ret < 0)
+				return -1;
+		}
+
 		ret = netlink_list_chains(ctx, &table->handle,
 					  &internal_location);
 		if (ret < 0)
@@ -932,18 +940,6 @@ static int do_command_delete(struct netlink_ctx *ctx, struct cmd *cmd)
 	}
 }
 
-static int do_list_sets(struct netlink_ctx *ctx, const struct location *loc,
-			struct table *table)
-{
-	struct set *set;
-
-	list_for_each_entry(set, &table->sets, list) {
-		if (netlink_get_setelems(ctx, &set->handle, loc, set) < 0)
-			return -1;
-	}
-	return 0;
-}
-
 static int do_command_export(struct netlink_ctx *ctx, struct cmd *cmd)
 {
 	struct nft_ruleset *rs = netlink_dump_ruleset(ctx, &cmd->handle,
@@ -962,25 +958,18 @@ static int do_command_export(struct netlink_ctx *ctx, struct cmd *cmd)
 static int do_list_table(struct netlink_ctx *ctx, struct cmd *cmd,
 			 struct table *table)
 {
-	if (do_list_sets(ctx, &cmd->location, table) < 0)
-		return -1;
 	table_print(table);
 	return 0;
 }
 
-static int do_list_sets_global(struct netlink_ctx *ctx, struct cmd *cmd)
+static int do_list_sets(struct netlink_ctx *ctx, struct cmd *cmd)
 {
 	struct table *table;
 	struct set *set;
 
 	list_for_each_entry(table, &table_list, list) {
-		list_for_each_entry(set, &table->sets, list) {
-			if (netlink_get_setelems(ctx, &set->handle,
-						 &cmd->location, set) < 0)
-				return -1;
-
+		list_for_each_entry(set, &table->sets, list)
 			set_print(set);
-		}
 	}
 	return 0;
 }
@@ -1029,9 +1018,6 @@ static int do_list_set(struct netlink_ctx *ctx, struct cmd *cmd,
 	if (set == NULL)
 		return -1;
 
-	if (netlink_get_setelems(ctx, &cmd->handle, &cmd->location, set) < 0)
-		return -1;
-
 	set_print(set);
 	return 0;
 }
@@ -1051,7 +1037,7 @@ static int do_command_list(struct netlink_ctx *ctx, struct cmd *cmd)
 	case CMD_OBJ_CHAIN:
 		return do_list_table(ctx, cmd, table);
 	case CMD_OBJ_SETS:
-		return do_list_sets_global(ctx, cmd);
+		return do_list_sets(ctx, cmd);
 	case CMD_OBJ_SET:
 		return do_list_set(ctx, cmd, table);
 	case CMD_OBJ_RULESET:
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

* [PATCH nft v5 14/14] src: get rid of EINTR handling for nft_netlink()
  2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
                   ` (12 preceding siblings ...)
  2015-08-10 11:50 ` [PATCH nft v5 13/14] src: use cache infrastructure for set element objects Pablo Neira Ayuso
@ 2015-08-10 11:50 ` Pablo Neira Ayuso
  13 siblings, 0 replies; 15+ messages in thread
From: Pablo Neira Ayuso @ 2015-08-10 11:50 UTC (permalink / raw)
  To: netfilter-devel; +Cc: kaber

The only remaining caller that needs this is netlink_dump_ruleset(), that is
used to export the ruleset using markup representation. We can remove it and
handle this from do_command_export() now that we have a centralized point to
build up the object cache.

Signed-off-by: Pablo Neira Ayuso <pablo@netfilter.org>
---
 src/main.c |    5 -----
 src/rule.c |   10 ++++++----
 2 files changed, 6 insertions(+), 9 deletions(-)

diff --git a/src/main.c b/src/main.c
index d6c9ccc..7bbcfc4 100644
--- a/src/main.c
+++ b/src/main.c
@@ -233,12 +233,7 @@ int nft_run(void *scanner, struct parser_state *state, struct list_head *msgs)
 		ret = -1;
 		goto err1;
 	}
-retry:
 	ret = nft_netlink(state, msgs);
-	if (ret < 0 && errno == EINTR) {
-		netlink_restart();
-		goto retry;
-	}
 err1:
 	list_for_each_entry_safe(cmd, next, &state->cmds, list) {
 		list_del(&cmd->list);
diff --git a/src/rule.c b/src/rule.c
index 310ff95..d369596 100644
--- a/src/rule.c
+++ b/src/rule.c
@@ -942,11 +942,13 @@ static int do_command_delete(struct netlink_ctx *ctx, struct cmd *cmd)
 
 static int do_command_export(struct netlink_ctx *ctx, struct cmd *cmd)
 {
-	struct nft_ruleset *rs = netlink_dump_ruleset(ctx, &cmd->handle,
-						      &cmd->location);
+	struct nft_ruleset *rs;
 
-	if (rs == NULL)
-		return -1;
+	do {
+		rs = netlink_dump_ruleset(ctx, &cmd->handle, &cmd->location);
+		if (rs == NULL && errno != EINTR)
+			return -1;
+	} while (rs == NULL);
 
 	nft_ruleset_fprintf(stdout, rs, cmd->export->format, 0);
 	fprintf(stdout, "\n");
-- 
1.7.10.4


^ permalink raw reply related	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2015-08-10 11:45 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-08-10 11:50 [PATCH nft v5 00/14] cache consolidation Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 01/14] src: add cache infrastructure and use it for table objects Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 02/14] src: add cmd_evaluate_list() Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 03/14] rule: add reference counter to the table object Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 04/14] src: add table declaration to cache Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 05/14] src: use cache infrastructure for set objects Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 06/14] src: add set declaration to cache Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 07/14] src: early allocation of the set ID Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 08/14] rule: add chain reference counter Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 09/14] src: use cache infrastructure for chain objects Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 10/14] evaluate: add cmd_evaluate_rename() Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 11/14] src: add chain declarations to cache Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 12/14] src: use cache infrastructure for rule objects Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 13/14] src: use cache infrastructure for set element objects Pablo Neira Ayuso
2015-08-10 11:50 ` [PATCH nft v5 14/14] src: get rid of EINTR handling for nft_netlink() Pablo Neira Ayuso

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).