[PATCH v3 0/29] nfacct changes and additions

[PATCH v3 0/29] nfacct changes and additions

[Michael Zintakis]

The following patch set fixes bugs and introduces a variety of new features
to the 3 nfacct components: kernel, libnetfilter_acct and nfacct executable.

All of the patches need to be applied in the order specified in this patch
set (1-29) as they are interdependent. The full list of bugfixes and
features added for each component are:


Michael Zintakis (29):

* kernel:
  bugfix: pkts/bytes need to be specified simultaneously
  bugfix: restore pkts/bytes counters in NLM_F_REPLACE
  add permanent byte/packet format capability to nfacct
  add byte threshold capability to nfacct
  add packets and bytes mark capability to nfacct

 include/uapi/linux/netfilter/nfnetlink_acct.h |  16 +++
 net/netfilter/nfnetlink_acct.c                | 138 +++++++++++++++++++++++++-
 2 files changed, 150 insertions(+), 4 deletions(-)


* libnetfilter_acct:
  bugfix: correct xml name parsing
  bugfix: correct (plain) name parsing
  add *_SAVE template allowing save/restore
  add *_BONLY template to show bytes-only
  add variable width and on-the-fly formatting
  add *permanent* number formatting support
  add byte threshold capability support
  add *_EXTENDED template support
  add packets/bytes mark capability support
  add *_MONLY template support

 configure.ac                                  |    2 +-
 include/libnetfilter_acct/libnetfilter_acct.h |  100 +++
 include/linux/netfilter/nfnetlink_acct.h      |   16 +
 src/libnetfilter_acct.c                       | 1055 ++++++++++++++++++++++++-
 src/libnetfilter_acct.map                     |   13 +
 5 files changed, 1158 insertions(+), 28 deletions(-)


* nfacct:
  bugfix: prevent 0-sized parameter being accepted
  bugfix: prevent 0-sized nfacct name being accepted
  code-refactoring changes to the "command menu"
  add 2 new options: "replace" and "flush"
  add variable width and on-the-fly number formatting
  add new "save" and correct existing "restore" commands
  add sort option to the "list" command
  add "show bytes" option to "list" and "get" commands
  add permanent number formatting to nfacct objects
  add byte threshold capabilities to nfacct objects
  add "show extended" option to "list" and "get" commands
  add setmark and clrmark to "get" and "list" commands
  add "show marks" option to "list" and "get" commands
  change man page to describe all new features

 include/linux/netfilter/nfnetlink_acct.h |   17 +
 nfacct.8                                 |  551 +++++++++++++++-
 src/Makefile.am                          |    2 +-
 src/nfacct.c                             | 1026 ++++++++++++++++++++++++------
 src/nfacct_list.c                        |  149 +++++
 src/nfacct_list.h                        |   95 +++
 src/nfacct_utils.c                       |  333 ++++++++++
 src/nfacct_utils.h                       |   33 +
 8 files changed, 1983 insertions(+), 223 deletions(-)
 create mode 100644 src/nfacct_list.c
 create mode 100644 src/nfacct_list.h
 create mode 100644 src/nfacct_utils.c
 create mode 100644 src/nfacct_utils.h

-- 
1.8.3.1

[PATCH v3 kernel 1/29] bugfix: pkts/bytes need to be specified simultaneously

[Michael Zintakis]

* in nfnetlink_acct.c::nfnl_acct_new enforce a check ensuring that
packet and byte values are specified simultaneously - return -EINVAL if
that is not the case.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 net/netfilter/nfnetlink_acct.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/net/netfilter/nfnetlink_acct.c b/net/netfilter/nfnetlink_acct.c
index c7b6d46..526abd7 100644
--- a/net/netfilter/nfnetlink_acct.c
+++ b/net/netfilter/nfnetlink_acct.c
@@ -45,7 +45,9 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 	struct nf_acct *nfacct, *matching = NULL;
 	char *acct_name;
 
-	if (!tb[NFACCT_NAME])
+	if (!tb[NFACCT_NAME] ||
+	    (tb[NFACCT_BYTES] && !tb[NFACCT_PKTS]) ||
+	    (!tb[NFACCT_BYTES] && tb[NFACCT_PKTS]))
 		return -EINVAL;
 
 	acct_name = nla_data(tb[NFACCT_NAME]);
-- 
1.8.3.1

Re: [PATCH v3 kernel 1/29] bugfix: pkts/bytes need to be specified simultaneously

[Florian Westphal]

Michael Zintakis <michael.zintakis@googlemail.com> wrote:
> * in nfnetlink_acct.c::nfnl_acct_new enforce a check ensuring that
> packet and byte values are specified simultaneously - return -EINVAL if
> that is not the case.

Ok, I understand why it appears to be un-intuitive to have
an accounting object that shows '0 bytes in 100000 packets'.

But still, whats the point?

I could also set '1 byte in 100000 packets', no?

[ so my point is, why bother with checking this? ]

Re: [PATCH v3 kernel 1/29] bugfix: pkts/bytes need to be specified simultaneously

[Michael Zintakis]

Florian Westphal wrote:
> Michael Zintakis <michael.zintakis@googlemail.com> wrote:
>> * in nfnetlink_acct.c::nfnl_acct_new enforce a check ensuring that
>> packet and byte values are specified simultaneously - return -EINVAL if
>> that is not the case.
> 
> Ok, I understand why it appears to be un-intuitive to have
> an accounting object that shows '0 bytes in 100000 packets'.
> 
> But still, whats the point?
The fact that it isn't un-intuitive is not the reason for creating that patch.

> I could also set '1 byte in 100000 packets', no?
> 
> [ so my point is, why bother with checking this? ]
Indeed you can, but that would be your own (conscious) decision and you will be responsible for it (which is what counts here).

When either packet or byte counters aren't specified that isn't necessarily the case (you may forgot to set them up for example).

Let me illustrate my point with a little real-life example: suppose you have an alarm clock which lags behind quite often and you need to set it up to wake you in the morning for an important meeting.

Now, if you deliberately choose to put the clock backwards (the equivalent of specifying "0 bytes in 100000 packets") you will be late for that meeting and, as you did not set your clock properly, you are responsible for the flak you are going to get when you are late, while if you leave the clock to drift and be late, you may have an excuse (provided anyone believes that excuse, but that is not the point I am trying to make).

So, nothing is stopping you from specifying "0 bytes in 100000 packets" if you want to, but that would ultimately be your own conscious decision (and responsibility!), as oppose to "forgetting" to specify either bytes or packets, which may or may not be entirely your fault.

[PATCH v3 kernel 2/29] bugfix: restore pkts/bytes counters in NLM_F_REPLACE

[Michael Zintakis]

* allow packet and byte counters to be replaced in NLM_F_REPLACE mode,
allowing 'restore' to function properly

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 net/netfilter/nfnetlink_acct.c | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/net/netfilter/nfnetlink_acct.c b/net/netfilter/nfnetlink_acct.c
index 526abd7..c14046c 100644
--- a/net/netfilter/nfnetlink_acct.c
+++ b/net/netfilter/nfnetlink_acct.c
@@ -67,9 +67,16 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 
 	if (matching) {
 		if (nlh->nlmsg_flags & NLM_F_REPLACE) {
-			/* reset counters if you request a replacement. */
-			atomic64_set(&matching->pkts, 0);
-			atomic64_set(&matching->bytes, 0);
+			/* (re)set counters if you request a replacement */
+			if (tb[NFACCT_PKTS] && tb[NFACCT_BYTES]) {
+				atomic64_set(&matching->pkts,
+				be64_to_cpu(nla_get_be64(tb[NFACCT_PKTS])));
+				atomic64_set(&matching->bytes,
+				be64_to_cpu(nla_get_be64(tb[NFACCT_BYTES])));
+			} else {
+				atomic64_set(&matching->pkts, 0);
+				atomic64_set(&matching->bytes, 0);
+			}
 			return 0;
 		}
 		return -EBUSY;
-- 
1.8.3.1

[PATCH v3 libnetfilter_acct 3/29] bugfix: correct xml name parsing

[Michael Zintakis]

* allow accounting object names to be properly encoded and displayed when xml
output is needed, to fully conform to the xml specification.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/libnetfilter_acct.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 42 insertions(+), 1 deletion(-)

diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index ba89e2d..4d87da3 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -228,6 +228,43 @@ uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type)
 }
 EXPORT_SYMBOL(nfacct_attr_get_u64);
 
+static
+void parse_nfacct_name_xml(char *buf, const char *name)
+{
+	static const char escape_chars[] = "\"'<>&";
+	int length;
+	int n;
+	char e[10];
+	const char *p;
+
+	if (buf == NULL)
+		return;
+
+	buf[0] = '\0';
+	if (name == NULL)
+		return;
+
+	length = strcspn(name, escape_chars);
+	if (length > 0 && name[length] == 0) {
+		/* no escaping required */
+		strncat(buf, name, length);
+	} else {
+		for (p = strpbrk(name, escape_chars); p != NULL;
+		     p = strpbrk(name, escape_chars)) {
+			if (p > name)
+				strncat(buf, name, p - name);
+
+			n = *p;
+			snprintf(e, sizeof(e), "&#%d;", n);
+			strncat(buf, e, strlen(e));
+			name = p + 1;
+		}
+
+		/* strncat the rest */
+		strncat(buf, name, length);
+	}
+}
+
 static int
 nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		      uint16_t flags)
@@ -292,12 +329,16 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 {
 	int ret = 0;
 	unsigned int size = 0, offset = 0;
+	char nfacct_name[NFACCT_NAME_MAX * 6 + 1];
 
+	parse_nfacct_name_xml(nfacct_name,
+				nfacct_attr_get_str(nfacct,
+						    NFACCT_ATTR_NAME));
 	ret = snprintf(buf, rem,
 			"<obj><name>%s</name>"
 			"<pkts>%.20"PRIu64"</pkts>"
 			"<bytes>%.20"PRIu64"</bytes>",
-			nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME),
+			nfacct_name,
 			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
 			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS));
 	BUFFER_SIZE(ret, size, rem, offset);
-- 
1.8.3.1

Re: [PATCH v3 libnetfilter_acct 3/29] bugfix: correct xml name parsing

[Pablo Neira Ayuso]

On Wed, Jul 10, 2013 at 07:25:01PM +0100, Michael Zintakis wrote:
> * allow accounting object names to be properly encoded and displayed when xml
> output is needed, to fully conform to the xml specification.
> 
> Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
> ---
>  src/libnetfilter_acct.c | 43 ++++++++++++++++++++++++++++++++++++++++++-
>  1 file changed, 42 insertions(+), 1 deletion(-)
> 
> diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
> index ba89e2d..4d87da3 100644
> --- a/src/libnetfilter_acct.c
> +++ b/src/libnetfilter_acct.c
> @@ -228,6 +228,43 @@ uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type)
>  }
>  EXPORT_SYMBOL(nfacct_attr_get_u64);
>  
> +static
> +void parse_nfacct_name_xml(char *buf, const char *name)

Please, use the prefix nfacct_* for consistency. My suggestion for
this function name is nfacct_name_to_xml.

> +{
> +	static const char escape_chars[] = "\"'<>&";

You have to escape the apostroph as well.

> +	int length;
> +	int n;
> +	char e[10];
> +	const char *p;
> +
> +	if (buf == NULL)
> +		return;

This checking is superfluous, it always evaluates false.

> +
> +	buf[0] = '\0';

Don't need this initial zeroing.

> +	if (name == NULL)
> +		return;

Same thing above.

> +	length = strcspn(name, escape_chars);
> +	if (length > 0 && name[length] == 0) {
> +		/* no escaping required */
> +		strncat(buf, name, length);
> +	} else {
> +		for (p = strpbrk(name, escape_chars); p != NULL;
> +		     p = strpbrk(name, escape_chars)) {
> +			if (p > name)
> +				strncat(buf, name, p - name);
> +
> +			n = *p;
> +			snprintf(e, sizeof(e), "&#%d;", n);
> +			strncat(buf, e, strlen(e));

Please use snprintf to build the string (instead of strncat) you can
use BUFFER_SIZE to update the buffer offset.

> +			name = p + 1;
> +		}
> +
> +		/* strncat the rest */
> +		strncat(buf, name, length);
> +	}
> +}
> +
>  static int
>  nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
>  		      uint16_t flags)
> @@ -292,12 +329,16 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
>  {
>  	int ret = 0;
>  	unsigned int size = 0, offset = 0;
> +	char nfacct_name[NFACCT_NAME_MAX * 6 + 1];

Why this buffer length?

> +	parse_nfacct_name_xml(nfacct_name,
> +				nfacct_attr_get_str(nfacct,
> +						    NFACCT_ATTR_NAME));
>  	ret = snprintf(buf, rem,
>  			"<obj><name>%s</name>"
>  			"<pkts>%.20"PRIu64"</pkts>"
>  			"<bytes>%.20"PRIu64"</bytes>",
> -			nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME),
> +			nfacct_name,
>  			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
>  			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS));
>  	BUFFER_SIZE(ret, size, rem, offset);
> -- 
> 1.8.3.1
> 

[PATCH v3 libnetfilter_acct 4/29] bugfix: correct (plain) name parsing

[Michael Zintakis]

* allow accounting object names containing space or other "odd" characters
to be properly parsed and displayed.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/libnetfilter_acct/libnetfilter_acct.h |  2 +
 src/libnetfilter_acct.c                       | 54 +++++++++++++++++++++++++--
 src/libnetfilter_acct.map                     |  1 +
 3 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index b00e366..9f66c39 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -28,6 +28,8 @@ const void *nfacct_attr_get(struct nfacct *nfacct, enum nfacct_attr_type type);
 const char *nfacct_attr_get_str(struct nfacct *nfacct, enum nfacct_attr_type type);
 uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type);
 
+void parse_nfacct_name(char *buf, const char *name);
+
 struct nlmsghdr;
 
 struct nlmsghdr *nfacct_nlmsg_build_hdr(char *buf, uint8_t cmd, uint16_t flags, uint32_t seq);
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index 4d87da3..493be3b 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -228,6 +228,52 @@ uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type)
 }
 EXPORT_SYMBOL(nfacct_attr_get_u64);
 
+void
+parse_nfacct_name(char *buf, const char *name)
+{
+	static const char no_quote_chars[] = ",._-0123456789"
+		"abcdefghijklmnopqrstuvwxyz"
+		"ABCDEFGHIJKLMNOPQRSTUVWXYZ";
+	static const char escape_chars[] = "\"\\'";
+	size_t length;
+	const char *p;
+
+	if (buf == NULL)
+		return;
+
+	buf[0] = '\0';
+	if (name == NULL)
+		return;
+
+	length = strspn(name, no_quote_chars);
+
+	if (length > 0 && name[length] == 0) {
+		/* no quoting required */
+		strncat(buf, name, length);
+	} else {
+		/* there is at least one character in the name, which
+		   we have to quote.  Write double quotes around the
+		   name and escape special characters with a backslash */
+		strncat(buf,"\"",1);
+
+		for (p = strpbrk(name, escape_chars); p != NULL;
+		     p = strpbrk(name, escape_chars)) {
+			if (p > name) {
+				strncat(buf,name, p - name);
+			}
+			strncat(buf,"\\",1);
+			strncat(buf,p,1);
+			name = p + 1;
+		}
+
+		/* strcat the rest and finish the double quoted
+		   string */
+		strncat(buf,name,strlen(name));
+		strncat(buf,"\"",1);
+	}
+}
+EXPORT_SYMBOL(parse_nfacct_name);
+
 static
 void parse_nfacct_name_xml(char *buf, const char *name)
 {
@@ -270,16 +316,18 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		      uint16_t flags)
 {
 	int ret;
+	char nfacct_name[NFACCT_NAME_MAX * 2 + 4];
 
+	parse_nfacct_name(nfacct_name,
+			  nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
 	if (flags & NFACCT_SNPRINTF_F_FULL) {
 		ret = snprintf(buf, rem,
 			"{ pkts = %.20"PRIu64", bytes = %.20"PRIu64" } = %s;",
 			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS),
 			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
-			nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
+			nfacct_name);
 	} else {
-		ret = snprintf(buf, rem, "%s\n",
-			nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
+		ret = snprintf(buf, rem, "%s\n", nfacct_name);
 	}
 
 	return ret;
diff --git a/src/libnetfilter_acct.map b/src/libnetfilter_acct.map
index e71a6b3..f12bc8e 100644
--- a/src/libnetfilter_acct.map
+++ b/src/libnetfilter_acct.map
@@ -21,4 +21,5 @@ local: *;
 
 LIBNETFILTER_ACCT_1.1 {
   nfacct_snprintf;
+  parse_nfacct_name;
 } LIBNETFILTER_ACCT_1.0;
-- 
1.8.3.1

Re: [PATCH v3 libnetfilter_acct 4/29] bugfix: correct (plain) name parsing

[Pablo Neira Ayuso]

On Wed, Jul 10, 2013 at 07:25:02PM +0100, Michael Zintakis wrote:
> * allow accounting object names containing space or other "odd" characters
> to be properly parsed and displayed.

The update in the parser should come together with this patch.
The repository cannot remain inconsistent between patches.

> Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
> ---
>  include/libnetfilter_acct/libnetfilter_acct.h |  2 +
>  src/libnetfilter_acct.c                       | 54 +++++++++++++++++++++++++--
>  src/libnetfilter_acct.map                     |  1 +
>  3 files changed, 54 insertions(+), 3 deletions(-)
> 
> diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
> index b00e366..9f66c39 100644
> --- a/include/libnetfilter_acct/libnetfilter_acct.h
> +++ b/include/libnetfilter_acct/libnetfilter_acct.h
> @@ -28,6 +28,8 @@ const void *nfacct_attr_get(struct nfacct *nfacct, enum nfacct_attr_type type);
>  const char *nfacct_attr_get_str(struct nfacct *nfacct, enum nfacct_attr_type type);
>  uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type);
>  
> +void parse_nfacct_name(char *buf, const char *name);

The description does not explain why you need to export this function.

>  struct nlmsghdr;
>  
>  struct nlmsghdr *nfacct_nlmsg_build_hdr(char *buf, uint8_t cmd, uint16_t flags, uint32_t seq);
> diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
> index 4d87da3..493be3b 100644
> --- a/src/libnetfilter_acct.c
> +++ b/src/libnetfilter_acct.c
> @@ -228,6 +228,52 @@ uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type)
>  }
>  EXPORT_SYMBOL(nfacct_attr_get_u64);
>  
> +void
> +parse_nfacct_name(char *buf, const char *name)
> +{
> +	static const char no_quote_chars[] = ",._-0123456789"
> +		"abcdefghijklmnopqrstuvwxyz"
> +		"ABCDEFGHIJKLMNOPQRSTUVWXYZ";
> +	static const char escape_chars[] = "\"\\'";
> +	size_t length;
> +	const char *p;
> +
> +	if (buf == NULL)
> +		return;
> +
> +	buf[0] = '\0';
> +	if (name == NULL)
> +		return;

Like in the previous patch, these checking above are superfluous.

> +	length = strspn(name, no_quote_chars);
> +
> +	if (length > 0 && name[length] == 0) {
> +		/* no quoting required */
> +		strncat(buf, name, length);
> +	} else {
> +		/* there is at least one character in the name, which
> +		   we have to quote.  Write double quotes around the
> +		   name and escape special characters with a backslash */
> +		strncat(buf,"\"",1);
> +
> +		for (p = strpbrk(name, escape_chars); p != NULL;
> +		     p = strpbrk(name, escape_chars)) {
> +			if (p > name) {
> +				strncat(buf,name, p - name);
> +			}
> +			strncat(buf,"\\",1);
> +			strncat(buf,p,1);
> +			name = p + 1;
> +		}
> +
> +		/* strcat the rest and finish the double quoted
> +		   string */
> +		strncat(buf,name,strlen(name));
> +		strncat(buf,"\"",1);
> +	}
> +}
> +EXPORT_SYMBOL(parse_nfacct_name);
> +
>  static
>  void parse_nfacct_name_xml(char *buf, const char *name)
>  {
> @@ -270,16 +316,18 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
>  		      uint16_t flags)
>  {
>  	int ret;
> +	char nfacct_name[NFACCT_NAME_MAX * 2 + 4];

Why this buffer length?

> +	parse_nfacct_name(nfacct_name,
> +			  nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
>  	if (flags & NFACCT_SNPRINTF_F_FULL) {
>  		ret = snprintf(buf, rem,
>  			"{ pkts = %.20"PRIu64", bytes = %.20"PRIu64" } = %s;",
>  			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS),
>  			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
> -			nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
> +			nfacct_name);
>  	} else {
> -		ret = snprintf(buf, rem, "%s\n",
> -			nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
> +		ret = snprintf(buf, rem, "%s\n", nfacct_name);
>  	}
>  
>  	return ret;
> diff --git a/src/libnetfilter_acct.map b/src/libnetfilter_acct.map
> index e71a6b3..f12bc8e 100644
> --- a/src/libnetfilter_acct.map
> +++ b/src/libnetfilter_acct.map
> @@ -21,4 +21,5 @@ local: *;
>  
>  LIBNETFILTER_ACCT_1.1 {
>    nfacct_snprintf;
> +  parse_nfacct_name;

This is wrong. If you want to export a new symbol, you have add a new
label, but I don't think that exporting this is what you want.

>  } LIBNETFILTER_ACCT_1.0;
> -- 
> 1.8.3.1
> 

[PATCH v3 nfacct 5/29] bugfix: prevent 0-sized parameter being accepted

[Michael Zintakis]

* add nfacct_matches function to prevent zero-sized string being accepted
as a command line parameter;

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/nfacct.c | 39 +++++++++++++++++++++++++++------------
 1 file changed, 27 insertions(+), 12 deletions(-)

diff --git a/src/nfacct.c b/src/nfacct.c
index 2ef93c3..1324da7 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -59,6 +59,21 @@ static void nfacct_perror(const char *msg)
 	}
 }
 
+/* Matches two strings, including partial matches */
+static int nfacct_matches(const char *cmd, const char *pattern)
+{
+	size_t len;
+
+	if (cmd == NULL || pattern == NULL)
+		return 0;
+
+	len = strlen(cmd);
+	if (len == 0 || len > strlen(pattern))
+		return 0;
+
+	return (strncmp(cmd, pattern, len) == 0);
+}
+
 int main(int argc, char *argv[])
 {
 	int cmd = NFACCT_CMD_NONE, ret = 0;
@@ -68,21 +83,21 @@ int main(int argc, char *argv[])
 		exit(EXIT_FAILURE);
 	}
 
-	if (strncmp(argv[1], "list", strlen(argv[1])) == 0)
+	if (nfacct_matches(argv[1], "list"))
 		cmd = NFACCT_CMD_LIST;
-	else if (strncmp(argv[1], "add", strlen(argv[1])) == 0)
+	else if (nfacct_matches(argv[1], "add"))
 		cmd = NFACCT_CMD_ADD;
-	else if (strncmp(argv[1], "delete", strlen(argv[1])) == 0)
+	else if (nfacct_matches(argv[1], "delete"))
 		cmd = NFACCT_CMD_DELETE;
-	else if (strncmp(argv[1], "get", strlen(argv[1])) == 0)
+	else if (nfacct_matches(argv[1], "get"))
 		cmd = NFACCT_CMD_GET;
-	else if (strncmp(argv[1], "flush", strlen(argv[1])) == 0)
+	else if (nfacct_matches(argv[1], "flush"))
 		cmd = NFACCT_CMD_FLUSH;
-	else if (strncmp(argv[1], "version", strlen(argv[1])) == 0)
+	else if (nfacct_matches(argv[1], "version"))
 		cmd = NFACCT_CMD_VERSION;
-	else if (strncmp(argv[1], "help", strlen(argv[1])) == 0)
+	else if (nfacct_matches(argv[1], "help"))
 		cmd = NFACCT_CMD_HELP;
-	else if (strncmp(argv[1], "restore", strlen(argv[1])) == 0)
+	else if (nfacct_matches(argv[1], "restore"))
 		cmd = NFACCT_CMD_RESTORE;
 	else {
 		fprintf(stderr, "nfacct v%s: Unknown command: %s\n",
@@ -167,9 +182,9 @@ static int nfacct_cmd_list(int argc, char *argv[])
 	int ret, i;
 
 	for (i=2; i<argc; i++) {
-		if (strncmp(argv[i], "reset", strlen(argv[i])) == 0) {
+		if (nfacct_matches(argv[i], "reset")) {
 			zeroctr = true;
-		} else if (strncmp(argv[i], "xml", strlen(argv[i])) == 0) {
+		} else if (nfacct_matches(argv[i], "xml")) {
 			xml = true;
 		} else {
 			nfacct_perror("unknown argument");
@@ -375,9 +390,9 @@ static int nfacct_cmd_get(int argc, char *argv[])
 		return -1;
 	}
 	for (i=3; i<argc; i++) {
-		if (strncmp(argv[i], "reset", strlen(argv[i])) == 0) {
+		if (nfacct_matches(argv[i], "reset")) {
 			zeroctr = true;
-		} else if (strncmp(argv[i], "xml", strlen(argv[i])) == 0) {
+		} else if (nfacct_matches(argv[i], "xml")) {
 			xml = true;
 		} else {
 			nfacct_perror("unknown argument");
-- 
1.8.3.1

[PATCH v3 nfacct 6/29] bugfix: prevent 0-sized nfacct name being accepted

[Michael Zintakis]

* add additional checks in nfacct_cmd_add, nfacct_cmd_delete and
nfacct_cmd_get functions to prevent zero-sized string being accepted as a
command line parameter for nfacct object name;

* add a separate check for the number of command-line arguments in
nfacct_cmd_restore, preventing arbitrary parameters being specified;

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/nfacct.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/src/nfacct.c b/src/nfacct.c
index 1324da7..bf50f50 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -298,7 +298,7 @@ static int _nfacct_cmd_add(char *name, uint64_t pkts, uint64_t bytes)
 
 static int nfacct_cmd_add(int argc, char *argv[])
 {
-	if (argc < 3) {
+	if (argc < 3 || strlen(argv[2]) == 0) {
 		nfacct_perror("missing object name");
 		return -1;
 	} else if (argc > 3) {
@@ -318,7 +318,7 @@ static int nfacct_cmd_delete(int argc, char *argv[])
 	struct nfacct *nfacct;
 	int ret;
 
-	if (argc < 3) {
+	if (argc < 3 || strlen(argv[2]) == 0) {
 		nfacct_perror("missing object name");
 		return -1;
 	} else if (argc > 3) {
@@ -385,7 +385,7 @@ static int nfacct_cmd_get(int argc, char *argv[])
 	struct nfacct *nfacct;
 	int ret, i;
 
-	if (argc < 3) {
+	if (argc < 3 || strlen(argv[2]) == 0) {
 		nfacct_perror("missing object name");
 		return -1;
 	}
@@ -546,6 +546,11 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 	char buffer[512];
 	int ret;
 
+	if (argc > 2) {
+		nfacct_perror("too many arguments");
+		return -1;
+	}
+
 	while (fgets(buffer, sizeof(buffer), stdin)) {
 		char *semicolon = strchr(buffer, ';');
 		if (semicolon == NULL) {
-- 
1.8.3.1

[PATCH v3 nfacct 7/29] code-refactoring changes to the "command menu"

[Michael Zintakis]

* replace the existing "command menu" with more efficient code, which is
clearer and easier to maintain;

* prevent wrong number of command-line arguments being specified

* prevent the correct command line parameters being specified more than once
(like "nfacct list xml xml" for example)

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/nfacct.c | 253 ++++++++++++++++++++++++++++-------------------------------
 1 file changed, 119 insertions(+), 134 deletions(-)

diff --git a/src/nfacct.c b/src/nfacct.c
index bf50f50..fbf9aa6 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -23,18 +23,6 @@
 #include <libmnl/libmnl.h>
 #include <libnetfilter_acct/libnetfilter_acct.h>
 
-enum {
-	NFACCT_CMD_NONE = 0,
-	NFACCT_CMD_LIST,
-	NFACCT_CMD_ADD,
-	NFACCT_CMD_DELETE,
-	NFACCT_CMD_GET,
-	NFACCT_CMD_FLUSH,
-	NFACCT_CMD_VERSION,
-	NFACCT_CMD_HELP,
-	NFACCT_CMD_RESTORE,
-};
-
 static int nfacct_cmd_list(int argc, char *argv[]);
 static int nfacct_cmd_add(int argc, char *argv[]);
 static int nfacct_cmd_delete(int argc, char *argv[]);
@@ -44,9 +32,57 @@ static int nfacct_cmd_version(int argc, char *argv[]);
 static int nfacct_cmd_help(int argc, char *argv[]);
 static int nfacct_cmd_restore(int argc, char *argv[]);
 
+/* Matches two strings, including partial matches */
+static int nfacct_matches(const char *cmd, const char *pattern)
+{
+	size_t len;
+
+	if (cmd == NULL || pattern == NULL)
+		return 0;
+
+	len = strlen(cmd);
+	if (len == 0 || len > strlen(pattern))
+		return 0;
+
+	return (strncmp(cmd, pattern, len) == 0);
+}
+
+/* main command 'menu' */
+static const struct cmd {
+	const char *cmd;
+	int (*func)(int argc, char **argv);
+} cmds[] = {
+	{ "list", 	nfacct_cmd_list },
+	{ "add",	nfacct_cmd_add },
+	{ "delete",	nfacct_cmd_delete },
+	{ "get",	nfacct_cmd_get },
+	{ "flush",	nfacct_cmd_flush },
+	{ "restore",	nfacct_cmd_restore },
+	{ "version",	nfacct_cmd_version },
+	{ "help",	nfacct_cmd_help },
+	{ NULL, NULL }
+};
+
+static int nfacct_do_cmd(const char *argv0, int argc, char **argv)
+{
+	const struct cmd *c;
+
+	for (c = cmds; c->cmd; ++c) {
+		if (nfacct_matches(argv0, c->cmd)) {
+			return (c->func(argc-1, argv+1));
+		}
+	}
+	fprintf(stderr, "nfacct v%s: Unknown command: %s\n",
+		VERSION, argv0);
+	return EXIT_FAILURE;
+}
+
+
+static void usage(char *argv[]) __attribute__((noreturn));
 static void usage(char *argv[])
 {
 	fprintf(stderr, "Usage: %s command [parameters]...\n", argv[0]);
+	exit(EXIT_FAILURE);
 }
 
 static void nfacct_perror(const char *msg)
@@ -59,80 +95,24 @@ static void nfacct_perror(const char *msg)
 	}
 }
 
-/* Matches two strings, including partial matches */
-static int nfacct_matches(const char *cmd, const char *pattern)
-{
-	size_t len;
-
-	if (cmd == NULL || pattern == NULL)
-		return 0;
-
-	len = strlen(cmd);
-	if (len == 0 || len > strlen(pattern))
-		return 0;
-
-	return (strncmp(cmd, pattern, len) == 0);
-}
+#define NFACCT_RET_ERR(x)	nfacct_perror(x); \
+				goto err;
+#define NFACCT_RET_ARG_ERR()	NFACCT_RET_ERR("unknown argument")
+#define NFACCT_NEXT_ARG() 	do {		\
+					argv++;	\
+					argc--;	\
+				} while(0)
 
 int main(int argc, char *argv[])
 {
-	int cmd = NFACCT_CMD_NONE, ret = 0;
-
-	if (argc < 2) {
-		usage(argv);
-		exit(EXIT_FAILURE);
-	}
-
-	if (nfacct_matches(argv[1], "list"))
-		cmd = NFACCT_CMD_LIST;
-	else if (nfacct_matches(argv[1], "add"))
-		cmd = NFACCT_CMD_ADD;
-	else if (nfacct_matches(argv[1], "delete"))
-		cmd = NFACCT_CMD_DELETE;
-	else if (nfacct_matches(argv[1], "get"))
-		cmd = NFACCT_CMD_GET;
-	else if (nfacct_matches(argv[1], "flush"))
-		cmd = NFACCT_CMD_FLUSH;
-	else if (nfacct_matches(argv[1], "version"))
-		cmd = NFACCT_CMD_VERSION;
-	else if (nfacct_matches(argv[1], "help"))
-		cmd = NFACCT_CMD_HELP;
-	else if (nfacct_matches(argv[1], "restore"))
-		cmd = NFACCT_CMD_RESTORE;
-	else {
-		fprintf(stderr, "nfacct v%s: Unknown command: %s\n",
-			VERSION, argv[1]);
-		usage(argv);
-		exit(EXIT_FAILURE);
-	}
-
-	switch(cmd) {
-	case NFACCT_CMD_LIST:
-		ret = nfacct_cmd_list(argc, argv);
-		break;
-	case NFACCT_CMD_ADD:
-		ret = nfacct_cmd_add(argc, argv);
-		break;
-	case NFACCT_CMD_DELETE:
-		ret = nfacct_cmd_delete(argc, argv);
-		break;
-	case NFACCT_CMD_GET:
-		ret = nfacct_cmd_get(argc, argv);
-		break;
-	case NFACCT_CMD_FLUSH:
-		ret = nfacct_cmd_flush(argc, argv);
-		break;
-	case NFACCT_CMD_VERSION:
-		ret = nfacct_cmd_version(argc, argv);
-		break;
-	case NFACCT_CMD_HELP:
-		ret = nfacct_cmd_help(argc, argv);
-		break;
-	case NFACCT_CMD_RESTORE:
-		ret = nfacct_cmd_restore(argc, argv);
-		break;
-	}
-	return ret < 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+	int ret = 0;
+
+	if (argc >= 2) {
+		ret = nfacct_do_cmd(argv[1], argc-1, argv+1);
+		if (ret != EXIT_FAILURE)
+			return ret < 0 ? EXIT_FAILURE : EXIT_SUCCESS;
+	}
+	usage(argv);
 }
 
 static bool xml_header = false;
@@ -174,29 +154,29 @@ err:
 
 static int nfacct_cmd_list(int argc, char *argv[])
 {
-	bool zeroctr = false, xml = false;
+	bool nfnl_msg = false, xml = false;
 	struct mnl_socket *nl;
 	char buf[MNL_SOCKET_BUFFER_SIZE];
 	struct nlmsghdr *nlh;
 	unsigned int seq, portid;
-	int ret, i;
+	int ret;
+	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
 
-	for (i=2; i<argc; i++) {
-		if (nfacct_matches(argv[i], "reset")) {
-			zeroctr = true;
-		} else if (nfacct_matches(argv[i], "xml")) {
+	while (argc > 0) {
+		if (!nfnl_msg && nfacct_matches(argv[0],"reset")) {
+			nfnl_cmd = NFNL_MSG_ACCT_GET_CTRZERO;
+			nfnl_msg = true;
+		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
 		} else {
 			nfacct_perror("unknown argument");
 			return -1;
 		}
+		argc--; argv++;
 	}
 
 	seq = time(NULL);
-	nlh = nfacct_nlmsg_build_hdr(buf, zeroctr ?
-					NFNL_MSG_ACCT_GET_CTRZERO :
-					NFNL_MSG_ACCT_GET,
-				     NLM_F_DUMP, seq);
+	nlh = nfacct_nlmsg_build_hdr(buf, nfnl_cmd, NLM_F_DUMP, seq);
 
 	nl = mnl_socket_open(NETLINK_NETFILTER);
 	if (nl == NULL) {
@@ -298,15 +278,15 @@ static int _nfacct_cmd_add(char *name, uint64_t pkts, uint64_t bytes)
 
 static int nfacct_cmd_add(int argc, char *argv[])
 {
-	if (argc < 3 || strlen(argv[2]) == 0) {
+	if (argc < 1 || strlen(argv[0]) == 0) {
 		nfacct_perror("missing object name");
 		return -1;
-	} else if (argc > 3) {
+	} else if (argc > 1) {
 		nfacct_perror("too many arguments");
 		return -1;
 	}
 
-	return _nfacct_cmd_add(argv[2], 0, 0);
+	return _nfacct_cmd_add(argv[0], 0, 0);
 }
 
 static int nfacct_cmd_delete(int argc, char *argv[])
@@ -318,10 +298,10 @@ static int nfacct_cmd_delete(int argc, char *argv[])
 	struct nfacct *nfacct;
 	int ret;
 
-	if (argc < 3 || strlen(argv[2]) == 0) {
+	if (argc < 1 || strlen(argv[0]) == 0) {
 		nfacct_perror("missing object name");
 		return -1;
-	} else if (argc > 3) {
+	} else if (argc > 1) {
 		nfacct_perror("too many arguments");
 		return -1;
 	}
@@ -332,7 +312,7 @@ static int nfacct_cmd_delete(int argc, char *argv[])
 		return -1;
 	}
 
-	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, argv[2]);
+	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, argv[0]);
 
 	seq = time(NULL);
 	nlh = nfacct_nlmsg_build_hdr(buf, NFNL_MSG_ACCT_DEL,
@@ -377,41 +357,46 @@ static int nfacct_cmd_delete(int argc, char *argv[])
 
 static int nfacct_cmd_get(int argc, char *argv[])
 {
-	bool zeroctr = false, xml = false;
+	bool nfnl_msg = false, xml = false;
 	struct mnl_socket *nl;
 	char buf[MNL_SOCKET_BUFFER_SIZE];
 	struct nlmsghdr *nlh;
 	uint32_t portid, seq;
 	struct nfacct *nfacct;
-	int ret, i;
+	int ret = -1, i;
+	char *name;
+	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
 
-	if (argc < 3 || strlen(argv[2]) == 0) {
+	if (argc < 1 || strlen(argv[0]) == 0) {
 		nfacct_perror("missing object name");
 		return -1;
 	}
-	for (i=3; i<argc; i++) {
-		if (nfacct_matches(argv[i], "reset")) {
-			zeroctr = true;
-		} else if (nfacct_matches(argv[i], "xml")) {
+	name = strdup(argv[0]);
+	if (!name) {
+		nfacct_perror("OOM");
+		return -1;
+	}
+	NFACCT_NEXT_ARG();
+	while (argc > 0) {
+		if (!nfnl_msg && nfacct_matches(argv[0],"reset")) {
+			nfnl_cmd = NFNL_MSG_ACCT_GET_CTRZERO;
+			nfnl_msg = true;
+		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
 		} else {
-			nfacct_perror("unknown argument");
-			return -1;
+			NFACCT_RET_ARG_ERR();
 		}
+		argc--; argv++;
 	}
 
 	nfacct = nfacct_alloc();
 	if (nfacct == NULL) {
-		nfacct_perror("OOM");
-		return -1;
+		NFACCT_RET_ERR("OOM");
 	}
-	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, argv[2]);
+	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, name);
 
 	seq = time(NULL);
-	nlh = nfacct_nlmsg_build_hdr(buf, zeroctr ?
-					NFNL_MSG_ACCT_GET_CTRZERO :
-					NFNL_MSG_ACCT_GET,
-				     NLM_F_ACK, seq);
+	nlh = nfacct_nlmsg_build_hdr(buf, nfnl_cmd, NLM_F_ACK, seq);
 
 	nfacct_nlmsg_build_payload(nlh, nfacct);
 
@@ -419,38 +404,38 @@ static int nfacct_cmd_get(int argc, char *argv[])
 
 	nl = mnl_socket_open(NETLINK_NETFILTER);
 	if (nl == NULL) {
-		nfacct_perror("mnl_socket_open");
-		return -1;
+		NFACCT_RET_ERR("mnl_socket_open");
 	}
 
 	if (mnl_socket_bind(nl, 0, MNL_SOCKET_AUTOPID) < 0) {
-		nfacct_perror("mnl_socket_bind");
-		return -1;
+		NFACCT_RET_ERR("mnl_socket_bind");
 	}
 	portid = mnl_socket_get_portid(nl);
 
 	if (mnl_socket_sendto(nl, nlh, nlh->nlmsg_len) < 0) {
-		nfacct_perror("mnl_socket_send");
-		return -1;
+		NFACCT_RET_ERR("mnl_socket_send");
 	}
 
-	ret = mnl_socket_recvfrom(nl, buf, sizeof(buf));
-	while (ret > 0) {
-		ret = mnl_cb_run(buf, ret, seq, portid, nfacct_cb, &xml);
-		if (ret <= 0)
+	i = mnl_socket_recvfrom(nl, buf, sizeof(buf));
+	while (i > 0) {
+		i = mnl_cb_run(buf, i, seq, portid, nfacct_cb, &xml);
+		if (i <= 0)
 			break;
-		ret = mnl_socket_recvfrom(nl, buf, sizeof(buf));
+		i = mnl_socket_recvfrom(nl, buf, sizeof(buf));
 	}
-	if (ret == -1) {
-		nfacct_perror("error");
-		return -1;
+	if (i == -1) {
+		NFACCT_RET_ERR("error");
 	}
 	mnl_socket_close(nl);
 
 	if (xml_header)
 		printf("</nfacct>\n");
 
-	return 0;
+	ret = 0;
+
+err:
+	free(name);
+	return ret;
 }
 
 static int nfacct_cmd_flush(int argc, char *argv[])
@@ -461,7 +446,7 @@ static int nfacct_cmd_flush(int argc, char *argv[])
 	uint32_t portid, seq;
 	int ret;
 
-	if (argc > 2) {
+	if (argc > 0) {
 		nfacct_perror("too many arguments");
 		return -1;
 	}
@@ -522,7 +507,7 @@ static int nfacct_cmd_version(int argc, char *argv[])
 static const char help_msg[] =
 	"nfacct v%s: utility for the Netfilter extended accounting "
 	"infrastructure\n"
-	"Usage: %s command [parameters]...\n\n"
+	"Usage: nfacct command [parameters]...\n\n"
 	"Commands:\n"
 	"  list [reset]\t\tList the accounting object table (and reset)\n"
 	"  add object-name\tAdd new accounting object to table\n"
@@ -535,7 +520,7 @@ static const char help_msg[] =
 
 static int nfacct_cmd_help(int argc, char *argv[])
 {
-	printf(help_msg, VERSION, argv[0]);
+	printf(help_msg, VERSION);
 	return 0;
 }
 
@@ -546,7 +531,7 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 	char buffer[512];
 	int ret;
 
-	if (argc > 2) {
+	if (argc > 0) {
 		nfacct_perror("too many arguments");
 		return -1;
 	}
-- 
1.8.3.1

Re: [PATCH v3 nfacct 7/29] code-refactoring changes to the "command menu"

[Pablo Neira Ayuso]

On Wed, Jul 10, 2013 at 07:25:05PM +0100, Michael Zintakis wrote:
> * replace the existing "command menu" with more efficient code, which is
> clearer and easier to maintain;

Can you provide some numbers to prove that efficiency gain?

> * prevent wrong number of command-line arguments being specified
> 
> * prevent the correct command line parameters being specified more than once
> (like "nfacct list xml xml" for example)
> 
> Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
> ---
>  src/nfacct.c | 253 ++++++++++++++++++++++++++++-------------------------------
>  1 file changed, 119 insertions(+), 134 deletions(-)
> 
> diff --git a/src/nfacct.c b/src/nfacct.c
> index bf50f50..fbf9aa6 100644
> --- a/src/nfacct.c
> +++ b/src/nfacct.c
> @@ -23,18 +23,6 @@
>  #include <libmnl/libmnl.h>
>  #include <libnetfilter_acct/libnetfilter_acct.h>
>  
> -enum {
> -	NFACCT_CMD_NONE = 0,
> -	NFACCT_CMD_LIST,
> -	NFACCT_CMD_ADD,
> -	NFACCT_CMD_DELETE,
> -	NFACCT_CMD_GET,
> -	NFACCT_CMD_FLUSH,
> -	NFACCT_CMD_VERSION,
> -	NFACCT_CMD_HELP,
> -	NFACCT_CMD_RESTORE,
> -};
> -
>  static int nfacct_cmd_list(int argc, char *argv[]);
>  static int nfacct_cmd_add(int argc, char *argv[]);
>  static int nfacct_cmd_delete(int argc, char *argv[]);
> @@ -44,9 +32,57 @@ static int nfacct_cmd_version(int argc, char *argv[]);
>  static int nfacct_cmd_help(int argc, char *argv[]);
>  static int nfacct_cmd_restore(int argc, char *argv[]);
>  
> +/* Matches two strings, including partial matches */
> +static int nfacct_matches(const char *cmd, const char *pattern)
> +{
> +	size_t len;
> +
> +	if (cmd == NULL || pattern == NULL)
> +		return 0;
> +
> +	len = strlen(cmd);
> +	if (len == 0 || len > strlen(pattern))
> +		return 0;

These checking above are superfluous.

> +	return (strncmp(cmd, pattern, len) == 0);

This shortened matching is not described in the patch description and
it does not handle clashes.

> +}
> +
> +/* main command 'menu' */
> +static const struct cmd {
> +	const char *cmd;
> +	int (*func)(int argc, char **argv);
> +} cmds[] = {
> +	{ "list", 	nfacct_cmd_list },
> +	{ "add",	nfacct_cmd_add },
> +	{ "delete",	nfacct_cmd_delete },
> +	{ "get",	nfacct_cmd_get },
> +	{ "flush",	nfacct_cmd_flush },
> +	{ "restore",	nfacct_cmd_restore },
> +	{ "version",	nfacct_cmd_version },
> +	{ "help",	nfacct_cmd_help },
> +	{ NULL, NULL }
> +};
> +
> +static int nfacct_do_cmd(const char *argv0, int argc, char **argv)
> +{
> +	const struct cmd *c;
> +
> +	for (c = cmds; c->cmd; ++c) {
> +		if (nfacct_matches(argv0, c->cmd)) {
> +			return (c->func(argc-1, argv+1));
> +		}
> +	}
> +	fprintf(stderr, "nfacct v%s: Unknown command: %s\n",
> +		VERSION, argv0);
> +	return EXIT_FAILURE;
> +}
> +
> +
> +static void usage(char *argv[]) __attribute__((noreturn));
>  static void usage(char *argv[])
>  {
>  	fprintf(stderr, "Usage: %s command [parameters]...\n", argv[0]);
> +	exit(EXIT_FAILURE);
>  }
>  
>  static void nfacct_perror(const char *msg)
> @@ -59,80 +95,24 @@ static void nfacct_perror(const char *msg)
>  	}
>  }
>  
> -/* Matches two strings, including partial matches */
> -static int nfacct_matches(const char *cmd, const char *pattern)
> -{
> -	size_t len;
> -
> -	if (cmd == NULL || pattern == NULL)
> -		return 0;
> -
> -	len = strlen(cmd);
> -	if (len == 0 || len > strlen(pattern))
> -		return 0;
> -
> -	return (strncmp(cmd, pattern, len) == 0);
> -}
> +#define NFACCT_RET_ERR(x)	nfacct_perror(x); \
> +				goto err;

Hiding a goto in a macro is not good idea.

> +#define NFACCT_RET_ARG_ERR()	NFACCT_RET_ERR("unknown argument")
>
> +#define NFACCT_NEXT_ARG() 	do {		\
> +					argv++;	\
> +					argc--;	\
> +				} while(0)
>  
>  int main(int argc, char *argv[])
>  {
> -	int cmd = NFACCT_CMD_NONE, ret = 0;
> -
> -	if (argc < 2) {
> -		usage(argv);
> -		exit(EXIT_FAILURE);
> -	}
> -
> -	if (nfacct_matches(argv[1], "list"))
> -		cmd = NFACCT_CMD_LIST;
> -	else if (nfacct_matches(argv[1], "add"))
> -		cmd = NFACCT_CMD_ADD;
> -	else if (nfacct_matches(argv[1], "delete"))
> -		cmd = NFACCT_CMD_DELETE;
> -	else if (nfacct_matches(argv[1], "get"))
> -		cmd = NFACCT_CMD_GET;
> -	else if (nfacct_matches(argv[1], "flush"))
> -		cmd = NFACCT_CMD_FLUSH;
> -	else if (nfacct_matches(argv[1], "version"))
> -		cmd = NFACCT_CMD_VERSION;
> -	else if (nfacct_matches(argv[1], "help"))
> -		cmd = NFACCT_CMD_HELP;
> -	else if (nfacct_matches(argv[1], "restore"))
> -		cmd = NFACCT_CMD_RESTORE;
> -	else {
> -		fprintf(stderr, "nfacct v%s: Unknown command: %s\n",
> -			VERSION, argv[1]);
> -		usage(argv);
> -		exit(EXIT_FAILURE);
> -	}
> -
> -	switch(cmd) {
> -	case NFACCT_CMD_LIST:
> -		ret = nfacct_cmd_list(argc, argv);
> -		break;
> -	case NFACCT_CMD_ADD:
> -		ret = nfacct_cmd_add(argc, argv);
> -		break;
> -	case NFACCT_CMD_DELETE:
> -		ret = nfacct_cmd_delete(argc, argv);
> -		break;
> -	case NFACCT_CMD_GET:
> -		ret = nfacct_cmd_get(argc, argv);
> -		break;
> -	case NFACCT_CMD_FLUSH:
> -		ret = nfacct_cmd_flush(argc, argv);
> -		break;
> -	case NFACCT_CMD_VERSION:
> -		ret = nfacct_cmd_version(argc, argv);
> -		break;
> -	case NFACCT_CMD_HELP:
> -		ret = nfacct_cmd_help(argc, argv);
> -		break;
> -	case NFACCT_CMD_RESTORE:
> -		ret = nfacct_cmd_restore(argc, argv);
> -		break;
> -	}
> -	return ret < 0 ? EXIT_FAILURE : EXIT_SUCCESS;
> +	int ret = 0;
> +
> +	if (argc >= 2) {
> +		ret = nfacct_do_cmd(argv[1], argc-1, argv+1);
> +		if (ret != EXIT_FAILURE)
> +			return ret < 0 ? EXIT_FAILURE : EXIT_SUCCESS;
> +	}
> +	usage(argv);
>  }
>  
>  static bool xml_header = false;
> @@ -174,29 +154,29 @@ err:
>  
>  static int nfacct_cmd_list(int argc, char *argv[])
>  {
> -	bool zeroctr = false, xml = false;
> +	bool nfnl_msg = false, xml = false;
>  	struct mnl_socket *nl;
>  	char buf[MNL_SOCKET_BUFFER_SIZE];
>  	struct nlmsghdr *nlh;
>  	unsigned int seq, portid;
> -	int ret, i;
> +	int ret;
> +	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
>  
> -	for (i=2; i<argc; i++) {
> -		if (nfacct_matches(argv[i], "reset")) {
> -			zeroctr = true;
> -		} else if (nfacct_matches(argv[i], "xml")) {
> +	while (argc > 0) {
> +		if (!nfnl_msg && nfacct_matches(argv[0],"reset")) {
                                                        ^
                                                 space after comma

> +			nfnl_cmd = NFNL_MSG_ACCT_GET_CTRZERO;
> +			nfnl_msg = true;
> +		} else if (!xml && nfacct_matches(argv[0],"xml")) {
>  			xml = true;
>  		} else {
>  			nfacct_perror("unknown argument");
>  			return -1;
>  		}
> +		argc--; argv++;
>  	}
>  
>  	seq = time(NULL);
> -	nlh = nfacct_nlmsg_build_hdr(buf, zeroctr ?
> -					NFNL_MSG_ACCT_GET_CTRZERO :
> -					NFNL_MSG_ACCT_GET,
> -				     NLM_F_DUMP, seq);
> +	nlh = nfacct_nlmsg_build_hdr(buf, nfnl_cmd, NLM_F_DUMP, seq);
>  
>  	nl = mnl_socket_open(NETLINK_NETFILTER);
>  	if (nl == NULL) {
> @@ -298,15 +278,15 @@ static int _nfacct_cmd_add(char *name, uint64_t pkts, uint64_t bytes)
>  
>  static int nfacct_cmd_add(int argc, char *argv[])
>  {
> -	if (argc < 3 || strlen(argv[2]) == 0) {
> +	if (argc < 1 || strlen(argv[0]) == 0) {
>  		nfacct_perror("missing object name");
>  		return -1;
> -	} else if (argc > 3) {
> +	} else if (argc > 1) {
>  		nfacct_perror("too many arguments");
>  		return -1;
>  	}
>  
> -	return _nfacct_cmd_add(argv[2], 0, 0);
> +	return _nfacct_cmd_add(argv[0], 0, 0);
>  }
>  
>  static int nfacct_cmd_delete(int argc, char *argv[])
> @@ -318,10 +298,10 @@ static int nfacct_cmd_delete(int argc, char *argv[])
>  	struct nfacct *nfacct;
>  	int ret;
>  
> -	if (argc < 3 || strlen(argv[2]) == 0) {
> +	if (argc < 1 || strlen(argv[0]) == 0) {
>  		nfacct_perror("missing object name");
>  		return -1;
> -	} else if (argc > 3) {
> +	} else if (argc > 1) {
>  		nfacct_perror("too many arguments");
>  		return -1;
>  	}
> @@ -332,7 +312,7 @@ static int nfacct_cmd_delete(int argc, char *argv[])
>  		return -1;
>  	}
>  
> -	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, argv[2]);
> +	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, argv[0]);
>  
>  	seq = time(NULL);
>  	nlh = nfacct_nlmsg_build_hdr(buf, NFNL_MSG_ACCT_DEL,
> @@ -377,41 +357,46 @@ static int nfacct_cmd_delete(int argc, char *argv[])
>  
>  static int nfacct_cmd_get(int argc, char *argv[])
>  {
> -	bool zeroctr = false, xml = false;
> +	bool nfnl_msg = false, xml = false;
>  	struct mnl_socket *nl;
>  	char buf[MNL_SOCKET_BUFFER_SIZE];
>  	struct nlmsghdr *nlh;
>  	uint32_t portid, seq;
>  	struct nfacct *nfacct;
> -	int ret, i;
> +	int ret = -1, i;
> +	char *name;
> +	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
>  
> -	if (argc < 3 || strlen(argv[2]) == 0) {
> +	if (argc < 1 || strlen(argv[0]) == 0) {
>  		nfacct_perror("missing object name");
>  		return -1;
>  	}
> -	for (i=3; i<argc; i++) {
> -		if (nfacct_matches(argv[i], "reset")) {
> -			zeroctr = true;
> -		} else if (nfacct_matches(argv[i], "xml")) {
> +	name = strdup(argv[0]);
> +	if (!name) {
> +		nfacct_perror("OOM");
> +		return -1;
> +	}
> +	NFACCT_NEXT_ARG();
> +	while (argc > 0) {
> +		if (!nfnl_msg && nfacct_matches(argv[0],"reset")) {
> +			nfnl_cmd = NFNL_MSG_ACCT_GET_CTRZERO;
> +			nfnl_msg = true;
> +		} else if (!xml && nfacct_matches(argv[0],"xml")) {
>  			xml = true;
>  		} else {
> -			nfacct_perror("unknown argument");
> -			return -1;
> +			NFACCT_RET_ARG_ERR();
>  		}
> +		argc--; argv++;
>  	}
>  
>  	nfacct = nfacct_alloc();
>  	if (nfacct == NULL) {
> -		nfacct_perror("OOM");
> -		return -1;
> +		NFACCT_RET_ERR("OOM");
>  	}
> -	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, argv[2]);
> +	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, name);
>  
>  	seq = time(NULL);
> -	nlh = nfacct_nlmsg_build_hdr(buf, zeroctr ?
> -					NFNL_MSG_ACCT_GET_CTRZERO :
> -					NFNL_MSG_ACCT_GET,
> -				     NLM_F_ACK, seq);
> +	nlh = nfacct_nlmsg_build_hdr(buf, nfnl_cmd, NLM_F_ACK, seq);
>  
>  	nfacct_nlmsg_build_payload(nlh, nfacct);
>  
> @@ -419,38 +404,38 @@ static int nfacct_cmd_get(int argc, char *argv[])
>  
>  	nl = mnl_socket_open(NETLINK_NETFILTER);
>  	if (nl == NULL) {
> -		nfacct_perror("mnl_socket_open");
> -		return -1;
> +		NFACCT_RET_ERR("mnl_socket_open");
>  	}
>  
>  	if (mnl_socket_bind(nl, 0, MNL_SOCKET_AUTOPID) < 0) {
> -		nfacct_perror("mnl_socket_bind");
> -		return -1;
> +		NFACCT_RET_ERR("mnl_socket_bind");
>  	}
>  	portid = mnl_socket_get_portid(nl);
>  
>  	if (mnl_socket_sendto(nl, nlh, nlh->nlmsg_len) < 0) {
> -		nfacct_perror("mnl_socket_send");
> -		return -1;
> +		NFACCT_RET_ERR("mnl_socket_send");
>  	}
>  
> -	ret = mnl_socket_recvfrom(nl, buf, sizeof(buf));
> -	while (ret > 0) {
> -		ret = mnl_cb_run(buf, ret, seq, portid, nfacct_cb, &xml);
> -		if (ret <= 0)
> +	i = mnl_socket_recvfrom(nl, buf, sizeof(buf));
> +	while (i > 0) {
> +		i = mnl_cb_run(buf, i, seq, portid, nfacct_cb, &xml);
> +		if (i <= 0)
>  			break;
> -		ret = mnl_socket_recvfrom(nl, buf, sizeof(buf));
> +		i = mnl_socket_recvfrom(nl, buf, sizeof(buf));
>  	}
> -	if (ret == -1) {
> -		nfacct_perror("error");
> -		return -1;
> +	if (i == -1) {
> +		NFACCT_RET_ERR("error");
>  	}
>  	mnl_socket_close(nl);
>  
>  	if (xml_header)
>  		printf("</nfacct>\n");
>  
> -	return 0;
> +	ret = 0;
> +
> +err:
> +	free(name);
> +	return ret;
>  }
>  
>  static int nfacct_cmd_flush(int argc, char *argv[])
> @@ -461,7 +446,7 @@ static int nfacct_cmd_flush(int argc, char *argv[])
>  	uint32_t portid, seq;
>  	int ret;
>  
> -	if (argc > 2) {
> +	if (argc > 0) {
>  		nfacct_perror("too many arguments");
>  		return -1;
>  	}
> @@ -522,7 +507,7 @@ static int nfacct_cmd_version(int argc, char *argv[])
>  static const char help_msg[] =
>  	"nfacct v%s: utility for the Netfilter extended accounting "
>  	"infrastructure\n"
> -	"Usage: %s command [parameters]...\n\n"
> +	"Usage: nfacct command [parameters]...\n\n"
>  	"Commands:\n"
>  	"  list [reset]\t\tList the accounting object table (and reset)\n"
>  	"  add object-name\tAdd new accounting object to table\n"
> @@ -535,7 +520,7 @@ static const char help_msg[] =
>  
>  static int nfacct_cmd_help(int argc, char *argv[])
>  {
> -	printf(help_msg, VERSION, argv[0]);
> +	printf(help_msg, VERSION);
>  	return 0;
>  }
>  
> @@ -546,7 +531,7 @@ static int nfacct_cmd_restore(int argc, char *argv[])
>  	char buffer[512];
>  	int ret;
>  
> -	if (argc > 2) {
> +	if (argc > 0) {
>  		nfacct_perror("too many arguments");
>  		return -1;
>  	}
> -- 
> 1.8.3.1
> 

[PATCH v3 nfacct 8/29] add 2 new options: "replace" and "flush"

[Michael Zintakis]

* add "replace" option to the "add" command, allowing nfacct object to be
replaced even if it is used by the kernel/iptables;

* add "replace" and "flush" options to the "restore" command, allowing nfacct
objects to be replaced even if used by the kernel/iptables and also allow the
entire nfacct object list to be deleted, when permissable, prior to "restore"
being executed;

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/nfacct.c | 99 +++++++++++++++++++++++++++++++++++++++++++++---------------
 1 file changed, 74 insertions(+), 25 deletions(-)

diff --git a/src/nfacct.c b/src/nfacct.c
index fbf9aa6..7808ebb 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -214,29 +214,21 @@ static int nfacct_cmd_list(int argc, char *argv[])
 	return 0;
 }
 
-static int _nfacct_cmd_add(char *name, uint64_t pkts, uint64_t bytes)
+static int _nfacct_cmd_add(struct nfacct *nfacct, bool replace)
 {
 	struct mnl_socket *nl;
 	char buf[MNL_SOCKET_BUFFER_SIZE];
 	struct nlmsghdr *nlh;
 	uint32_t portid, seq;
-	struct nfacct *nfacct;
 	int ret;
 
-	nfacct = nfacct_alloc();
-	if (nfacct == NULL) {
-		nfacct_perror("OOM");
+	if (nfacct == NULL)
 		return -1;
-	}
-
-	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, name);
-
-	nfacct_attr_set_u64(nfacct, NFACCT_ATTR_PKTS, pkts);
-	nfacct_attr_set_u64(nfacct, NFACCT_ATTR_BYTES, bytes);
 
 	seq = time(NULL);
 	nlh = nfacct_nlmsg_build_hdr(buf, NFNL_MSG_ACCT_NEW,
-				     NLM_F_CREATE | NLM_F_ACK, seq);
+				     NLM_F_CREATE | NLM_F_ACK |
+				     (replace ? NLM_F_REPLACE : 0), seq);
 	nfacct_nlmsg_build_payload(nlh, nfacct);
 
 	nfacct_free(nfacct);
@@ -278,15 +270,41 @@ static int _nfacct_cmd_add(char *name, uint64_t pkts, uint64_t bytes)
 
 static int nfacct_cmd_add(int argc, char *argv[])
 {
+	int ret = -1;
+	bool replace = false;
+	char *name;
+	struct nfacct *nfacct;
+
 	if (argc < 1 || strlen(argv[0]) == 0) {
 		nfacct_perror("missing object name");
 		return -1;
-	} else if (argc > 1) {
-		nfacct_perror("too many arguments");
+	}
+	name = strdup(argv[0]);
+	if (!name) {
+		nfacct_perror("OOM");
 		return -1;
 	}
+	NFACCT_NEXT_ARG();
+	while (argc > 0) {
+		if (!replace && nfacct_matches(argv[0],"replace")) {
+			replace = true;
+		} else {
+			NFACCT_RET_ARG_ERR();
+		}
+		argc--; argv++;
+	}
 
-	return _nfacct_cmd_add(argv[0], 0, 0);
+	nfacct = nfacct_alloc();
+	if (nfacct == NULL) {
+		NFACCT_RET_ERR("OOM");
+	}
+
+	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, name);
+	ret = _nfacct_cmd_add(nfacct, replace);
+
+err:
+	free(name);
+	return ret;
 }
 
 static int nfacct_cmd_delete(int argc, char *argv[])
@@ -509,14 +527,14 @@ static const char help_msg[] =
 	"infrastructure\n"
 	"Usage: nfacct command [parameters]...\n\n"
 	"Commands:\n"
-	"  list [reset]\t\tList the accounting object table (and reset)\n"
-	"  add object-name\tAdd new accounting object to table\n"
-	"  delete object-name\tDelete existing accounting object\n"
-	"  get object-name\tGet existing accounting object\n"
+	"  list [reset]\t\t\tList the accounting object table (and reset)\n"
+	"  add object-name [replace]\tAdd new accounting object to table\n"
+	"  delete object-name\t\tDelete existing accounting object\n"
+	"  get object-name\t\tGet existing accounting object\n"
 	"  flush\t\t\tFlush accounting object table\n"
-	"  restore\t\tRestore accounting object table reading 'list' output from stdin\n"
-	"  version\t\tDisplay version and disclaimer\n"
-	"  help\t\t\tDisplay this help message\n";
+	"  restore [flush] [replace]\tRestore accounting object table reading 'list' output from stdin\n"
+	"  version\t\t\tDisplay version and disclaimer\n"
+	"  help\t\t\t\tDisplay this help message\n";
 
 static int nfacct_cmd_help(int argc, char *argv[])
 {
@@ -530,11 +548,30 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 	char name[512];
 	char buffer[512];
 	int ret;
+	bool replace = false, flush = false;
+	struct nfacct *nfacct;
 
-	if (argc > 0) {
+	if (argc > 2) {
 		nfacct_perror("too many arguments");
 		return -1;
 	}
+	while (argc > 0) {
+		if (!replace && nfacct_matches(argv[0],"replace")) {
+			replace = true;
+		} else if (!flush && nfacct_matches(argv[0],"flush")) {
+			flush = true;
+		} else {
+			NFACCT_RET_ARG_ERR();
+		}
+		argc--; argv++;
+	}
+
+	if (flush) {
+		ret = nfacct_cmd_flush(0, NULL);
+		if (ret == -1) {
+			NFACCT_RET_ERR("flush not successful");
+		}
+	}
 
 	while (fgets(buffer, sizeof(buffer), stdin)) {
 		char *semicolon = strchr(buffer, ';');
@@ -550,9 +587,21 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 			nfacct_perror("error reading input");
 			return -1;
 		}
-		if ((ret = _nfacct_cmd_add(name, pkts, bytes)) != 0)
-			return ret;
+		nfacct = nfacct_alloc();
+		if (nfacct == NULL) {
+			NFACCT_RET_ERR("OOM error");
+		}
+		nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, name);
+		nfacct_attr_set_u64(nfacct, NFACCT_ATTR_PKTS, pkts);
+		nfacct_attr_set_u64(nfacct, NFACCT_ATTR_BYTES, bytes);
+		ret = _nfacct_cmd_add(nfacct, replace);
+		if (ret != 0) {
+			NFACCT_RET_ERR("error during add");
+		}
 
 	}
 	return 0;
+
+err:
+	return -1;
 }
-- 
1.8.3.1

[PATCH v3 libnetfilter_acct 9/29] add *_SAVE template allowing save/restore

[Michael Zintakis]

* add NFACCT_SNPRINTF_F_SAVE plaintext template, allowing printing of
accounting objects in a format suitable for the 'restore' command.
All nfacct properties are listed as name=value pairs, properly encoding space
and other "odd" characters when needed. Example:

name="ALL IN net 17" pkts=1234 bytes=3456789

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/libnetfilter_acct/libnetfilter_acct.h |  3 +++
 src/libnetfilter_acct.c                       | 11 +++++++++++
 2 files changed, 14 insertions(+)

diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index 9f66c39..ecf8280 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -39,6 +39,9 @@ int nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct
 #define NFACCT_SNPRINTF_F_FULL		(1 << 0)
 #define NFACCT_SNPRINTF_F_TIME		(1 << 1)
 
+/* print in a format suitable for 'restore' */
+#define NFACCT_SNPRINTF_F_SAVE		(1 << 2)
+
 #define NFACCT_SNPRINTF_T_PLAIN 0
 #define NFACCT_SNPRINTF_T_XML 1
 
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index 493be3b..e4fdf84 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -228,6 +228,9 @@ uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type)
 }
 EXPORT_SYMBOL(nfacct_attr_get_u64);
 
+#define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s pkts=%"PRIu64 \
+					" bytes=%"PRIu64
+
 void
 parse_nfacct_name(char *buf, const char *name)
 {
@@ -321,12 +324,20 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 	parse_nfacct_name(nfacct_name,
 			  nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
 	if (flags & NFACCT_SNPRINTF_F_FULL) {
+		/* default: print pkts + bytes + name */
 		ret = snprintf(buf, rem,
 			"{ pkts = %.20"PRIu64", bytes = %.20"PRIu64" } = %s;",
 			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS),
 			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
 			nfacct_name);
+	} else if (flags & NFACCT_SNPRINTF_F_SAVE) {
+		/* save: format useful for 'restore' */
+		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_SAVE_BASE,
+			       nfacct_name,
+			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_PKTS),
+			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BYTES));
 	} else {
+		/* print out name only */
 		ret = snprintf(buf, rem, "%s\n", nfacct_name);
 	}
 
-- 
1.8.3.1

[PATCH v3 libnetfilter_acct 10/29] add *_BONLY template to show bytes-only

[Michael Zintakis]

* add NFACCT_SNPRINTF_F_BONLY plaintext and xml templates, allowing printing
of only the bytes and name columns for each accounting object. Example:

{ bytes = 00000000018921291776 } = "ALL IN net 17"

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/libnetfilter_acct/libnetfilter_acct.h |  3 ++
 src/libnetfilter_acct.c                       | 41 ++++++++++++++++++++-------
 2 files changed, 34 insertions(+), 10 deletions(-)

diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index ecf8280..aec26d3 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -42,6 +42,9 @@ int nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct
 /* print in a format suitable for 'restore' */
 #define NFACCT_SNPRINTF_F_SAVE		(1 << 2)
 
+/* print only the bytes and name columns */
+#define NFACCT_SNPRINTF_F_BONLY		(1 << 3)
+
 #define NFACCT_SNPRINTF_T_PLAIN 0
 #define NFACCT_SNPRINTF_T_XML 1
 
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index e4fdf84..23f7616 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -230,6 +230,16 @@ EXPORT_SYMBOL(nfacct_attr_get_u64);
 
 #define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s pkts=%"PRIu64 \
 					" bytes=%"PRIu64
+#define NFACCT_STR_PLAIN		"{ pkts = %.20"PRIu64", " \
+					"bytes = %.20"PRIu64" } = %s"
+#define NFACCT_STR_PLAIN_BONLY		"{ bytes = %.20"PRIu64 " } = %s"
+#define NFACCT_XML_NAME			"<name>%s</name>"
+#define NFACCT_XML_PKTS			"<pkts>%.20"PRIu64"</pkts>"
+#define NFACCT_XML_BYTES		"<bytes>%.20"PRIu64"</bytes>"
+#define NFACCT_STR_XML_BONLY		"<obj>"	NFACCT_XML_NAME \
+					NFACCT_XML_BYTES
+#define NFACCT_STR_XML			"<obj>"	NFACCT_XML_NAME \
+					NFACCT_XML_PKTS NFACCT_XML_BYTES
 
 void
 parse_nfacct_name(char *buf, const char *name)
@@ -325,8 +335,7 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 			  nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
 	if (flags & NFACCT_SNPRINTF_F_FULL) {
 		/* default: print pkts + bytes + name */
-		ret = snprintf(buf, rem,
-			"{ pkts = %.20"PRIu64", bytes = %.20"PRIu64" } = %s;",
+		ret = snprintf(buf, rem, NFACCT_STR_PLAIN,
 			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS),
 			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
 			nfacct_name);
@@ -336,6 +345,11 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 			       nfacct_name,
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_PKTS),
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BYTES));
+	} else if (flags & NFACCT_SNPRINTF_F_BONLY) {
+		/* print bytes + name only */
+		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_BONLY,
+				nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
+				nfacct_name);
 	} else {
 		/* print out name only */
 		ret = snprintf(buf, rem, "%s\n", nfacct_name);
@@ -393,14 +407,21 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 	parse_nfacct_name_xml(nfacct_name,
 				nfacct_attr_get_str(nfacct,
 						    NFACCT_ATTR_NAME));
-	ret = snprintf(buf, rem,
-			"<obj><name>%s</name>"
-			"<pkts>%.20"PRIu64"</pkts>"
-			"<bytes>%.20"PRIu64"</bytes>",
-			nfacct_name,
-			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
-			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS));
-	BUFFER_SIZE(ret, size, rem, offset);
+	if (flags & NFACCT_SNPRINTF_F_BONLY) {
+		/* print name + bytes only */
+		ret = snprintf(buf, rem, NFACCT_STR_XML_BONLY, nfacct_name,
+				nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_BYTES));
+		BUFFER_SIZE(ret, size, rem, offset);
+	} else {
+		/* default/everything else: print name + pkts + bytes */
+		ret = snprintf(buf, rem, NFACCT_STR_XML, nfacct_name,
+				nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_BYTES),
+				nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_PKTS));
+		BUFFER_SIZE(ret, size, rem, offset);
+	}
 
 	if (flags & NFACCT_SNPRINTF_F_TIME) {
 		time_t t;
-- 
1.8.3.1

Re: [PATCH v3 libnetfilter_acct 10/29] add *_BONLY template to show bytes-only

[Pablo Neira Ayuso]

On Wed, Jul 10, 2013 at 07:25:08PM +0100, Michael Zintakis wrote:
> * add NFACCT_SNPRINTF_F_BONLY plaintext and xml templates, allowing printing
> of only the bytes and name columns for each accounting object. Example:
> 
> { bytes = 00000000018921291776 } = "ALL IN net 17"
> 
> Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
> ---
>  include/libnetfilter_acct/libnetfilter_acct.h |  3 ++
>  src/libnetfilter_acct.c                       | 41 ++++++++++++++++++++-------
>  2 files changed, 34 insertions(+), 10 deletions(-)
> 
> diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
> index ecf8280..aec26d3 100644
> --- a/include/libnetfilter_acct/libnetfilter_acct.h
> +++ b/include/libnetfilter_acct/libnetfilter_acct.h
> @@ -42,6 +42,9 @@ int nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct
>  /* print in a format suitable for 'restore' */
>  #define NFACCT_SNPRINTF_F_SAVE		(1 << 2)
>  
> +/* print only the bytes and name columns */
> +#define NFACCT_SNPRINTF_F_BONLY		(1 << 3)
> +
>  #define NFACCT_SNPRINTF_T_PLAIN 0
>  #define NFACCT_SNPRINTF_T_XML 1
>  
> diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
> index e4fdf84..23f7616 100644
> --- a/src/libnetfilter_acct.c
> +++ b/src/libnetfilter_acct.c
> @@ -230,6 +230,16 @@ EXPORT_SYMBOL(nfacct_attr_get_u64);
>  
>  #define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s pkts=%"PRIu64 \
>  					" bytes=%"PRIu64
> +#define NFACCT_STR_PLAIN		"{ pkts = %.20"PRIu64", " \
> +					"bytes = %.20"PRIu64" } = %s"
> +#define NFACCT_STR_PLAIN_BONLY		"{ bytes = %.20"PRIu64 " } = %s"
> +#define NFACCT_XML_NAME			"<name>%s</name>"
> +#define NFACCT_XML_PKTS			"<pkts>%.20"PRIu64"</pkts>"
> +#define NFACCT_XML_BYTES		"<bytes>%.20"PRIu64"</bytes>"
> +#define NFACCT_STR_XML_BONLY		"<obj>"	NFACCT_XML_NAME \
> +					NFACCT_XML_BYTES
> +#define NFACCT_STR_XML			"<obj>"	NFACCT_XML_NAME \
> +					NFACCT_XML_PKTS NFACCT_XML_BYTES

No macros please. They are not reused and you have to scroll up and
down to see the definition and its use.

>  void
>  parse_nfacct_name(char *buf, const char *name)
> @@ -325,8 +335,7 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
>  			  nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
>  	if (flags & NFACCT_SNPRINTF_F_FULL) {
>  		/* default: print pkts + bytes + name */
> -		ret = snprintf(buf, rem,
> -			"{ pkts = %.20"PRIu64", bytes = %.20"PRIu64" } = %s;",
> +		ret = snprintf(buf, rem, NFACCT_STR_PLAIN,
>  			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS),
>  			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
>  			nfacct_name);
> @@ -336,6 +345,11 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
>  			       nfacct_name,
>  			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_PKTS),
>  			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BYTES));
> +	} else if (flags & NFACCT_SNPRINTF_F_BONLY) {
> +		/* print bytes + name only */
> +		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_BONLY,
> +				nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
> +				nfacct_name);
>  	} else {
>  		/* print out name only */
>  		ret = snprintf(buf, rem, "%s\n", nfacct_name);
> @@ -393,14 +407,21 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
>  	parse_nfacct_name_xml(nfacct_name,
>  				nfacct_attr_get_str(nfacct,
>  						    NFACCT_ATTR_NAME));
> -	ret = snprintf(buf, rem,
> -			"<obj><name>%s</name>"
> -			"<pkts>%.20"PRIu64"</pkts>"
> -			"<bytes>%.20"PRIu64"</bytes>",
> -			nfacct_name,
> -			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
> -			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS));
> -	BUFFER_SIZE(ret, size, rem, offset);
> +	if (flags & NFACCT_SNPRINTF_F_BONLY) {
> +		/* print name + bytes only */
> +		ret = snprintf(buf, rem, NFACCT_STR_XML_BONLY, nfacct_name,
> +				nfacct_attr_get_u64(nfacct,
> +						    NFACCT_ATTR_BYTES));
> +		BUFFER_SIZE(ret, size, rem, offset);
> +	} else {
> +		/* default/everything else: print name + pkts + bytes */
> +		ret = snprintf(buf, rem, NFACCT_STR_XML, nfacct_name,
> +				nfacct_attr_get_u64(nfacct,
> +						    NFACCT_ATTR_BYTES),
> +				nfacct_attr_get_u64(nfacct,
> +						    NFACCT_ATTR_PKTS));
> +		BUFFER_SIZE(ret, size, rem, offset);
> +	}
>  
>  	if (flags & NFACCT_SNPRINTF_F_TIME) {
>  		time_t t;
> -- 
> 1.8.3.1
> 

[PATCH v3 libnetfilter_acct 11/29] add variable width and on-the-fly formatting

[Michael Zintakis]

* add a separate nfacct_options struct with its get/set functions, allowing
userspace programs, like nfacct, to specify the width of the columns which
are to be printed, so that they are no longer hard-coded into the printing
template itself. This allows column width to be "adjusted", depending on the
value of each property;

* add a separate NFACCT_SNPRINTF_F_NUMONLY template, which prints the
numbers-only properties of the accounting objects (formatted - see below) -
useful to the nfnetlink callback function in order to determine the maximum
column width, which needs to be used to print each column.

* add a new nfacct_snprintf_with_options function enhancing the existing API
to enable "options" to be specified during printing properties of nfacct
objects.

* allow on-the-fly formatting, using over 14 different formats when printing
accounting objects. This formatting, if specified, applies to *all*
accounting objects.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 configure.ac                                  |   2 +-
 include/libnetfilter_acct/libnetfilter_acct.h |  75 ++++
 src/libnetfilter_acct.c                       | 512 ++++++++++++++++++++++++--
 src/libnetfilter_acct.map                     |  10 +
 4 files changed, 569 insertions(+), 30 deletions(-)

diff --git a/configure.ac b/configure.ac
index ad1bef8..e8f21e7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -29,7 +29,7 @@ esac
 regular_CPPFLAGS="-D_FILE_OFFSET_BITS=64 -D_REENTRANT"
 regular_CFLAGS="-Wall -Waggregate-return -Wmissing-declarations \
 	-Wmissing-prototypes -Wshadow -Wstrict-prototypes \
-	-Wformat=2 -pipe"
+	-Wformat=2 -Wno-format-nonliteral -pipe"
 AC_SUBST([regular_CPPFLAGS])
 AC_SUBST([regular_CFLAGS])
 AC_CONFIG_FILES([Makefile src/Makefile include/Makefile include/libnetfilter_acct/Makefile include/linux/Makefile include/linux/netfilter/Makefile examples/Makefile libnetfilter_acct.pc doxygen.cfg])
diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index aec26d3..3328fdb 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -16,9 +16,62 @@ enum nfacct_attr_type {
 	NFACCT_ATTR_BYTES,
 };
 
+struct nfacct_options;
+
+enum nfacct_option_type {
+	NFACCT_OPT_FMT = 0,	/* number format option 	*/
+	NFACCT_OPT_PCW,		/* packets count column width 	*/
+	NFACCT_OPT_BCW,		/* bytes count column width 	*/
+};
+
+enum nfacct_format {
+	NFACCT_FMT_DEFAULT=0,
+	NFACCT_FMT_NONE,
+	NFACCT_FMT_TRIPLETS,
+	NFACCT_FMT_IEC,
+	NFACCT_FMT_IEC_KIBIBYTE,
+	NFACCT_FMT_IEC_MEBIBYTE,
+	NFACCT_FMT_IEC_GIBIBYTE,
+	NFACCT_FMT_IEC_TEBIBYTE,
+	NFACCT_FMT_IEC_PEBIBYTE,
+	NFACCT_FMT_IEC_EXBIBYTE,
+	NFACCT_FMT_SI,
+	NFACCT_FMT_SI_KILOBYTE,
+	NFACCT_FMT_SI_MEGABYTE,
+	NFACCT_FMT_SI_GIGABYTE,
+	NFACCT_FMT_SI_TERABYTE,
+	NFACCT_FMT_SI_PETABYTE,
+	NFACCT_FMT_SI_EXABYTE,
+	NFACCT_FMT_MAX,
+};
+
+static const char *nfacct_fmt_keys[NFACCT_FMT_MAX + 1] = {
+	[NFACCT_FMT_DEFAULT] 		= "def",
+	[NFACCT_FMT_NONE] 		= "raw",
+	[NFACCT_FMT_TRIPLETS] 		= "3pl",
+	[NFACCT_FMT_IEC] 		= "iec",
+	[NFACCT_FMT_IEC_KIBIBYTE] 	= "kib",
+	[NFACCT_FMT_IEC_MEBIBYTE] 	= "mib",
+	[NFACCT_FMT_IEC_GIBIBYTE] 	= "gib",
+	[NFACCT_FMT_IEC_TEBIBYTE] 	= "tib",
+	[NFACCT_FMT_IEC_PEBIBYTE] 	= "pib",
+	[NFACCT_FMT_IEC_EXBIBYTE] 	= "eib",
+	[NFACCT_FMT_SI] 		= "si",
+	[NFACCT_FMT_SI_KILOBYTE] 	= "kb",
+	[NFACCT_FMT_SI_MEGABYTE] 	= "mb",
+	[NFACCT_FMT_SI_GIGABYTE] 	= "gb",
+	[NFACCT_FMT_SI_TERABYTE] 	= "tb",
+	[NFACCT_FMT_SI_PETABYTE] 	= "pb",
+	[NFACCT_FMT_SI_EXABYTE] 	= "eb",
+	[NFACCT_FMT_MAX] 		= "",
+};
+
 struct nfacct *nfacct_alloc(void);
 void nfacct_free(struct nfacct *nfacct);
 
+struct nfacct_options *nfacct_options_alloc(void);
+void nfacct_options_free(struct nfacct_options *options);
+
 void nfacct_attr_set(struct nfacct *nfacct, enum nfacct_attr_type type, const void *data);
 void nfacct_attr_set_str(struct nfacct *nfacct, enum nfacct_attr_type type, const char *name);
 void nfacct_attr_set_u64(struct nfacct *nfacct, enum nfacct_attr_type type, uint64_t value);
@@ -28,6 +81,22 @@ const void *nfacct_attr_get(struct nfacct *nfacct, enum nfacct_attr_type type);
 const char *nfacct_attr_get_str(struct nfacct *nfacct, enum nfacct_attr_type type);
 uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type);
 
+void nfacct_option_set(struct nfacct_options *options,
+			enum nfacct_option_type type, const void *data);
+void nfacct_option_set_u16(struct nfacct_options *options,
+			enum nfacct_option_type type, uint16_t value);
+void nfacct_option_set_tsize(struct nfacct_options *options,
+			enum nfacct_option_type type, size_t value);
+void nfacct_option_unset(struct nfacct_options *options,
+			enum nfacct_option_type type);
+
+const void *nfacct_option_get(struct nfacct_options *options,
+			enum nfacct_option_type type);
+uint16_t nfacct_option_get_u16(struct nfacct_options *options,
+			enum nfacct_option_type type);
+size_t nfacct_option_get_tsize(struct nfacct_options *options,
+			enum nfacct_option_type type);
+
 void parse_nfacct_name(char *buf, const char *name);
 
 struct nlmsghdr;
@@ -45,10 +114,16 @@ int nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct
 /* print only the bytes and name columns */
 #define NFACCT_SNPRINTF_F_BONLY		(1 << 3)
 
+/* print numbers only (formatted), useful for determining max column width */
+#define NFACCT_SNPRINTF_F_NUMONLY	(1 << 4)
+
 #define NFACCT_SNPRINTF_T_PLAIN 0
 #define NFACCT_SNPRINTF_T_XML 1
 
 int nfacct_snprintf(char *buf, size_t size, struct nfacct *nfacct, uint16_t type, uint16_t flags);
+int nfacct_snprintf_with_options(char *buf, size_t size, struct nfacct *nfacct,
+				uint16_t type, uint16_t flags,
+				struct nfacct_options *options);
 
 #ifdef __cplusplus
 } /* extern "C" */
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index 23f7616..440bc0b 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -13,6 +13,7 @@
 #include <endian.h>
 #include <stdlib.h>
 #include <string.h>
+#include <locale.h>
 #include <inttypes.h>
 
 #include <libmnl/libmnl.h>
@@ -63,6 +64,13 @@ struct nfacct {
 	uint32_t	bitset;
 };
 
+struct nfacct_options {
+	uint16_t	fmt;
+	size_t		pcw;
+	size_t		bcw;
+	uint32_t	bitset;
+};
+
 /**
  * \defgroup nfacct Accounting object handling
  * @{
@@ -91,6 +99,28 @@ void nfacct_free(struct nfacct *nfacct)
 EXPORT_SYMBOL(nfacct_free);
 
 /**
+ * nfacct_options_alloc - allocate a new accounting options object
+ *
+ * In case of success, this function returns a valid pointer, otherwise NULL
+ * s returned and errno is appropriately set.
+ */
+struct nfacct_options *nfacct_options_alloc(void)
+{
+	return calloc(1, sizeof(struct nfacct_options));
+}
+EXPORT_SYMBOL(nfacct_options_alloc);
+
+/**
+ * nfacct_options_free - release one accounting options object
+ * \param nfacct pointer to the accounting options object
+ */
+void nfacct_options_free(struct nfacct_options *options)
+{
+	free(options);
+}
+EXPORT_SYMBOL(nfacct_options_free);
+
+/**
  * nfacct_attr_set - set one attribute of the accounting object
  * \param nfacct pointer to the accounting object
  * \param type attribute type you want to set
@@ -228,18 +258,294 @@ uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type)
 }
 EXPORT_SYMBOL(nfacct_attr_get_u64);
 
+/**
+ * nfacct_option_set - set one option of the accounting options object
+ * \param options pointer to the accounting options object
+ * \param type option type you want to set
+ * \param data pointer to data that will be used to set this option
+ */
+void
+nfacct_option_set(struct nfacct_options *options,
+		enum nfacct_option_type type,
+		const void *data)
+{
+	switch(type) {
+	case NFACCT_OPT_FMT:
+		options->fmt = *((uint16_t *) data);
+		options->bitset |= (1 << NFACCT_OPT_FMT);
+		break;
+	case NFACCT_OPT_PCW:
+		options->pcw = *((size_t *) data);
+		options->bitset |= (1 << NFACCT_OPT_PCW);
+		break;
+	case NFACCT_OPT_BCW:
+		options->bcw = *((size_t *) data);
+		options->bitset |= (1 << NFACCT_OPT_BCW);
+		break;
+	}
+}
+EXPORT_SYMBOL(nfacct_option_set);
+
+/**
+ * nfacct_option_set_u16 - set one option in the accounting options object
+ * \param options pointer to the accounting options object
+ * \param type option type you want to set
+ * \param value unsigned 16-bit integer
+ */
+void
+nfacct_option_set_u16(struct nfacct_options *options,
+		      enum nfacct_option_type type,
+		      uint16_t value)
+{
+	nfacct_option_set(options, type, &value);
+}
+EXPORT_SYMBOL(nfacct_option_set_u16);
+
+/**
+ * nfacct_attr_set_tsize - set one options in the accounting options object
+ * \param options pointer to the accounting options object
+ * \param type option type you want to set
+ * \param value size_t (arch-dependent) integer
+ */
+void
+nfacct_option_set_tsize(struct nfacct_options *options,
+			enum nfacct_option_type type,
+			size_t value)
+{
+	nfacct_option_set(options, type, &value);
+}
+EXPORT_SYMBOL(nfacct_option_set_tsize);
+
+/**
+ * nfacct_option_unset - unset one option in the accounting options object
+ * \param options pointer to the accounting options object
+ * \param type option type you want to unset
+ */
+void
+nfacct_option_unset(struct nfacct_options *options,
+		    enum nfacct_option_type type)
+{
+	switch(type) {
+	case NFACCT_OPT_FMT:
+		options->bitset &= ~(1 << NFACCT_OPT_FMT);
+		break;
+	case NFACCT_OPT_PCW:
+		options->bitset &= ~(1 << NFACCT_OPT_PCW);
+		break;
+	case NFACCT_OPT_BCW:
+		options->bitset &= ~(1 << NFACCT_OPT_BCW);
+		break;
+	}
+}
+EXPORT_SYMBOL(nfacct_option_unset);
+
+/**
+ * nfacct_option_get - get one option from the accounting options object
+ * \param options pointer to the accounting options object
+ * \param type option type you want to get
+ *
+ * This function returns a valid pointer to the option data. If a
+ * unsupported option is used, this returns NULL.
+ */
+const void *nfacct_option_get(struct nfacct_options *options,
+			      enum nfacct_option_type type)
+{
+	const void *ret = NULL;
+
+	switch(type) {
+	case NFACCT_OPT_FMT:
+		if (options->bitset & (1 << NFACCT_OPT_FMT))
+			ret = &options->fmt;
+		break;
+	case NFACCT_OPT_PCW:
+		if (options->bitset & (1 << NFACCT_OPT_PCW))
+			ret = &options->pcw;
+		break;
+	case NFACCT_OPT_BCW:
+		if (options->bitset & (1 << NFACCT_OPT_BCW))
+			ret = &options->bcw;
+		break;
+	}
+	return ret;
+}
+EXPORT_SYMBOL(nfacct_option_get);
+
+/**
+ * nfacct_option_get_u16 - get one option from the accounting options object
+ * \param options pointer to the accounting options object
+ * \param type option type you want to get
+ *
+ * This function returns a unsigned 16-bits integer. If the option is
+ * unsupported, this returns NULL.
+ */
+uint16_t nfacct_option_get_u16(struct nfacct_options *options,
+				enum nfacct_option_type type)
+{
+	const void *ret = nfacct_option_get(options, type);
+	return ret ? *((uint16_t *)ret) : 0;
+}
+EXPORT_SYMBOL(nfacct_option_get_u16);
+
+/**
+ * nfacct_attr_get_tsize - get one option from the accounting options object
+ * \param options pointer to the accounting options object
+ * \param type option type you want to get
+ *
+ * This function returns a size_t (arch-dependent) integer. If the option is
+ * unsupported, this returns NULL.
+ */
+size_t nfacct_option_get_tsize(struct nfacct_options *options,
+				enum nfacct_option_type type)
+{
+	const void *ret = nfacct_option_get(options, type);
+	return ret ? *((size_t *)ret) : 0;
+}
+EXPORT_SYMBOL(nfacct_option_get_tsize);
+
+#define KiB ((uint64_t)   1 << 10)
+#define MiB ((uint64_t) KiB << 10)
+#define GiB ((uint64_t) MiB << 10)
+#define TiB ((uint64_t) GiB << 10)
+#define PiB ((uint64_t) TiB << 10)
+#define EiB ((uint64_t) PiB << 10)
+#define KB ((uint64_t)  1*1000)
+#define MB ((uint64_t) KB*1000)
+#define GB ((uint64_t) MB*1000)
+#define TB ((uint64_t) GB*1000)
+#define PB ((uint64_t) TB*1000)
+#define EB ((uint64_t) PB*1000)
+
+#define NFACCT_STR_PLAIN_NUMONLY	"%s %s"
 #define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s pkts=%"PRIu64 \
 					" bytes=%"PRIu64
-#define NFACCT_STR_PLAIN		"{ pkts = %.20"PRIu64", " \
-					"bytes = %.20"PRIu64" } = %s"
-#define NFACCT_STR_PLAIN_BONLY		"{ bytes = %.20"PRIu64 " } = %s"
+#define NFACCT_STR_PLAIN_NEW		"[ pkts = %%%zus" \
+					" bytes = %%%zus ] = %%s"
+#define NFACCT_STR_PLAIN		"{ pkts = %%%zus, " \
+					"bytes = %%%zus } = %%s;"
+#define NFACCT_STR_PLAIN_BONLY		"[ bytes = %%%zus ] = %%s"
 #define NFACCT_XML_NAME			"<name>%s</name>"
-#define NFACCT_XML_PKTS			"<pkts>%.20"PRIu64"</pkts>"
-#define NFACCT_XML_BYTES		"<bytes>%.20"PRIu64"</bytes>"
+#define NFACCT_XML_PKTS			"<pkts fmt=\"%s\">%s</pkts>"
+#define NFACCT_XML_BYTES		"<bytes fmt=\"%s\">%s</bytes>"
 #define NFACCT_STR_XML_BONLY		"<obj>"	NFACCT_XML_NAME \
 					NFACCT_XML_BYTES
+#define NFACCT_STR_XML_COMPAT		"<obj><name>%s</name>"	\
+					"<pkts>%s</pkts>"	\
+					"<bytes>%s</bytes>"
 #define NFACCT_STR_XML			"<obj>"	NFACCT_XML_NAME \
 					NFACCT_XML_PKTS NFACCT_XML_BYTES
+#define NFACCT_STR_DEFAULT		"%020.0f%s"
+#define NFACCT_STR_NONE			"%.0f%s"
+#define NFACCT_STR_TRIPLETS		"%'.0f%s"
+#define NFACCT_STR_SI_IEC		"%'.3f%s"
+
+#define NFACCT_NUM_DEFAULT		{ .value = 0., .str = "" }
+
+struct nfacct_number {
+	float value;
+	char str[30];
+};
+
+struct nfacct_num {
+	uint64_t num;
+	char name[4];
+};
+
+static struct nfacct_num nfacct_num_keys[] = {
+	[NFACCT_FMT_DEFAULT] 		= { .num = 1, .name = "" },
+	[NFACCT_FMT_NONE] 		= { .num = 1, .name = "" },
+	[NFACCT_FMT_TRIPLETS] 		= { .num = 1, .name = "" },
+	[NFACCT_FMT_IEC] 		= { .num = 1, .name = "" },
+	[NFACCT_FMT_IEC_KIBIBYTE] 	= { .num = KiB,	.name = "KiB" },
+	[NFACCT_FMT_IEC_MEBIBYTE] 	= { .num = MiB,	.name = "MiB" },
+	[NFACCT_FMT_IEC_GIBIBYTE] 	= { .num = GiB,	.name = "GiB" },
+	[NFACCT_FMT_IEC_TEBIBYTE] 	= { .num = TiB,	.name = "TiB" },
+	[NFACCT_FMT_IEC_PEBIBYTE] 	= { .num = PiB,	.name = "PiB" },
+	[NFACCT_FMT_IEC_EXBIBYTE] 	= { .num = EiB,	.name = "EiB" },
+	[NFACCT_FMT_SI] 		= { .num = 1, .name = "" },
+	[NFACCT_FMT_SI_KILOBYTE] 	= { .num = KB, .name = "KB" },
+	[NFACCT_FMT_SI_MEGABYTE] 	= { .num = MB, .name = "MB" },
+	[NFACCT_FMT_SI_GIGABYTE] 	= { .num = GB, .name = "GB" },
+	[NFACCT_FMT_SI_TERABYTE] 	= { .num = TB, .name = "TB" },
+	[NFACCT_FMT_SI_PETABYTE] 	= { .num = PB, .name = "PB" },
+	[NFACCT_FMT_SI_EXABYTE] 	= { .num = EB, .name = "EB" },
+};
+
+#define NFACCT_SET_RET(x)	nf->value /= nfacct_num_keys[x].num; \
+				name = nfacct_num_keys[x].name;
+#define NFACCT_SET_STR_FMT(x) 	NFACCT_STR_##x
+#define NFACCT_GET_FMT(x)	nfacct_fmt_keys[x]
+#define NFACCT_SET_RET_FMT(x)	snprintf(nf->str,sizeof(nf->str), \
+				NFACCT_SET_STR_FMT(x),nf->value, name)
+
+static void
+format_number(struct nfacct_number *nf, const uint64_t val,
+		const enum nfacct_format fmt)
+{
+	nf->value = (float) val;
+	char *name = "";
+	switch (fmt) {	
+	case NFACCT_FMT_IEC:
+		if (nf->value >= EiB) {
+			NFACCT_SET_RET(NFACCT_FMT_IEC_EXBIBYTE);
+	        } else if (nf->value >= PiB) {
+			NFACCT_SET_RET(NFACCT_FMT_IEC_PEBIBYTE);
+	        } else if (nf->value >= TiB) {
+			NFACCT_SET_RET(NFACCT_FMT_IEC_TEBIBYTE);
+	        } else if (nf->value >= GiB) {
+			NFACCT_SET_RET(NFACCT_FMT_IEC_GIBIBYTE);
+	        } else if (nf->value >= MiB) {
+			NFACCT_SET_RET(NFACCT_FMT_IEC_MEBIBYTE);
+	        } else if (nf->value >= KiB) {
+			NFACCT_SET_RET(NFACCT_FMT_IEC_KIBIBYTE);
+		}
+		NFACCT_SET_RET_FMT(SI_IEC);
+		break;
+	case NFACCT_FMT_SI:
+		if (nf->value >= EB) {
+			NFACCT_SET_RET(NFACCT_FMT_SI_EXABYTE);
+	        } else if (nf->value >= PB) {
+			NFACCT_SET_RET(NFACCT_FMT_SI_PETABYTE);
+	        } else if (nf->value >= TB) {
+			NFACCT_SET_RET(NFACCT_FMT_SI_TERABYTE);
+	        } else if (nf->value >= GB) {
+			NFACCT_SET_RET(NFACCT_FMT_SI_GIGABYTE);
+	        } else if (nf->value >= MB) {
+			NFACCT_SET_RET(NFACCT_FMT_SI_MEGABYTE);
+	        } else if (nf->value >= KB) {
+			NFACCT_SET_RET(NFACCT_FMT_SI_KILOBYTE);
+		}
+		NFACCT_SET_RET_FMT(SI_IEC);
+		break;
+	case NFACCT_FMT_IEC_EXBIBYTE:
+	case NFACCT_FMT_IEC_PEBIBYTE:
+	case NFACCT_FMT_IEC_TEBIBYTE:
+	case NFACCT_FMT_IEC_GIBIBYTE:
+	case NFACCT_FMT_IEC_MEBIBYTE:
+	case NFACCT_FMT_IEC_KIBIBYTE:
+	case NFACCT_FMT_SI_EXABYTE:
+	case NFACCT_FMT_SI_PETABYTE:
+	case NFACCT_FMT_SI_TERABYTE:
+	case NFACCT_FMT_SI_GIGABYTE:
+	case NFACCT_FMT_SI_MEGABYTE:
+	case NFACCT_FMT_SI_KILOBYTE:
+		NFACCT_SET_RET(fmt);
+		NFACCT_SET_RET_FMT(SI_IEC);
+		break;
+	case NFACCT_FMT_DEFAULT:
+		NFACCT_SET_RET(NFACCT_FMT_DEFAULT);
+		NFACCT_SET_RET_FMT(DEFAULT);
+		break;
+	case NFACCT_FMT_NONE:
+		NFACCT_SET_RET(NFACCT_FMT_NONE);
+		NFACCT_SET_RET_FMT(NONE);
+		break;
+	case NFACCT_FMT_TRIPLETS:
+		NFACCT_SET_RET(NFACCT_FMT_TRIPLETS);
+		NFACCT_SET_RET_FMT(TRIPLETS);
+	default:
+		break;
+	}
+}
 
 void
 parse_nfacct_name(char *buf, const char *name)
@@ -324,34 +630,130 @@ void parse_nfacct_name_xml(char *buf, const char *name)
 	}
 }
 
+#define NFACCT_DEFAULT_LOCALE "en_GB"
+
+static void init_locale(void) {
+	char *lang;
+	char *env = "LANG";
+	lang = getenv(env);
+	setlocale(LC_ALL,(lang == NULL ? NFACCT_DEFAULT_LOCALE : lang));
+}
+
+/* fmt field bit definitions */
+#define _nfacct_offset_fmt	0
+#define _nfacct_offset_bytes	_nfacct_offset_fmt
+#define _nfacct_offset_pkts	8
+#define _nfacct_bitsize_fmt	0xffff
+#define _nfacct_bitsize_bytes	0xff
+#define _nfacct_bitsize_pkts	_nfacct_bitsize_bytes
+
+/* internal fmt help functions */
+#define _nfacct_get_mask(x)	((uint16_t)_nfacct_bitsize_##x \
+				<< _nfacct_offset_##x)
+#define _nfacct_get_value(x,f)	(((uint16_t)x & \
+				_nfacct_get_mask(f)) >> _nfacct_offset_##f)
+
+/* fmt help functions */
+#define nfacct_get_fmt(x)		_nfacct_get_value(x,fmt)
+#define nfacct_get_bytes_fmt(x)		_nfacct_get_value(x,bytes)
+#define nfacct_get_pkt_fmt(x)		_nfacct_get_value(x,pkts)
+
 static int
 nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
-		      uint16_t flags)
+		      uint16_t flags, struct nfacct_options *options)
 {
-	int ret;
+	int ret = 0;
+	bool compat = (options == NULL);
+	uint16_t fmt;
+	uint64_t pkts = 0, bytes = 0;
 	char nfacct_name[NFACCT_NAME_MAX * 2 + 4];
+	char fmt_str[sizeof(NFACCT_STR_PLAIN_NEW) +
+		     sizeof(NFACCT_STR_DEFAULT) * 5 + 10];
+	struct nfacct_number pn = NFACCT_NUM_DEFAULT,
+			     bn = NFACCT_NUM_DEFAULT;
 
-	parse_nfacct_name(nfacct_name,
-			  nfacct_attr_get_str(nfacct, NFACCT_ATTR_NAME));
 	if (flags & NFACCT_SNPRINTF_F_FULL) {
 		/* default: print pkts + bytes + name */
-		ret = snprintf(buf, rem, NFACCT_STR_PLAIN,
-			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS),
-			nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
-			nfacct_name);
+		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		parse_nfacct_name(nfacct_name,
+				  nfacct_attr_get_str(nfacct,
+						      NFACCT_ATTR_NAME));
+
+		if (compat) {
+			fmt = NFACCT_FMT_MAX;
+			snprintf(fmt_str, sizeof(fmt_str), NFACCT_STR_PLAIN,
+				 (size_t)0, (size_t)0);
+		} else {
+			fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
+			snprintf(fmt_str, sizeof(fmt_str),
+				NFACCT_STR_PLAIN_NEW,
+				 nfacct_option_get_tsize(options,
+							 NFACCT_OPT_PCW),
+				 nfacct_option_get_tsize(options,
+							 NFACCT_OPT_BCW));
+		}
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = NFACCT_FMT_DEFAULT;
+
+		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
+			init_locale();
+
+		format_number(&pn, pkts, nfacct_get_pkt_fmt(fmt));
+		format_number(&bn, bytes, nfacct_get_bytes_fmt(fmt));
+
+		ret = snprintf(buf, rem, fmt_str, pn.str, bn.str,
+				nfacct_name);
 	} else if (flags & NFACCT_SNPRINTF_F_SAVE) {
 		/* save: format useful for 'restore' */
+		parse_nfacct_name(nfacct_name,
+				  nfacct_attr_get_str(nfacct,
+						      NFACCT_ATTR_NAME));
 		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_SAVE_BASE,
 			       nfacct_name,
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_PKTS),
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BYTES));
 	} else if (flags & NFACCT_SNPRINTF_F_BONLY) {
 		/* print bytes + name only */
-		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_BONLY,
-				nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES),
-				nfacct_name);
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		parse_nfacct_name(nfacct_name,
+				nfacct_attr_get_str(nfacct,
+						    NFACCT_ATTR_NAME));
+		fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = NFACCT_FMT_DEFAULT;
+
+		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
+			init_locale();
+
+		format_number(&bn, bytes, nfacct_get_bytes_fmt(fmt));
+		snprintf(fmt_str, sizeof(fmt_str), NFACCT_STR_PLAIN_BONLY,
+			 nfacct_option_get_tsize(options,NFACCT_OPT_BCW));
+		ret = snprintf(buf, rem, fmt_str, bn.str, nfacct_name);
+	} else if (flags & NFACCT_SNPRINTF_F_NUMONLY) {
+		/* numbers only: to determine the maximum column width */
+		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = NFACCT_FMT_DEFAULT;
+
+		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
+			init_locale();
+
+		format_number(&pn, pkts, nfacct_get_pkt_fmt(fmt));
+		format_number(&bn, bytes, nfacct_get_bytes_fmt(fmt));
+
+		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_NUMONLY,
+				pn.str, bn.str);
 	} else {
 		/* print out name only */
+		parse_nfacct_name(nfacct_name,
+				  nfacct_attr_get_str(nfacct,
+						      NFACCT_ATTR_NAME));
 		ret = snprintf(buf, rem, "%s\n", nfacct_name);
 	}
 
@@ -398,28 +800,58 @@ nfacct_snprintf_xml_localtime(char *buf, unsigned int rem, const struct tm *tm)
 
 static int
 nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
-		    uint16_t flags)
+		    uint16_t flags, struct nfacct_options *options)
 {
 	int ret = 0;
+	bool compat = (options == NULL);
 	unsigned int size = 0, offset = 0;
+	uint16_t fmt;
+	uint64_t pkts = 0, bytes = 0;
 	char nfacct_name[NFACCT_NAME_MAX * 6 + 1];
+	struct nfacct_number pn = NFACCT_NUM_DEFAULT,
+			     bn = NFACCT_NUM_DEFAULT;
 
 	parse_nfacct_name_xml(nfacct_name,
 				nfacct_attr_get_str(nfacct,
 						    NFACCT_ATTR_NAME));
+	if (compat) {
+		fmt = NFACCT_FMT_MAX;
+	} else {
+		fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
+	}
 	if (flags & NFACCT_SNPRINTF_F_BONLY) {
 		/* print name + bytes only */
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = NFACCT_FMT_DEFAULT;
+
+		format_number(&bn, bytes, NFACCT_FMT_NONE);
 		ret = snprintf(buf, rem, NFACCT_STR_XML_BONLY, nfacct_name,
-				nfacct_attr_get_u64(nfacct,
-						    NFACCT_ATTR_BYTES));
+				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
+				bn.str);
 		BUFFER_SIZE(ret, size, rem, offset);
 	} else {
 		/* default/everything else: print name + pkts + bytes */
-		ret = snprintf(buf, rem, NFACCT_STR_XML, nfacct_name,
-				nfacct_attr_get_u64(nfacct,
-						    NFACCT_ATTR_BYTES),
-				nfacct_attr_get_u64(nfacct,
-						    NFACCT_ATTR_PKTS));
+		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = NFACCT_FMT_DEFAULT;
+
+		format_number(&pn, pkts, NFACCT_FMT_NONE);
+		format_number(&bn, bytes, NFACCT_FMT_NONE);
+
+		if (compat) {
+			ret = snprintf(buf, rem, NFACCT_STR_XML_COMPAT,
+					nfacct_name, pn.str, bn.str);
+		} else {
+			ret = snprintf(buf, rem, NFACCT_STR_XML, nfacct_name,
+				NFACCT_GET_FMT(nfacct_get_pkt_fmt(fmt)),
+				pn.str,
+				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
+				bn.str);
+		}
 		BUFFER_SIZE(ret, size, rem, offset);
 	}
 
@@ -443,27 +875,30 @@ err:
 }
 
 /**
- * nfacct_snprintf - print accounting object into one buffer
+ * nfacct_snprintf_with_options - print accounting object into one buffer
  * \param buf: pointer to buffer that is used to print the object
  * \param size: size of the buffer (or remaining room in it).
  * \param nfacct: pointer to a valid accounting object.
  * \param type: format output type, NFACCT_SNPRINTF_T_[PLAIN|XML]
  * \param flags: output flags (NFACCT_SNPRINTF_F_FULL).
+ * \param options: nfacct options structure.
  *
  * This function returns -1 in case that some mandatory attributes are
  * missing. On sucess, it returns 0.
  */
-int nfacct_snprintf(char *buf, size_t size, struct nfacct *nfacct,
-		    uint16_t type, uint16_t flags)
+int nfacct_snprintf_with_options(char *buf, size_t size,
+				struct nfacct *nfacct,
+				uint16_t type, uint16_t flags,
+				struct nfacct_options *options)
 {
 	int ret = 0;
 
 	switch(type) {
 	case NFACCT_SNPRINTF_T_PLAIN:
-		ret = nfacct_snprintf_plain(buf, size, nfacct, flags);
+		ret = nfacct_snprintf_plain(buf,size,nfacct,flags,options);
 		break;
 	case NFACCT_SNPRINTF_T_XML:
-		ret = nfacct_snprintf_xml(buf, size, nfacct, flags);
+		ret = nfacct_snprintf_xml(buf,size,nfacct,flags,options);
 		break;
 	default:
 		ret = -1;
@@ -471,6 +906,25 @@ int nfacct_snprintf(char *buf, size_t size, struct nfacct *nfacct,
 	}
 	return ret;
 }
+EXPORT_SYMBOL(nfacct_snprintf_with_options);
+
+/**
+ * nfacct_snprintf - print accounting object into one buffer
+ * \param buf: pointer to buffer that is used to print the object
+ * \param size: size of the buffer (or remaining room in it).
+ * \param nfacct: pointer to a valid accounting object.
+ * \param type: format output type, NFACCT_SNPRINTF_T_[PLAIN|XML]
+ * \param flags: output flags (NFACCT_SNPRINTF_F_FULL).
+ *
+ * This function returns -1 in case that some mandatory attributes are
+ * missing. On sucess, it returns 0.
+ */
+int nfacct_snprintf(char *buf, size_t size, struct nfacct *nfacct,
+		    uint16_t type, uint16_t flags)
+{
+	return nfacct_snprintf_with_options(buf, size, nfacct,
+					    type, flags, NULL);
+}
 EXPORT_SYMBOL(nfacct_snprintf);
 
 /**
diff --git a/src/libnetfilter_acct.map b/src/libnetfilter_acct.map
index f12bc8e..ded60a9 100644
--- a/src/libnetfilter_acct.map
+++ b/src/libnetfilter_acct.map
@@ -21,5 +21,15 @@ local: *;
 
 LIBNETFILTER_ACCT_1.1 {
   nfacct_snprintf;
+  nfacct_options_alloc;
+  nfacct_options_free;
+  nfacct_option_set;
+  nfacct_option_set_u16;
+  nfacct_option_set_tsize;
+  nfacct_option_unset;
+  nfacct_option_get;
+  nfacct_option_get_u16;
+  nfacct_option_get_tsize;
   parse_nfacct_name;
+  nfacct_snprintf_with_options;
 } LIBNETFILTER_ACCT_1.0;
-- 
1.8.3.1

Re: [PATCH v3 libnetfilter_acct 11/29] add variable width and on-the-fly formatting

[Pablo Neira Ayuso]

On Wed, Jul 10, 2013 at 07:25:09PM +0100, Michael Zintakis wrote:
> * add a separate nfacct_options struct with its get/set functions, allowing
> userspace programs, like nfacct, to specify the width of the columns which
> are to be printed, so that they are no longer hard-coded into the printing
> template itself. This allows column width to be "adjusted", depending on the
> value of each property;
> 
> * add a separate NFACCT_SNPRINTF_F_NUMONLY template, which prints the
> numbers-only properties of the accounting objects (formatted - see below) -
> useful to the nfnetlink callback function in order to determine the maximum
> column width, which needs to be used to print each column.
> 
> * add a new nfacct_snprintf_with_options function enhancing the existing API
> to enable "options" to be specified during printing properties of nfacct
> objects.
> 
> * allow on-the-fly formatting, using over 14 different formats when printing
> accounting objects. This formatting, if specified, applies to *all*
> accounting objects.

This patch adds many things in one single patch that need to be split
in many of them.

> Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
> ---
>  configure.ac                                  |   2 +-
>  include/libnetfilter_acct/libnetfilter_acct.h |  75 ++++
>  src/libnetfilter_acct.c                       | 512 ++++++++++++++++++++++++--
>  src/libnetfilter_acct.map                     |  10 +
>  4 files changed, 569 insertions(+), 30 deletions(-)
>
[...]
> +void nfacct_option_set(struct nfacct_options *options,
> +			enum nfacct_option_type type, const void *data);
> +void nfacct_option_set_u16(struct nfacct_options *options,
> +			enum nfacct_option_type type, uint16_t value);
> +void nfacct_option_set_tsize(struct nfacct_options *options,
> +			enum nfacct_option_type type, size_t value);
> +void nfacct_option_unset(struct nfacct_options *options,
> +			enum nfacct_option_type type);
> +const void *nfacct_option_get(struct nfacct_options *options,
> +			enum nfacct_option_type type);
> +uint16_t nfacct_option_get_u16(struct nfacct_options *options,
> +			enum nfacct_option_type type);
> +size_t nfacct_option_get_tsize(struct nfacct_options *options,
> +			enum nfacct_option_type type);

You can get rid of all these if you use flags.

[PATCH v3 nfacct 12/29] add variable width and on-the-fly number formatting

[Michael Zintakis]

* re-implement nfacct_cb callback function to make use of the new *_NUMONLY
linfnetfilter_acct template, allowing the maximum width of each column to be
determined in advance prior to printing the nfacct object properties. This
involves using a linked-list to store all nfacct objects temporarily.

* add a new "format" option to the "list" and "get" commands, allowing custom
number formatting to be applied to each of the nfacct object properties. More
than 14 different formats are available for use.

* add nfacct_tools.c file where all help functions are to be stored, thus avoid
cluttering the existing code unnecessarily.

* do some minor code re-factoring tweaks;

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/Makefile.am    |   2 +-
 src/nfacct.c       | 279 +++++++++++++++++++++++++++++++++++++++--------------
 src/nfacct_list.h  |  91 +++++++++++++++++
 src/nfacct_utils.c | 242 ++++++++++++++++++++++++++++++++++++++++++++++
 src/nfacct_utils.h |  30 ++++++
 5 files changed, 572 insertions(+), 72 deletions(-)
 create mode 100644 src/nfacct_list.h
 create mode 100644 src/nfacct_utils.c
 create mode 100644 src/nfacct_utils.h

diff --git a/src/Makefile.am b/src/Makefile.am
index 28822fe..133ce61 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -2,6 +2,6 @@ include $(top_srcdir)/Make_global.am
 
 sbin_PROGRAMS = nfacct
 
-nfacct_SOURCES = nfacct.c
+nfacct_SOURCES = nfacct.c nfacct_utils.c
 
 nfacct_LDADD = ${LIBMNL_LIBS} ${LIBNETFILTER_ACCT_LIBS}
diff --git a/src/nfacct.c b/src/nfacct.c
index 7808ebb..aebb6ac 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -21,7 +21,52 @@
 #include <errno.h>
 
 #include <libmnl/libmnl.h>
-#include <libnetfilter_acct/libnetfilter_acct.h>
+#include "nfacct_list.h"
+#include "nfacct_utils.h"
+
+struct nfa {
+	struct nfacct *nfacct;
+	struct nfacct_list_head head;
+};
+
+/* store nfacct objects information */
+static NFACCT_LIST_HEAD(nfa_list);
+
+static struct nfa *nfa_alloc(void)
+{
+	return calloc(1, sizeof(struct nfa));
+}
+
+static void nfa_free(struct nfa *nfa)
+{
+	if (nfa)
+		free(nfa);
+}
+
+static void free_nfa_list(void) {
+	struct nfa *nfa, *nnfa;
+	nfacct_list_for_each_entry_safe(nfa, nnfa, &nfa_list, head) {
+		if (nfa->nfacct)
+			nfacct_free(nfa->nfacct);
+
+		nfa_free(nfa);
+	}
+}
+
+/* xml header & footer strings */
+#define NFACCT_XML_HEADER	"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n" \
+				"<nfacct>\n"
+#define NFACCT_XML_FOOTER	"</nfacct>\n"
+
+/*
+ * maximum total of columns to be shown, except the
+ * "name" column as that is not width-dependent:
+ * "pkts bytes"
+ */
+#define NFACCT_MAX_COLUMNS 2
+
+/* stores nfacct options for snprintf_* and nfacct_cb functions */
+static struct nfacct_options *options;
 
 static int nfacct_cmd_list(int argc, char *argv[]);
 static int nfacct_cmd_add(int argc, char *argv[]);
@@ -32,21 +77,6 @@ static int nfacct_cmd_version(int argc, char *argv[]);
 static int nfacct_cmd_help(int argc, char *argv[]);
 static int nfacct_cmd_restore(int argc, char *argv[]);
 
-/* Matches two strings, including partial matches */
-static int nfacct_matches(const char *cmd, const char *pattern)
-{
-	size_t len;
-
-	if (cmd == NULL || pattern == NULL)
-		return 0;
-
-	len = strlen(cmd);
-	if (len == 0 || len > strlen(pattern))
-		return 0;
-
-	return (strncmp(cmd, pattern, len) == 0);
-}
-
 /* main command 'menu' */
 static const struct cmd {
 	const char *cmd;
@@ -98,6 +128,12 @@ static void nfacct_perror(const char *msg)
 #define NFACCT_RET_ERR(x)	nfacct_perror(x); \
 				goto err;
 #define NFACCT_RET_ARG_ERR()	NFACCT_RET_ERR("unknown argument")
+#define NFACCT_GET_NEXT_ARG() 	do {					\
+					argv++;				\
+					if (--argc <= 0) {		\
+						NFACCT_RET_ARG_ERR();	\
+					}				\
+				} while(0)
 #define NFACCT_NEXT_ARG() 	do {		\
 					argv++;	\
 					argc--;	\
@@ -115,39 +151,65 @@ int main(int argc, char *argv[])
 	usage(argv);
 }
 
-static bool xml_header = false;
-
 static int nfacct_cb(const struct nlmsghdr *nlh, void *data)
 {
-	struct nfacct *nfacct;
-	char buf[4096];
-	bool *xml = (bool *)data;
+	struct nfa *nfa = NULL;
+	size_t bl[NFACCT_MAX_COLUMNS];
+	char buf[MAX_TOKEN_SIZE];
+	int i;
+	bool *ignore_col_width = (bool *)data;
+	static const enum nfacct_option_type o_num[NFACCT_MAX_COLUMNS] =
+				{ NFACCT_OPT_PCW,
+				  NFACCT_OPT_BCW,
+				};
+
+	if (ignore_col_width == NULL || options == NULL) {
+		NFACCT_RET_ERR("nfacct_cb_data_and_options");
+	}
+
+	nfa = nfa_alloc();
+	if (nfa == NULL) {
+		NFACCT_RET_ERR("OOM");
+	}
 
-	nfacct = nfacct_alloc();
-	if (nfacct == NULL) {
+	nfa->nfacct = nfacct_alloc();
+	if (nfa->nfacct == NULL) {
 		nfacct_perror("OOM");
-		goto err;
+		goto err_nfa_free;
 	}
 
-	if (nfacct_nlmsg_parse_payload(nlh, nfacct) < 0) {
+	if (nfacct_nlmsg_parse_payload(nlh, nfa->nfacct) < 0) {
 		nfacct_perror("nfacct_parse_nl_msg");
-		goto err_free;
+		goto err_nfacct_free;
 	}
 
-	if (*xml && !xml_header) {
-		printf("<?xml version=\"1.0\" encoding=\"utf-8\"?>\n"
-			"<nfacct>\n");
-		xml_header = true;
+	if (!*ignore_col_width) {
+		nfacct_snprintf_with_options(buf, ARRAY_SIZE(buf),
+				nfa->nfacct,
+			       NFACCT_SNPRINTF_T_PLAIN,
+				NFACCT_SNPRINTF_F_NUMONLY, options);
+		i = nfacct_parse_tokens_length(buf, " ", ARRAY_SIZE(bl),
+					       false, bl);
+		if (i < NFACCT_MAX_COLUMNS) {
+			nfacct_perror("nfacct_parse_tokens_length");
+			goto err_nfacct_free;
+		}
+		for (i = 0; i < NFACCT_MAX_COLUMNS; i++) {
+			if (bl[(i)] > nfacct_option_get_tsize(options,
+							       o_num[(i)]))
+				nfacct_option_set_tsize(options, o_num[(i)],
+							bl[i]);
+		}
 	}
+	nfacct_list_add_tail(&nfa->head, &nfa_list);
+	return MNL_CB_OK;
 
-	nfacct_snprintf(buf, sizeof(buf), nfacct,
-			*xml ? NFACCT_SNPRINTF_T_XML :
-			       NFACCT_SNPRINTF_T_PLAIN,
-			NFACCT_SNPRINTF_F_FULL);
-	printf("%s\n", buf);
+err_nfacct_free:
+	nfacct_free(nfa->nfacct);
+
+err_nfa_free:
+	nfa_free(nfa);
 
-err_free:
-	nfacct_free(nfacct);
 err:
 	return MNL_CB_OK;
 }
@@ -155,12 +217,15 @@ err:
 static int nfacct_cmd_list(int argc, char *argv[])
 {
 	bool nfnl_msg = false, xml = false;
+	bool b_fmt = false;
 	struct mnl_socket *nl;
-	char buf[MNL_SOCKET_BUFFER_SIZE];
+	char buf[70000];
 	struct nlmsghdr *nlh;
 	unsigned int seq, portid;
-	int ret;
+	int ret = -1, i;
+	struct nfa *nfa = NULL;
 	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
+	uint16_t fmt = NFACCT_FMT_MAX;
 
 	while (argc > 0) {
 		if (!nfnl_msg && nfacct_matches(argv[0],"reset")) {
@@ -168,9 +233,16 @@ static int nfacct_cmd_list(int argc, char *argv[])
 			nfnl_msg = true;
 		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
+		} else if (!b_fmt && (nfacct_matches(argv[0],"format") ||
+			   nfacct_matches(argv[0],"fmt"))) {
+			NFACCT_GET_NEXT_ARG();
+			fmt = nfacct_parse_format_options(argv[0]);
+			if (fmt == NFACCT_FMT_MAX) {
+				NFACCT_RET_ARG_ERR();
+			}
+			b_fmt = true;
 		} else {
-			nfacct_perror("unknown argument");
-			return -1;
+			NFACCT_RET_ARG_ERR();
 		}
 		argc--; argv++;
 	}
@@ -180,38 +252,60 @@ static int nfacct_cmd_list(int argc, char *argv[])
 
 	nl = mnl_socket_open(NETLINK_NETFILTER);
 	if (nl == NULL) {
-		nfacct_perror("mnl_socket_open");
-		return -1;
+		NFACCT_RET_ERR("mnl_socket_open");
 	}
 
 	if (mnl_socket_bind(nl, 0, MNL_SOCKET_AUTOPID) < 0) {
-		nfacct_perror("mnl_socket_bind");
-		return -1;
+		NFACCT_RET_ERR("mnl_socket_bind");
 	}
 	portid = mnl_socket_get_portid(nl);
 
 	if (mnl_socket_sendto(nl, nlh, nlh->nlmsg_len) < 0) {
-		nfacct_perror("mnl_socket_send");
-		return -1;
+		NFACCT_RET_ERR("mnl_socket_send");
 	}
 
-	ret = mnl_socket_recvfrom(nl, buf, sizeof(buf));
-	while (ret > 0) {
-		ret = mnl_cb_run(buf, ret, seq, portid, nfacct_cb, &xml);
-		if (ret <= 0)
+	options = nfacct_options_alloc();
+	if (options == NULL) {
+		NFACCT_RET_ERR("OOM");
+	}
+	nfacct_option_set_u16(options, NFACCT_OPT_FMT, fmt);
+
+	i = mnl_socket_recvfrom(nl, buf, ARRAY_SIZE(buf));
+	while (i > 0) {
+		i = mnl_cb_run(buf, i, seq, portid, nfacct_cb, &xml);
+		if (i <= 0)
 			break;
-		ret = mnl_socket_recvfrom(nl, buf, sizeof(buf));
+		i = mnl_socket_recvfrom(nl, buf, ARRAY_SIZE(buf));
 	}
-	if (ret == -1) {
+	if (i == -1) {
 		nfacct_perror("error");
-		return -1;
+		goto err_free_nfa;
 	}
 	mnl_socket_close(nl);
 
-	if (xml_header)
-		printf("</nfacct>\n");
+	if (xml)
+		printf(NFACCT_XML_HEADER);
 
-	return 0;
+	nfacct_list_for_each_entry(nfa, &nfa_list, head) {
+		nfacct_snprintf_with_options(buf, ARRAY_SIZE(buf),
+				nfa->nfacct,
+				xml ? NFACCT_SNPRINTF_T_XML :
+				      NFACCT_SNPRINTF_T_PLAIN,
+				NFACCT_SNPRINTF_F_FULL, options);
+		printf("%s\n",buf);
+	}
+
+	if (xml)
+		printf(NFACCT_XML_FOOTER);
+
+	ret = 0;
+
+err_free_nfa:
+	free_nfa_list();
+	nfacct_options_free(options);
+
+err:
+	return ret;
 }
 
 static int _nfacct_cmd_add(struct nfacct *nfacct, bool replace)
@@ -376,14 +470,17 @@ static int nfacct_cmd_delete(int argc, char *argv[])
 static int nfacct_cmd_get(int argc, char *argv[])
 {
 	bool nfnl_msg = false, xml = false;
+	bool b_fmt = false;
 	struct mnl_socket *nl;
 	char buf[MNL_SOCKET_BUFFER_SIZE];
 	struct nlmsghdr *nlh;
 	uint32_t portid, seq;
 	struct nfacct *nfacct;
 	int ret = -1, i;
+	struct nfa *nfa = NULL;
 	char *name;
 	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
+	uint16_t fmt = NFACCT_FMT_MAX;
 
 	if (argc < 1 || strlen(argv[0]) == 0) {
 		nfacct_perror("missing object name");
@@ -401,6 +498,14 @@ static int nfacct_cmd_get(int argc, char *argv[])
 			nfnl_msg = true;
 		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
+		} else if (!b_fmt && (nfacct_matches(argv[0],"format") ||
+			   nfacct_matches(argv[0],"fmt"))) {
+			NFACCT_GET_NEXT_ARG();
+			fmt = nfacct_parse_format_options(argv[0]);
+			if (fmt == NFACCT_FMT_MAX) {
+				NFACCT_RET_ARG_ERR();
+			}
+			b_fmt = true;
 		} else {
 			NFACCT_RET_ARG_ERR();
 		}
@@ -434,23 +539,46 @@ static int nfacct_cmd_get(int argc, char *argv[])
 		NFACCT_RET_ERR("mnl_socket_send");
 	}
 
-	i = mnl_socket_recvfrom(nl, buf, sizeof(buf));
+	options = nfacct_options_alloc();
+	if (options == NULL) {
+		NFACCT_RET_ERR("OOM");
+	}
+	nfacct_option_set_u16(options, NFACCT_OPT_FMT, fmt);
+
+	i = mnl_socket_recvfrom(nl, buf, ARRAY_SIZE(buf));
 	while (i > 0) {
 		i = mnl_cb_run(buf, i, seq, portid, nfacct_cb, &xml);
 		if (i <= 0)
 			break;
-		i = mnl_socket_recvfrom(nl, buf, sizeof(buf));
+		i = mnl_socket_recvfrom(nl, buf, ARRAY_SIZE(buf));
 	}
 	if (i == -1) {
-		NFACCT_RET_ERR("error");
+		nfacct_perror("error");
+		goto err_free_nfa;
 	}
 	mnl_socket_close(nl);
 
-	if (xml_header)
-		printf("</nfacct>\n");
+	if (xml)
+		printf(NFACCT_XML_HEADER);
+
+	nfacct_list_for_each_entry(nfa, &nfa_list, head) {
+		nfacct_snprintf_with_options(buf, ARRAY_SIZE(buf),
+				nfa->nfacct,
+				xml ? NFACCT_SNPRINTF_T_XML :
+				      NFACCT_SNPRINTF_T_PLAIN,
+				NFACCT_SNPRINTF_F_FULL, options);
+		printf("%s\n",buf);
+	}
+
+	if (xml)
+		printf(NFACCT_XML_FOOTER);
 
 	ret = 0;
 
+err_free_nfa:
+	free_nfa_list();
+	nfacct_options_free(options);
+
 err:
 	free(name);
 	return ret;
@@ -489,12 +617,12 @@ static int nfacct_cmd_flush(int argc, char *argv[])
 		return -1;
 	}
 
-	ret = mnl_socket_recvfrom(nl, buf, sizeof(buf));
+	ret = mnl_socket_recvfrom(nl, buf, ARRAY_SIZE(buf));
 	while (ret > 0) {
 		ret = mnl_cb_run(buf, ret, seq, portid, NULL, NULL);
 		if (ret <= 0)
 			break;
-		ret = mnl_socket_recvfrom(nl, buf, sizeof(buf));
+		ret = mnl_socket_recvfrom(nl, buf, ARRAY_SIZE(buf));
 	}
 	if (ret == -1) {
 		nfacct_perror("error");
@@ -527,14 +655,23 @@ static const char help_msg[] =
 	"infrastructure\n"
 	"Usage: nfacct command [parameters]...\n\n"
 	"Commands:\n"
-	"  list [reset]\t\t\tList the accounting object table (and reset)\n"
-	"  add object-name [replace]\tAdd new accounting object to table\n"
-	"  delete object-name\t\tDelete existing accounting object\n"
-	"  get object-name\t\tGet existing accounting object\n"
+	"  list LST_PARAMS\tList the accounting object table\n"
+	"  add NAME ADD_PARAMS\tAdd new accounting object NAME to table\n"
+	"  delete NAME\t\tDelete existing accounting object NAME\n"
+	"  get NAME GET_PARAMS\tGet and list existing accounting object NAME\n"
 	"  flush\t\t\tFlush accounting object table\n"
-	"  restore [flush] [replace]\tRestore accounting object table reading 'list' output from stdin\n"
-	"  version\t\t\tDisplay version and disclaimer\n"
-	"  help\t\t\t\tDisplay this help message\n";
+	"  restore RST_PARAMS\tRestore accounting object table from stdin\n"
+	"  version\t\tDisplay version and disclaimer\n"
+	"  help\t\t\tDisplay this help message\n\n"
+	"Parameters:\n"
+	"  LST_PARAMS := [ reset ] [ format FMT_SPEC ] [ xml ]\n"
+	"  ADD_PARAMS := [ replace ]\n"
+	"  GET_PARAMS := [ reset ] [ format FMT_SPEC ] [ xml ]\n"
+	"  RST_PARAMS := [ flush ] [ replace ]\n"
+	"  FMT_SPEC := { [FMT] | [,] | [FMT] ... }\n"
+	"  FMT := { def | raw | 3pl | iec | kib | mib | gib | tib | pib |"
+		  " eib |\n"
+	"  \t   si | kb | mb | gb | tb | pb | eb }\n";
 
 static int nfacct_cmd_help(int argc, char *argv[])
 {
diff --git a/src/nfacct_list.h b/src/nfacct_list.h
new file mode 100644
index 0000000..5ef1b9d
--- /dev/null
+++ b/src/nfacct_list.h
@@ -0,0 +1,91 @@
+/*
+ * 	Based on netlink/list.h
+ *	by Thomas Graf <tgraf@suug.ch>
+ *
+ *	This library is free software; you can redistribute it and/or
+ *	modify it under the terms of the GNU Lesser General Public
+ *	License as published by the Free Software Foundation version 2.1
+ *	of the License.
+ *
+ */
+
+#ifndef _NFACCT_LIST_H_
+#define _NFACCT_LIST_H_
+
+struct nfacct_list_head
+{
+	struct nfacct_list_head *	next;
+	struct nfacct_list_head *	prev;
+};
+
+
+static inline void __nfacct_list_add(struct nfacct_list_head *obj,
+				 struct nfacct_list_head *prev,
+				 struct nfacct_list_head *next)
+{
+	prev->next = obj;
+	obj->prev = prev;
+	next->prev = obj;
+	obj->next = next;
+}
+
+static inline void nfacct_list_add_tail(struct nfacct_list_head *obj,
+					struct nfacct_list_head *head)
+{
+	__nfacct_list_add(obj, head->prev, head);
+}
+
+static inline void nfacct_list_add_head(struct nfacct_list_head *obj,
+					struct nfacct_list_head *head)
+{
+	__nfacct_list_add(obj, head, head->next);
+}
+
+static inline void nfacct_list_del(struct nfacct_list_head *obj)
+{
+	obj->next->prev = obj->prev;
+	obj->prev->next = obj->next;
+}
+
+static inline int nfacct_list_empty(struct nfacct_list_head *head)
+{
+	return head->next == head;
+}
+
+#define nfacct_container_of(ptr, type, member) ({			\
+        typeof( ((type *)0)->member ) *__mptr = (ptr);			\
+        (type *)( (char *)__mptr - ((size_t) &((type *)0)->member));})
+
+#define nfacct_list_entry(ptr, type, member) \
+	nfacct_container_of(ptr, type, member)
+
+#define nfacct_list_at_tail(pos, head, member) \
+	((pos)->member.next == (head))
+
+#define nfacct_list_at_head(pos, head, member) \
+	((pos)->member.prev == (head))
+
+#define NFACCT_LIST_HEAD(name) \
+	struct nfacct_list_head name = { &(name), &(name) }
+
+#define nfacct_list_for_each_entry(pos, head, member)			\
+	for (pos = nfacct_list_entry((head)->next,			\
+				     typeof(*pos), member);		\
+	     &(pos)->member != (head); 					\
+	     (pos) = nfacct_list_entry((pos)->member.next,		\
+				    typeof(*(pos)), member))
+
+#define nfacct_list_for_each_entry_safe(pos, n, head, member)		\
+	for (pos = nfacct_list_entry((head)->next,			\
+				  typeof(*pos), member),		\
+		n = nfacct_list_entry(pos->member.next,			\
+				   typeof(*pos), member);		\
+	     &(pos)->member != (head); 					\
+	     pos = n, n = nfacct_list_entry(n->member.next,		\
+					 typeof(*n), member))
+
+#define nfacct_init_list_head(head) \
+	do { (head)->next = (head); (head)->prev = (head); } while (0)
+
+#endif
+
diff --git a/src/nfacct_utils.c b/src/nfacct_utils.c
new file mode 100644
index 0000000..61b2ad5
--- /dev/null
+++ b/src/nfacct_utils.c
@@ -0,0 +1,242 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published
+ * by the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#include <stdbool.h>
+#include <stdlib.h>
+#include <string.h>
+#include <errno.h>
+#include <limits.h>
+
+#include <libnetfilter_acct/libnetfilter_acct.h>
+
+#include "nfacct_utils.h"
+
+/* Matches two strings, including partial matches */
+int nfacct_matches(const char *cmd, const char *pattern)
+{
+	size_t len;
+
+	if (cmd == NULL || pattern == NULL)
+		return 0;
+
+	len = strlen(cmd);
+	if (len == 0 || len > strlen(pattern))
+		return 0;
+
+	return (strncmp(cmd, pattern, len) == 0);
+}
+
+/*
+ * Takes 'str' and breaks it in maximum of 'len' tokens, using 'sep'
+ * as separators, taking into account character escaping (\) and
+ * string quotation (e.g. "abc de"). Each token length is stored in 't_len'.
+ * The function returns the number of tokens actually processed.
+ *
+ * N.B.:
+ *	1. Character escaping is NOT translated/taken into account. In
+ *	   other words, "\t" translates to "t".
+ *	2. If skip_comment=true and if 'str' starts with 'comment',
+ *	   then the parsing is skipped and nothing is returned.
+ *	3. It is assumed that each resulting token is no more than
+ *	   MAX_TOKEN_SIZE characters.
+ *
+ */
+int nfacct_parse_tokens_length(const char *str, const char *sep, const size_t len,
+			   	const int skip_comment, size_t t_len[])
+{
+	bool quote_open = false, escaped = false;
+	static const char comment = '#';
+	size_t param_len = 0, i = 0;
+	char buf[MAX_TOKEN_SIZE], *ptr, *tmp;
+
+	if (str == NULL || strlen(str) == 0 || t_len == NULL ||
+	    sep == NULL || strlen(sep) == 0 ||
+	    (skip_comment && strchr(str, comment) != NULL &&
+	     *strchr(str, comment) == str[0]))
+		goto err;
+
+	tmp = strdup(str);
+	for (ptr = tmp; *ptr; ptr++) {
+		if (quote_open) {
+			if (escaped) {
+				if (param_len >= ARRAY_SIZE(buf))
+					goto err_free;
+				buf[param_len++] = *ptr;
+				escaped = false;
+				continue;
+			} else if (*ptr == '\\') {
+				escaped = true;
+				continue;
+			} else if (*ptr == '"') {
+				quote_open = false;
+				*ptr = *sep;
+			} else {
+				if (param_len >= ARRAY_SIZE(buf))
+					goto err_free;
+				buf[param_len++] = *ptr;
+				continue;
+			}
+		} else {
+			if (*ptr == '"') {
+				quote_open = true;
+				continue;
+			}
+		}
+		if (strchr(sep, *ptr)) {
+			if (!param_len)
+				continue;
+
+			if (!param_len || i >= len ||
+			    param_len >= ARRAY_SIZE(buf))
+				goto err_free;
+
+			buf[param_len] = '\0';
+			t_len[i] = strlen(buf);
+			i++;
+			param_len = 0;
+		} else {
+			/* regular character, copy to buffer */
+			if (param_len >= ARRAY_SIZE(buf))
+				goto err_free;
+			buf[param_len++] = *ptr;
+		}
+	}
+
+	if (escaped || quote_open || !param_len || i >= len)
+		goto err_free;
+
+	if (param_len && param_len < ARRAY_SIZE(buf)) {
+		buf[param_len] = '\0';
+		t_len[i] = strlen(buf);
+		i++;
+	}
+
+err_free:
+	free(tmp);
+err:
+	return i;
+}
+
+/* maximum length of any format options */
+#define NFACCT_MAX_FMT_LEN 3
+
+/*
+ *
+ * Parses 'str' into a pair of nfacct_format values
+ * as defined in nfacct_fmt_keys and returns either
+ * 8bits:packet_fmt, 8bits:bytes_fmt or NFACCT_FMT_MAX if incorrect
+ * format was specified.
+ *
+ * N.B.:
+ *	1. "3pl" is the equivalent of "3pl,3pl";
+ *	2. ",3pl" is the equivalent of "def,3pl";
+ *	3. "3pl," is the equivalent of "3pl,def";
+ *	4. "," is the equivalent of "def,def";
+ *
+ */
+uint16_t nfacct_parse_format_options(const char *str)
+{
+	uint16_t ret = NFACCT_FMT_MAX;
+	size_t i = 0, j = 0, param_len = 0;
+	char buf[(NFACCT_MAX_FMT_LEN + 1) * 2], *ptr, *tmp;
+	enum nfacct_format fmt[2] = { NFACCT_FMT_MAX, NFACCT_FMT_MAX };
+
+	if (str == NULL || strlen(str) > ARRAY_SIZE(buf))
+		goto err;
+
+	tmp = strdup(str);
+	for (ptr = tmp; *ptr && i < ARRAY_SIZE(fmt); ptr++) {
+		if (strchr(",", *ptr)) {
+			if (!param_len) {
+				fmt[i] = NFACCT_FMT_DEFAULT;
+				i++;
+				continue;
+			}
+			if (param_len >= ARRAY_SIZE(buf))
+				goto err_free;
+
+			buf[param_len] = '\0';
+			for (j = NFACCT_FMT_DEFAULT;
+			     j <= NFACCT_FMT_MAX &&
+			     strncmp(buf, nfacct_fmt_keys[j],
+				strlen(nfacct_fmt_keys[j])+1) != 0; j++) {;}
+			if (j >= NFACCT_FMT_MAX) {
+				/* syntax error, exit */
+				goto err_free;
+			}
+
+			fmt[i] = j;
+			i++;
+			param_len = 0;
+		} else {
+			/* regular character, copy to buffer */
+			if (param_len >= ARRAY_SIZE(buf))
+				goto err_free;
+			buf[param_len++] = *ptr;
+		}
+	}
+	if (!param_len) {
+		fmt[i] = NFACCT_FMT_DEFAULT;
+	} else if (param_len < ARRAY_SIZE(buf)) {
+		buf[param_len] = '\0';
+		for (j = NFACCT_FMT_DEFAULT; j <= NFACCT_FMT_MAX &&
+		     strncmp(buf, nfacct_fmt_keys[j],
+				strlen(nfacct_fmt_keys[j])+1) != 0; j++) {;}
+
+		if (j >= NFACCT_FMT_MAX) {
+			/* syntax error, exit */
+			goto err_free;
+		}
+		fmt[i] = j;
+	}
+
+	if (fmt[0] == NFACCT_FMT_MAX)
+		fmt[0] = fmt[1];
+
+	if (fmt[1] == NFACCT_FMT_MAX)
+		fmt[1] = fmt[0];
+
+	ret = (fmt[0] << 8) | fmt[1];
+
+err_free:
+	free(tmp);
+	if (i > 2 || j >= NFACCT_FMT_MAX)
+		ret = NFACCT_FMT_MAX;
+err:
+	return ret;
+}
+
+/* converts 'arg' to 64-bit unsigned long long */
+int nfacct_get_uint64_t(uint64_t *val, const char *arg)
+{
+	unsigned long long res;
+	char *ptr;
+
+	if (!arg || !*arg || strchr(arg,'-') ||
+	    strlen(arg) == 0)
+		return -1;
+
+	errno = 0;
+	res = strtoull(arg, &ptr, 10);
+
+	/* empty string or trailing non-digits */
+	if (!ptr || ptr == arg || *ptr)
+		return -1;
+
+	/* overflow */
+	if (res == ULLONG_MAX && errno == ERANGE)
+		return -1;
+
+	/* in case ULL is 128 bits */
+	if (res > 0xFFFFFFFFFFFFFFFFULL)
+		return -1;
+
+	*val = res;
+	return 0;
+}
+
+
diff --git a/src/nfacct_utils.h b/src/nfacct_utils.h
new file mode 100644
index 0000000..56f6f33
--- /dev/null
+++ b/src/nfacct_utils.h
@@ -0,0 +1,30 @@
+/*
+ *	This library is free software; you can redistribute it and/or
+ *	modify it under the terms of the GNU Lesser General Public
+ *	License as published by the Free Software Foundation version 2.1
+ *	of the License.
+ *
+ */
+
+#ifndef _NFACCT_UTILS_H_
+#define _NFACCT_UTILS_H_
+
+#include <libnetfilter_acct/libnetfilter_acct.h>
+
+#ifndef ARRAY_SIZE
+#define ARRAY_SIZE(x) (sizeof(x) / sizeof(*(x)))
+#endif
+
+/* maximum length of a token */
+#define MAX_TOKEN_SIZE 256
+
+extern int nfacct_matches(const char *cmd, const char *pattern);
+extern int nfacct_parse_tokens_length(const char *str, const char *sep,
+				      const size_t len,
+				      const int skip_comment,
+				      size_t t_len[]);
+extern uint16_t nfacct_parse_format_options(const char *str);
+extern int nfacct_get_uint64_t(uint64_t *val, const char *arg);
+
+#endif
+
-- 
1.8.3.1

[PATCH v3 nfacct 13/29] add new "save" and correct existing "restore" commands

[Michael Zintakis]

* add a new "save" command, which allows nfacct object list to be printed in a
format suitable for use by "restore". All nfacct object properties are shown
as name=value pairs, allowing for flexibility with further (properties)
expansion as the order in which these properties are specified is not
important. Account object names are properly quoted, when necessary. Allow
comments (lines starting with "#") and blank lines to be ommitted from
processing;

* change the existing "restore" command to make a good use of the new format
used by "save" and implement proper data integrity checks prior to adding
nfacct objects. Implement proper error reporting, giving additional
information, like line number, nature of the error occured and so on;

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/nfacct.c       | 229 +++++++++++++++++++++++++++++++++++++++++++++++------
 src/nfacct_utils.c |  91 +++++++++++++++++++++
 src/nfacct_utils.h |   3 +
 3 files changed, 300 insertions(+), 23 deletions(-)

diff --git a/src/nfacct.c b/src/nfacct.c
index aebb6ac..c778d1c 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -76,6 +76,7 @@ static int nfacct_cmd_flush(int argc, char *argv[]);
 static int nfacct_cmd_version(int argc, char *argv[]);
 static int nfacct_cmd_help(int argc, char *argv[]);
 static int nfacct_cmd_restore(int argc, char *argv[]);
+static int nfacct_cmd_save(int argc, char *argv[]);
 
 /* main command 'menu' */
 static const struct cmd {
@@ -87,6 +88,7 @@ static const struct cmd {
 	{ "delete",	nfacct_cmd_delete },
 	{ "get",	nfacct_cmd_get },
 	{ "flush",	nfacct_cmd_flush },
+	{ "save",	nfacct_cmd_save },
 	{ "restore",	nfacct_cmd_restore },
 	{ "version",	nfacct_cmd_version },
 	{ "help",	nfacct_cmd_help },
@@ -660,6 +662,7 @@ static const char help_msg[] =
 	"  delete NAME\t\tDelete existing accounting object NAME\n"
 	"  get NAME GET_PARAMS\tGet and list existing accounting object NAME\n"
 	"  flush\t\t\tFlush accounting object table\n"
+	"  save\t\t\tDump current accounting object table to stdout\n"
 	"  restore RST_PARAMS\tRestore accounting object table from stdin\n"
 	"  version\t\tDisplay version and disclaimer\n"
 	"  help\t\t\tDisplay this help message\n\n"
@@ -679,13 +682,119 @@ static int nfacct_cmd_help(int argc, char *argv[])
 	return 0;
 }
 
+static int nfacct_cmd_save(int argc, char *argv[])
+{
+	struct mnl_socket *nl;
+	char buf[MNL_SOCKET_BUFFER_SIZE];
+	struct nlmsghdr *nlh;
+	unsigned int seq, portid;
+	int ret = -1, i;
+	bool ignore_width = true;
+	struct nfa *nfa = NULL;
+
+	if (argc > 0) {
+		NFACCT_RET_ERR("too many arguments");
+	}
+	seq = time(NULL);
+	nlh = nfacct_nlmsg_build_hdr(buf, NFNL_MSG_ACCT_GET,
+				     NLM_F_DUMP, seq);
+
+	nl = mnl_socket_open(NETLINK_NETFILTER);
+	if (nl == NULL) {
+		NFACCT_RET_ERR("mnl_socket_open");
+	}
+
+	if (mnl_socket_bind(nl, 0, MNL_SOCKET_AUTOPID) < 0) {
+		NFACCT_RET_ERR("mnl_socket_bind");
+	}
+	portid = mnl_socket_get_portid(nl);
+
+	if (mnl_socket_sendto(nl, nlh, nlh->nlmsg_len) < 0) {
+		NFACCT_RET_ERR("mnl_socket_send");
+	}
+
+	options = nfacct_options_alloc();
+	if (options == NULL) {
+		NFACCT_RET_ERR("OOM");
+	}
+	nfacct_option_set_u16(options, NFACCT_OPT_FMT, NFACCT_FMT_MAX);
+
+	i = mnl_socket_recvfrom(nl, buf, ARRAY_SIZE(buf));
+	while (i > 0) {
+		i = mnl_cb_run(buf, i, seq, portid, nfacct_cb,
+				&ignore_width);
+		if (i <= 0)
+			break;
+		i = mnl_socket_recvfrom(nl, buf, ARRAY_SIZE(buf));
+	}
+	if (i == -1) {
+		nfacct_perror("error");
+		goto err_free_nfa;
+	}
+	mnl_socket_close(nl);
+
+	nfacct_list_for_each_entry(nfa, &nfa_list, head) {
+		nfacct_snprintf_with_options(buf, ARRAY_SIZE(buf),
+				nfa->nfacct,
+				NFACCT_SNPRINTF_T_PLAIN,
+				NFACCT_SNPRINTF_F_SAVE, options);
+		printf("%s\n",buf);
+	}
+
+	ret = 0;
+
+err_free_nfa:
+	free_nfa_list();
+	nfacct_options_free(options);
+
+err:
+	return ret;
+}
+
+/*
+ * Maximum number of restore tokens accepted:
+ * name= pkts= bytes=
+ *
+ */
+#define NFACCT_MAX_TOKENS 3
+
+/*
+ * Maximum number of value tokens accepted:
+ * t_name=t_value
+ *
+ */
+#define NFACCT_MAX_VTOKENS 2
+
+#define NFACCT_FREE_TOKENS 	for (i = 0; i < ret; i++) \
+				free(tokens[i]);
+
+#define NFACCT_FREE_VTOKENS 	for (n = 0; n < vret; n++) \
+				free(vtokens[n]);
+
+#define NFACCT_SNPRINTF(x,y)	snprintf(err_str,ARRAY_SIZE(err_str), \
+					 x,line,y);
+
+#define NFACCT_PRINT_VERR(x,y)	NFACCT_SNPRINTF(x,y);	\
+				goto err_free_vtokens;
+
+#define NFACCT_PRINT_CERR(x,y)	NFACCT_SNPRINTF(x,y);	\
+				goto err_free_nfacct;
+
+#define NFACCT_PRINT_ERR(x,y)	NFACCT_SNPRINTF(x,y);	\
+				goto err_free_tokens;
+
 static int nfacct_cmd_restore(int argc, char *argv[])
 {
-	uint64_t pkts, bytes;
-	char name[512];
-	char buffer[512];
-	int ret;
 	bool replace = false, flush = false;
+	bool b_name = false, b_pkts = false, b_bytes = false;
+	uint64_t pkts = 0, bytes = 0;
+	char *tokens[NFACCT_MAX_TOKENS + 1];
+	char *vtokens[NFACCT_MAX_VTOKENS + 1];
+	char buf[MAX_TOKEN_SIZE];
+	char err_str[80];
+	int line = 1;
+	int i, j, n;
+	int ret, vret;
 	struct nfacct *nfacct;
 
 	if (argc > 2) {
@@ -710,35 +819,109 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 		}
 	}
 
-	while (fgets(buffer, sizeof(buffer), stdin)) {
-		char *semicolon = strchr(buffer, ';');
-		if (semicolon == NULL) {
-			nfacct_perror("invalid line");
-			return -1;
-		}
-		*semicolon = 0;
-		ret = sscanf(buffer,
-			     "{ pkts = %"PRIu64", bytes = %"PRIu64" } = %s",
-		       &pkts, &bytes, name);
-		if (ret != 3) {
-			nfacct_perror("error reading input");
-			return -1;
+	for (; fgets(buf, ARRAY_SIZE(buf), stdin); pkts = 0, bytes = 0,
+	     b_name = false, b_pkts = false, b_bytes = false, line++) {
+		ret = nfacct_parse_tokens(buf, " \n", NFACCT_MAX_TOKENS + 1,
+					  true, tokens);
+		if (ret == 0)
+			continue;
+
+		if (ret > NFACCT_MAX_TOKENS) {
+			NFACCT_PRINT_ERR("error on line %d: " 
+					 "%d tokens retrieved", ret);
 		}
+
 		nfacct = nfacct_alloc();
 		if (nfacct == NULL) {
-			NFACCT_RET_ERR("OOM error");
+			NFACCT_PRINT_ERR("error on line %d: " 
+					 "OOM error.%s","");
+		}
+		for (j = 0; j < ret; j++) {
+			vret = nfacct_parse_tokens(tokens[j], "=",
+						   NFACCT_MAX_VTOKENS + 1,
+						   false, vtokens);
+			if (vret != NFACCT_MAX_VTOKENS) {
+				NFACCT_PRINT_VERR("error on line %d: "
+					  "invalid token '%s'", tokens[j]);
+			}
+			if (!b_name && strncmp(vtokens[0], "name",
+				    strlen("name") + 1) == 0) {
+				if (strlen(vtokens[1]) == 0) {
+					NFACCT_PRINT_VERR("error on line %d: "
+					"invalid 'name' token (%s)",
+					"not set");
+				}
+				nfacct_attr_set(nfacct, NFACCT_ATTR_NAME,
+						vtokens[1]);
+				b_name = true;
+			} else if (!b_pkts && strncmp(vtokens[0], "pkts",
+						   strlen("pkts") + 1) == 0) {
+				if (nfacct_get_uint64_t(&pkts,
+							vtokens[1]) != 0) {
+					NFACCT_PRINT_VERR("error on line %d: "
+						"invalid 'pkts' token (%s)",
+						vtokens[1]);
+				}
+				nfacct_attr_set_u64(nfacct,
+						    NFACCT_ATTR_PKTS, pkts);
+				b_pkts = true;
+			} else if (!b_bytes && strncmp(vtokens[0], "bytes",
+					         strlen("bytes") + 1) == 0) {
+				if (nfacct_get_uint64_t(&bytes,
+						 vtokens[1]) != 0) {
+					NFACCT_PRINT_VERR("error on line %d: "
+						"invalid 'bytes' token (%s)",
+						vtokens[1]);
+				}
+				nfacct_attr_set_u64(nfacct,
+						    NFACCT_ATTR_BYTES, bytes);
+				b_bytes = true;
+			} else {
+				NFACCT_PRINT_VERR("error on line %d: "
+					  "invalid token '%s'", tokens[j]);
+			}
+			NFACCT_FREE_VTOKENS;
 		}
-		nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, name);
-		nfacct_attr_set_u64(nfacct, NFACCT_ATTR_PKTS, pkts);
-		nfacct_attr_set_u64(nfacct, NFACCT_ATTR_BYTES, bytes);
+
+		/* Final integrity checks before adding */
+		if (!nfacct_attr_get(nfacct, NFACCT_ATTR_NAME)) {
+			NFACCT_PRINT_CERR("error on line %d: "
+					"invalid 'name' token (%s)",
+					"not set");
+		}
+		if (nfacct_attr_get(nfacct, NFACCT_ATTR_BYTES) &&
+		    !nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS)) {
+			NFACCT_PRINT_CERR("error on line %d: "
+					"invalid 'pkts' token (%s)",
+					"not set");
+		}
+		if (nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS) &&
+		    !nfacct_attr_get(nfacct, NFACCT_ATTR_BYTES)) {
+			NFACCT_PRINT_CERR("error on line %d: "
+					"invalid 'bytes' token (%s)",
+					"not set");
+		}
+		NFACCT_FREE_TOKENS;
+
 		ret = _nfacct_cmd_add(nfacct, replace);
 		if (ret != 0) {
-			NFACCT_RET_ERR("error during add");
+			NFACCT_SNPRINTF("error on line %d: "
+					"element not added%s","");
+			return -1;
 		}
-
 	}
 	return 0;
 
+err_free_vtokens:
+	NFACCT_FREE_VTOKENS;
+
+err_free_nfacct:
+	nfacct_free(nfacct);
+
+err_free_tokens:
+	NFACCT_FREE_TOKENS;
+	nfacct_perror(err_str);
+
 err:
 	return -1;
 }
diff --git a/src/nfacct_utils.c b/src/nfacct_utils.c
index 61b2ad5..96cc811 100644
--- a/src/nfacct_utils.c
+++ b/src/nfacct_utils.c
@@ -33,6 +33,97 @@ int nfacct_matches(const char *cmd, const char *pattern)
 /*
  * Takes 'str' and breaks it in maximum of 'len' tokens, using 'sep'
  * as separators, taking into account character escaping (\) and
+ * string quotation (e.g. "abc de"). Each token is stored in 'tokens'.
+ * The function returns the number of tokens actually processed and stored.
+ *
+ * N.B.:
+ *	1. Character escaping is NOT translated/taken into account. In
+ *	   other words, "\t" translates to "t".
+ *	2. If skip_comment=true and if 'str' starts with 'comment',
+ *	   then the parsing is skipped and nothing is returned.
+ *	3. It is assumed that each resulting token is no more than
+ *	   MAX_TOKEN_SIZE characters.
+ *
+ */
+int nfacct_parse_tokens(const char *str, const char *sep, const size_t len,
+			const int skip_comment, char *tokens[])
+{
+	bool quote_open = false, escaped = false;
+	size_t param_len = 0, i = 0;
+	static const char comment = '#';
+	char buf[MAX_TOKEN_SIZE], *ptr, *tmp;
+
+	if (str == NULL || strlen(str) == 0 || tokens == NULL ||
+	    sep == NULL || strlen(sep) == 0 ||
+	    (skip_comment && strchr(str, comment) != NULL &&
+	     *strchr(str, comment) == str[0]))
+		goto err;
+
+	tmp = strdup(str);
+	for (ptr = tmp; *ptr; ptr++) {
+		if (quote_open) {
+			if (escaped) {
+				if (param_len >= ARRAY_SIZE(buf))
+					goto err_free;
+				buf[param_len++] = *ptr;
+				escaped = false;
+				continue;
+			} else if (*ptr == '\\') {
+				escaped = true;
+				continue;
+			} else if (*ptr == '"') {
+				quote_open = false;
+				*ptr = *sep;
+			} else {
+				if (param_len >= ARRAY_SIZE(buf))
+					goto err_free;
+				buf[param_len++] = *ptr;
+				continue;
+			}
+		} else {
+			if (*ptr == '"') {
+				quote_open = true;
+				continue;
+			}
+		}
+		if (strchr(sep, *ptr)) {
+			if (!param_len)
+				continue;
+
+			if (!param_len || i >= len ||
+			    param_len >= ARRAY_SIZE(buf))
+				goto err_free;
+
+			buf[param_len] = '\0';
+			tokens[i] = strdup(buf);
+			i++;
+			param_len = 0;
+		} else {
+			/* regular character, copy to buffer */
+			if (param_len >= ARRAY_SIZE(buf))
+				goto err_free;
+			buf[param_len++] = *ptr;
+		}
+	}
+
+	if (escaped || quote_open || !param_len || i >= len)
+		goto err_free;
+
+	if (param_len && param_len < ARRAY_SIZE(buf)) {
+		buf[param_len] = '\0';
+		tokens[i] = strdup(buf);
+		i++;
+	}
+
+err_free:
+	free(tmp);
+err:
+	return i;
+}
+
+/*
+ * Takes 'str' and breaks it in maximum of 'len' tokens, using 'sep'
+ * as separators, taking into account character escaping (\) and
  * string quotation (e.g. "abc de"). Each token length is stored in 't_len'.
  * The function returns the number of tokens actually processed.
  *
diff --git a/src/nfacct_utils.h b/src/nfacct_utils.h
index 56f6f33..d9c5bb7 100644
--- a/src/nfacct_utils.h
+++ b/src/nfacct_utils.h
@@ -19,6 +19,9 @@
 #define MAX_TOKEN_SIZE 256
 
 extern int nfacct_matches(const char *cmd, const char *pattern);
+extern int nfacct_parse_tokens(const char *str, const char *sep,
+				const size_t len, const int skip_comment,
+				char *tokens[]);
 extern int nfacct_parse_tokens_length(const char *str, const char *sep,
 				      const size_t len,
 				      const int skip_comment,
-- 
1.8.3.1

[PATCH v3 nfacct 14/29] add sort option to the "list" command

[Michael Zintakis]

* add a new "sort" option to the "list" command, allowing nfacct object listed
to be sorted by name (default), packets, bytes or not to be sorted at all (as
was the case up until now);

* add a new nfacct_list.c source (in addition to the nfacct_list.h)
implementing the main sort function, in addition to the existing linked-list
routines.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/Makefile.am   |   2 +-
 src/nfacct.c      |  70 ++++++++++++++++++++++++-
 src/nfacct_list.c | 149 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 src/nfacct_list.h |   4 ++
 4 files changed, 222 insertions(+), 3 deletions(-)
 create mode 100644 src/nfacct_list.c

diff --git a/src/Makefile.am b/src/Makefile.am
index 133ce61..ae60a51 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -2,6 +2,6 @@ include $(top_srcdir)/Make_global.am
 
 sbin_PROGRAMS = nfacct
 
-nfacct_SOURCES = nfacct.c nfacct_utils.c
+nfacct_SOURCES = nfacct.c nfacct_utils.c nfacct_list.c
 
 nfacct_LDADD = ${LIBMNL_LIBS} ${LIBNETFILTER_ACCT_LIBS}
diff --git a/src/nfacct.c b/src/nfacct.c
index c778d1c..3eb8b68 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -53,6 +53,49 @@ static void free_nfa_list(void) {
 	}
 }
 
+enum nfacct_sort_mode {
+	NFACCT_SORT_NONE = 0,
+	NFACCT_SORT_NAME,
+	NFACCT_SORT_PKTS,
+	NFACCT_SORT_BYTES,
+};
+
+static int nfacct_cmp(void *priv, struct nfacct_list_head *a, struct nfacct_list_head *b)
+{
+	struct nfa *nfa1, *nfa2;
+	uint64_t v1, v2;
+	enum nfacct_attr_type attr = NFACCT_ATTR_NAME;
+	enum nfacct_sort_mode *mode = (enum nfacct_sort_mode *)priv;
+
+	nfa1 = nfacct_list_entry(a, struct nfa, head);
+	nfa2 = nfacct_list_entry(b, struct nfa, head);
+
+	switch(*mode) {
+	case NFACCT_SORT_PKTS:
+				attr = NFACCT_ATTR_PKTS;
+				break;
+	case NFACCT_SORT_BYTES:
+				attr = NFACCT_ATTR_BYTES;
+	case NFACCT_SORT_NAME:
+				break;
+	default: /* unsorted */
+		return 0;
+	}
+	if (attr != NFACCT_ATTR_NAME) {
+		v1 = nfacct_attr_get_u64(nfa1->nfacct, attr);
+		v2 = nfacct_attr_get_u64(nfa2->nfacct, attr);
+		if (v1 < v2) {
+			return -1;
+		} else if (v1 > v2) {
+			return 1;
+		}
+	}
+	return strcmp(nfacct_attr_get_str(nfa1->nfacct,
+					  NFACCT_ATTR_NAME),
+		      nfacct_attr_get_str(nfa2->nfacct,
+					  NFACCT_ATTR_NAME));
+}
+
 /* xml header & footer strings */
 #define NFACCT_XML_HEADER	"<?xml version=\"1.0\" encoding=\"utf-8\"?>\n" \
 				"<nfacct>\n"
@@ -219,7 +262,7 @@ err:
 static int nfacct_cmd_list(int argc, char *argv[])
 {
 	bool nfnl_msg = false, xml = false;
-	bool b_fmt = false;
+	bool b_fmt = false, b_sort = false;
 	struct mnl_socket *nl;
 	char buf[70000];
 	struct nlmsghdr *nlh;
@@ -228,6 +271,7 @@ static int nfacct_cmd_list(int argc, char *argv[])
 	struct nfa *nfa = NULL;
 	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
 	uint16_t fmt = NFACCT_FMT_MAX;
+	enum nfacct_sort_mode sort_mode = NFACCT_SORT_NAME;
 
 	while (argc > 0) {
 		if (!nfnl_msg && nfacct_matches(argv[0],"reset")) {
@@ -235,6 +279,21 @@ static int nfacct_cmd_list(int argc, char *argv[])
 			nfnl_msg = true;
 		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
+		} else if (!b_sort && nfacct_matches(argv[0],"sort")) {
+			NFACCT_GET_NEXT_ARG();
+			if (nfacct_matches(argv[0],"name")) {
+				sort_mode = NFACCT_SORT_NAME;
+			} else if (nfacct_matches(argv[0],"packets") ||
+				   nfacct_matches(argv[0],"pkts")) {
+				sort_mode = NFACCT_SORT_PKTS;
+			} else if (nfacct_matches(argv[0],"bytes")) {
+				sort_mode = NFACCT_SORT_BYTES;
+			} else if (nfacct_matches(argv[0],"none")) {
+				sort_mode = NFACCT_SORT_NONE;
+			} else {
+				NFACCT_RET_ARG_ERR();
+			}
+			b_sort = true;
 		} else if (!b_fmt && (nfacct_matches(argv[0],"format") ||
 			   nfacct_matches(argv[0],"fmt"))) {
 			NFACCT_GET_NEXT_ARG();
@@ -288,6 +347,9 @@ static int nfacct_cmd_list(int argc, char *argv[])
 	if (xml)
 		printf(NFACCT_XML_HEADER);
 
+	if (sort_mode != NFACCT_SORT_NONE)
+		nfacct_list_sort(&sort_mode, &nfa_list, nfacct_cmp);
+
 	nfacct_list_for_each_entry(nfa, &nfa_list, head) {
 		nfacct_snprintf_with_options(buf, ARRAY_SIZE(buf),
 				nfa->nfacct,
@@ -667,11 +729,12 @@ static const char help_msg[] =
 	"  version\t\tDisplay version and disclaimer\n"
 	"  help\t\t\tDisplay this help message\n\n"
 	"Parameters:\n"
-	"  LST_PARAMS := [ reset ] [ format FMT_SPEC ] [ xml ]\n"
+	"  LST_PARAMS := [ reset ] [ format FMT_SPEC ] [ sort SORT_SPEC ] [ xml ]\n"
 	"  ADD_PARAMS := [ replace ]\n"
 	"  GET_PARAMS := [ reset ] [ format FMT_SPEC ] [ xml ]\n"
 	"  RST_PARAMS := [ flush ] [ replace ]\n"
 	"  FMT_SPEC := { [FMT] | [,] | [FMT] ... }\n"
+	"  SORT_SPEC := { none | name | packets | bytes }"
 	"  FMT := { def | raw | 3pl | iec | kib | mib | gib | tib | pib |"
 		  " eib |\n"
 	"  \t   si | kb | mb | gb | tb | pb | eb }\n";
@@ -691,6 +754,7 @@ static int nfacct_cmd_save(int argc, char *argv[])
 	int ret = -1, i;
 	bool ignore_width = true;
 	struct nfa *nfa = NULL;
+	enum nfacct_sort_mode sort_mode = NFACCT_SORT_NAME;
 
 	if (argc > 0) {
 		NFACCT_RET_ERR("too many arguments");
@@ -733,6 +797,8 @@ static int nfacct_cmd_save(int argc, char *argv[])
 	}
 	mnl_socket_close(nl);
 
+	nfacct_list_sort(&sort_mode, &nfa_list, nfacct_cmp);
+
 	nfacct_list_for_each_entry(nfa, &nfa_list, head) {
 		nfacct_snprintf_with_options(buf, ARRAY_SIZE(buf),
 				nfa->nfacct,
diff --git a/src/nfacct_list.c b/src/nfacct_list.c
new file mode 100644
index 0000000..242ac0c
--- /dev/null
+++ b/src/nfacct_list.c
@@ -0,0 +1,149 @@
+/*
+ *	This library is free software; you can redistribute it and/or
+ *	modify it under the terms of the GNU Lesser General Public
+ *	License as published by the Free Software Foundation version 2.1
+ *	of the License.
+ */
+
+#include <string.h>
+
+#include "nfacct_list.h"
+#include "nfacct_utils.h"
+
+#define MAX_LIST_LENGTH_BITS 20
+
+/*
+ * Returns a list organized in an intermediate format suited
+ * to chaining of merge() calls: null-terminated, no reserved or
+ * sentinel head node, "prev" links not maintained.
+ */
+static struct nfacct_list_head *merge(void *priv,
+				int (*cmp)(void *priv,
+					struct nfacct_list_head *a,
+					struct nfacct_list_head *b),
+				struct nfacct_list_head *a,
+				struct nfacct_list_head *b)
+{
+	struct nfacct_list_head head, *tail = &head;
+
+	while (a && b) {
+		/* if equal, take 'a' -- important for sort stability */
+		if ((*cmp)(priv, a, b) <= 0) {
+			tail->next = a;
+			a = a->next;
+		} else {
+			tail->next = b;
+			b = b->next;
+		}
+		tail = tail->next;
+	}
+	tail->next = a?:b;
+	return head.next;
+}
+
+/*
+ * Combine final list merge with restoration of standard doubly-linked
+ * list structure.  This approach duplicates code from merge(), but
+ * runs faster than the tidier alternatives of either a separate final
+ * prev-link restoration pass, or maintaining the prev links
+ * throughout.
+ */
+static void merge_and_restore_back_links(void *priv,
+				int (*cmp)(void *priv,
+					struct nfacct_list_head *a,
+					struct nfacct_list_head *b),
+				struct nfacct_list_head *head,
+				struct nfacct_list_head *a,
+				struct nfacct_list_head *b)
+{
+	struct nfacct_list_head *tail = head;
+
+	while (a && b) {
+		/* if equal, take 'a' -- important for sort stability */
+		if ((*cmp)(priv, a, b) <= 0) {
+			tail->next = a;
+			a->prev = tail;
+			a = a->next;
+		} else {
+			tail->next = b;
+			b->prev = tail;
+			b = b->next;
+		}
+		tail = tail->next;
+	}
+	tail->next = a ? : b;
+
+	do {
+		/*
+		 * In worst cases this loop may run many iterations.
+		 * Continue callbacks to the client even though no
+		 * element comparison is needed, so the client's cmp()
+		 * routine can invoke cond_resched() periodically.
+		 */
+		(*cmp)(priv, tail->next, tail->next);
+
+		tail->next->prev = tail;
+		tail = tail->next;
+	} while (tail->next);
+
+	tail->next = head;
+	head->prev = tail;
+}
+
+/**
+ * nfacct_list_sort - sort a list
+ * @priv: private data, opaque to nfacct_list_sort(), passed to @cmp
+ * @head: the list to sort
+ * @cmp: the elements comparison function
+ *
+ * This function implements "merge sort", which has O(nlog(n))
+ * complexity.
+ *
+ * The comparison function @cmp must return a negative value if @a
+ * should sort before @b, and a positive value if @a should sort after
+ * @b. If @a and @b are equivalent, and their original relative
+ * ordering is to be preserved, @cmp must return 0.
+ */
+void nfacct_list_sort(void *priv, struct nfacct_list_head *head,
+		int (*cmp)(void *priv, struct nfacct_list_head *a,
+			struct nfacct_list_head *b))
+{
+	/* sorted partial lists	-- last slot is a sentinel */
+	struct nfacct_list_head *part[MAX_LIST_LENGTH_BITS+1];
+	size_t lev;  /* index into part[] */
+	size_t max_lev = 0;
+	struct nfacct_list_head *list;
+
+	if (nfacct_list_empty(head))
+		return;
+
+	memset(part, 0, sizeof(part));
+
+	head->prev->next = NULL;
+	list = head->next;
+
+	while (list) {
+		struct nfacct_list_head *cur = list;
+		list = list->next;
+		cur->next = NULL;
+
+		for (lev = 0; part[lev]; lev++) {
+			cur = merge(priv, cmp, part[lev], cur);
+			part[lev] = NULL;
+		}
+		if (lev > max_lev) {
+			if (lev >= ARRAY_SIZE(part)-1) {
+				lev--;
+			}
+			max_lev = lev;
+		}
+		part[lev] = cur;
+	}
+
+	for (lev = 0; lev < max_lev; lev++)
+		if (part[lev])
+			list = merge(priv, cmp, part[lev], list);
+
+	merge_and_restore_back_links(priv, cmp, head, part[max_lev], list);
+}
+
diff --git a/src/nfacct_list.h b/src/nfacct_list.h
index 5ef1b9d..1fe5674 100644
--- a/src/nfacct_list.h
+++ b/src/nfacct_list.h
@@ -52,6 +52,10 @@ static inline int nfacct_list_empty(struct nfacct_list_head *head)
 	return head->next == head;
 }
 
+extern void nfacct_list_sort(void *priv, struct nfacct_list_head *head,
+		int (*cmp)(void *priv, struct nfacct_list_head *a,
+		struct nfacct_list_head *b));
+
 #define nfacct_container_of(ptr, type, member) ({			\
         typeof( ((type *)0)->member ) *__mptr = (ptr);			\
         (type *)( (char *)__mptr - ((size_t) &((type *)0)->member));})
-- 
1.8.3.1

[PATCH v3 nfacct 15/29] add "show bytes" option to "list" and "get" commands

[Michael Zintakis]

* add a new "show bytes" option to the "list" and get commands, allowing only
the byte counter column (in addition to nfacct name) to be shown.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/nfacct.c | 31 +++++++++++++++++++++++++------
 1 file changed, 25 insertions(+), 6 deletions(-)

diff --git a/src/nfacct.c b/src/nfacct.c
index 3eb8b68..3f25334 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -262,7 +262,7 @@ err:
 static int nfacct_cmd_list(int argc, char *argv[])
 {
 	bool nfnl_msg = false, xml = false;
-	bool b_fmt = false, b_sort = false;
+	bool b_fmt = false, b_sh = false, b_sort = false;
 	struct mnl_socket *nl;
 	char buf[70000];
 	struct nlmsghdr *nlh;
@@ -271,6 +271,7 @@ static int nfacct_cmd_list(int argc, char *argv[])
 	struct nfa *nfa = NULL;
 	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
 	uint16_t fmt = NFACCT_FMT_MAX;
+	uint16_t flags = NFACCT_SNPRINTF_F_FULL;
 	enum nfacct_sort_mode sort_mode = NFACCT_SORT_NAME;
 
 	while (argc > 0) {
@@ -279,6 +280,14 @@ static int nfacct_cmd_list(int argc, char *argv[])
 			nfnl_msg = true;
 		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
+		} else if (!b_sh && nfacct_matches(argv[0],"show")) {
+			NFACCT_GET_NEXT_ARG();
+			if (nfacct_matches(argv[0],"bytes")) {
+				flags = NFACCT_SNPRINTF_F_BONLY;
+			} else {
+				NFACCT_RET_ARG_ERR();
+			}
+			b_sh = true;
 		} else if (!b_sort && nfacct_matches(argv[0],"sort")) {
 			NFACCT_GET_NEXT_ARG();
 			if (nfacct_matches(argv[0],"name")) {
@@ -355,7 +364,7 @@ static int nfacct_cmd_list(int argc, char *argv[])
 				nfa->nfacct,
 				xml ? NFACCT_SNPRINTF_T_XML :
 				      NFACCT_SNPRINTF_T_PLAIN,
-				NFACCT_SNPRINTF_F_FULL, options);
+				flags, options);
 		printf("%s\n",buf);
 	}
 
@@ -534,7 +543,7 @@ static int nfacct_cmd_delete(int argc, char *argv[])
 static int nfacct_cmd_get(int argc, char *argv[])
 {
 	bool nfnl_msg = false, xml = false;
-	bool b_fmt = false;
+	bool b_sh = false, b_fmt = false;
 	struct mnl_socket *nl;
 	char buf[MNL_SOCKET_BUFFER_SIZE];
 	struct nlmsghdr *nlh;
@@ -545,6 +554,7 @@ static int nfacct_cmd_get(int argc, char *argv[])
 	char *name;
 	uint8_t nfnl_cmd = NFNL_MSG_ACCT_GET;
 	uint16_t fmt = NFACCT_FMT_MAX;
+	uint16_t flags = NFACCT_SNPRINTF_F_FULL;
 
 	if (argc < 1 || strlen(argv[0]) == 0) {
 		nfacct_perror("missing object name");
@@ -562,6 +572,14 @@ static int nfacct_cmd_get(int argc, char *argv[])
 			nfnl_msg = true;
 		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
+		} else if (!b_sh && nfacct_matches(argv[0],"show")) {
+			NFACCT_GET_NEXT_ARG();
+			if (nfacct_matches(argv[0],"bytes")) {
+				flags = NFACCT_SNPRINTF_F_BONLY;
+			} else {
+				NFACCT_RET_ARG_ERR();
+			}
+			b_sh = true;
 		} else if (!b_fmt && (nfacct_matches(argv[0],"format") ||
 			   nfacct_matches(argv[0],"fmt"))) {
 			NFACCT_GET_NEXT_ARG();
@@ -630,7 +648,7 @@ static int nfacct_cmd_get(int argc, char *argv[])
 				nfa->nfacct,
 				xml ? NFACCT_SNPRINTF_T_XML :
 				      NFACCT_SNPRINTF_T_PLAIN,
-				NFACCT_SNPRINTF_F_FULL, options);
+				flags, options);
 		printf("%s\n",buf);
 	}
 
@@ -729,9 +747,10 @@ static const char help_msg[] =
 	"  version\t\tDisplay version and disclaimer\n"
 	"  help\t\t\tDisplay this help message\n\n"
 	"Parameters:\n"
-	"  LST_PARAMS := [ reset ] [ format FMT_SPEC ] [ sort SORT_SPEC ] [ xml ]\n"
+	"  LST_PARAMS := [ reset ] [ show bytes ] [ format FMT_SPEC ]\n"
+			"\t\t[ sort SORT_SPEC ] [ xml ]\n"
 	"  ADD_PARAMS := [ replace ]\n"
-	"  GET_PARAMS := [ reset ] [ format FMT_SPEC ] [ xml ]\n"
+	"  GET_PARAMS := [ reset ] [ show bytes ] [ format FMT_SPEC ] [ xml ]\n"
 	"  RST_PARAMS := [ flush ] [ replace ]\n"
 	"  FMT_SPEC := { [FMT] | [,] | [FMT] ... }\n"
 	"  SORT_SPEC := { none | name | packets | bytes }"
-- 
1.8.3.1

[PATCH v3 kernel 16/29] add permanent byte/packet format capability to nfacct

[Michael Zintakis]

* add a 'fmt' variable to each nfacct object, allowing a permanent packets
and bytes formatting to be stored. The two packet and byte formats are
independent of each other.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/uapi/linux/netfilter/nfnetlink_acct.h |  1 +
 net/netfilter/nfnetlink_acct.c                | 28 +++++++++++++++++++++++++--
 2 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/include/uapi/linux/netfilter/nfnetlink_acct.h b/include/uapi/linux/netfilter/nfnetlink_acct.h
index c7b6269..0b65f9c1 100644
--- a/include/uapi/linux/netfilter/nfnetlink_acct.h
+++ b/include/uapi/linux/netfilter/nfnetlink_acct.h
@@ -19,6 +19,7 @@ enum nfnl_acct_type {
 	NFACCT_PKTS,
 	NFACCT_BYTES,
 	NFACCT_USE,
+	NFACCT_FMT,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
diff --git a/net/netfilter/nfnetlink_acct.c b/net/netfilter/nfnetlink_acct.c
index c14046c..92ecad1 100644
--- a/net/netfilter/nfnetlink_acct.c
+++ b/net/netfilter/nfnetlink_acct.c
@@ -32,6 +32,7 @@ static LIST_HEAD(nfnl_acct_list);
 struct nf_acct {
 	atomic64_t		pkts;
 	atomic64_t		bytes;
+	u16			fmt;
 	struct list_head	head;
 	atomic_t		refcnt;
 	char			name[NFACCT_NAME_MAX];
@@ -74,8 +75,26 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 				atomic64_set(&matching->bytes,
 				be64_to_cpu(nla_get_be64(tb[NFACCT_BYTES])));
 			} else {
-				atomic64_set(&matching->pkts, 0);
-				atomic64_set(&matching->bytes, 0);
+				/*
+				 * Prevent resetting the packet & byte counters
+				 * if any other parameters are specified.
+				 *
+				 * This is done for backward compatibility,
+				 * otherwise resetting these counters should
+				 * only be allowed when tb[NFACCT_PKTS] and
+				 * tb[NFACCT_BYTES] are explicitly specified
+				 * and == 0.
+				 *
+				 */
+				if (!tb[NFACCT_FMT]) {
+					atomic64_set(&matching->pkts, 0);
+					atomic64_set(&matching->bytes, 0);
+				}
+			}
+			/* ...and change the format... */
+			if (tb[NFACCT_FMT]) {
+				matching->fmt =
+				be16_to_cpu(nla_get_be16(tb[NFACCT_FMT]));
 			}
 			return 0;
 		}
@@ -96,6 +115,9 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 		atomic64_set(&nfacct->pkts,
 			     be64_to_cpu(nla_get_be64(tb[NFACCT_PKTS])));
 	}
+	if (tb[NFACCT_FMT]) {
+		nfacct->fmt = be16_to_cpu(nla_get_be16(tb[NFACCT_FMT]));
+	}
 	atomic_set(&nfacct->refcnt, 1);
 	list_add_tail_rcu(&nfacct->head, &nfnl_acct_list);
 	return 0;
@@ -132,6 +154,7 @@ nfnl_acct_fill_info(struct sk_buff *skb, u32 portid, u32 seq, u32 type,
 	}
 	if (nla_put_be64(skb, NFACCT_PKTS, cpu_to_be64(pkts)) ||
 	    nla_put_be64(skb, NFACCT_BYTES, cpu_to_be64(bytes)) ||
+	    nla_put_be16(skb, NFACCT_FMT, htons(acct->fmt)) ||
 	    nla_put_be32(skb, NFACCT_USE, htonl(atomic_read(&acct->refcnt))))
 		goto nla_put_failure;
 
@@ -279,6 +302,7 @@ static const struct nla_policy nfnl_acct_policy[NFACCT_MAX+1] = {
 	[NFACCT_NAME] = { .type = NLA_NUL_STRING, .len = NFACCT_NAME_MAX-1 },
 	[NFACCT_BYTES] = { .type = NLA_U64 },
 	[NFACCT_PKTS] = { .type = NLA_U64 },
+	[NFACCT_FMT] = { .type = NLA_U16 },
 };
 
 static const struct nfnl_callback nfnl_acct_cb[NFNL_MSG_ACCT_MAX] = {
-- 
1.8.3.1

Re: [PATCH v3 kernel 16/29] add permanent byte/packet format capability to nfacct

[Florian Westphal]

Michael Zintakis <michael.zintakis@googlemail.com> wrote:
> * add a 'fmt' variable to each nfacct object, allowing a permanent packets
> and bytes formatting to be stored. The two packet and byte formats are
> independent of each other.

Every other in-kernel byte counter (that I can think of) is a plain u64.

It might help if you'd explain why this is necessary.

Re: [PATCH v3 kernel 16/29] add permanent byte/packet format capability to nfacct

[Michael Zintakis]

Florian Westphal wrote:
> Michael Zintakis <michael.zintakis@googlemail.com> wrote:
>> * add a 'fmt' variable to each nfacct object, allowing a permanent packets
>> and bytes formatting to be stored. The two packet and byte formats are
>> independent of each other.
> 
> Every other in-kernel byte counter (that I can think of) is a plain u64.
> 
> It might help if you'd explain why this is necessary.
This isn't a counter. This field stores the formatting of byte and packet numbers for each accounting object registered with the kernel (8-bits each = 16 bits in total unsigned). If you look at the man page (section FORMAT OPTIONS) it is all explained there - with examples.

Re: [PATCH v3 kernel 16/29] add permanent byte/packet format capability to nfacct

[Florian Westphal]

Michael Zintakis <michael.zintakis@googlemail.com> wrote:
> Florian Westphal wrote:
> > Michael Zintakis <michael.zintakis@googlemail.com> wrote:
> >> * add a 'fmt' variable to each nfacct object, allowing a permanent packets
> >> and bytes formatting to be stored. The two packet and byte formats are
> >> independent of each other.
> > 
> > Every other in-kernel byte counter (that I can think of) is a plain u64.
> > 
> > It might help if you'd explain why this is necessary.
> This isn't a counter. This field stores the formatting of byte and packet numbers for each accounting object registered with the kernel (8-bits each = 16 bits in total unsigned). If you look at the man page (section FORMAT OPTIONS) it is all explained there - with examples.

It makes no sense to me.

Why should a 'display/representation property' be part of an operating
system kernel?  It seems completely out of place.

Re: [PATCH v3 kernel 16/29] add permanent byte/packet format capability to nfacct

[Michael Zintakis]

Florian Westphal wrote:
> Michael Zintakis <michael.zintakis@googlemail.com> wrote:
>> Florian Westphal wrote:
>>> Michael Zintakis <michael.zintakis@googlemail.com> wrote:
>>>> * add a 'fmt' variable to each nfacct object, allowing a permanent packets
>>>> and bytes formatting to be stored. The two packet and byte formats are
>>>> independent of each other.
>>> Every other in-kernel byte counter (that I can think of) is a plain u64.
>>>
>>> It might help if you'd explain why this is necessary.
>> This isn't a counter. This field stores the formatting of byte and packet numbers for each accounting object registered with the kernel (8-bits each = 16 bits in total unsigned). If you look at the man page (section FORMAT OPTIONS) it is all explained there - with examples.
> 
> It makes no sense to me.
> 
> Why should a 'display/representation property' be part of an operating
> system kernel?  It seems completely out of place.
So, what you are trying to tell me is that "display/presentation property", or, to extend that definition a bit more and include another of your and Pablo's objections from another message - "variables set/unset or interpreted from userspace" have no place in the kernel in your opinion?

[PATCH v3 libnetfilter_acct 17/29] add *permanent* number formatting support

[Michael Zintakis]

* add support for permanent number formatting of nfacct objects, using the
'fmt' value stored in each such object within the kernel;

* enhance the existing API to support get/set format functions;

* adjust all existing xml and plaintext templates to support this feature;

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/libnetfilter_acct/libnetfilter_acct.h |  3 ++
 include/linux/netfilter/nfnetlink_acct.h      |  1 +
 src/libnetfilter_acct.c                       | 72 ++++++++++++++++++++++++---
 src/libnetfilter_acct.map                     |  2 +
 4 files changed, 72 insertions(+), 6 deletions(-)

diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index 3328fdb..0944247 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -14,6 +14,7 @@ enum nfacct_attr_type {
 	NFACCT_ATTR_NAME = 0,
 	NFACCT_ATTR_PKTS,
 	NFACCT_ATTR_BYTES,
+	NFACCT_ATTR_FMT,
 };
 
 struct nfacct_options;
@@ -75,11 +76,13 @@ void nfacct_options_free(struct nfacct_options *options);
 void nfacct_attr_set(struct nfacct *nfacct, enum nfacct_attr_type type, const void *data);
 void nfacct_attr_set_str(struct nfacct *nfacct, enum nfacct_attr_type type, const char *name);
 void nfacct_attr_set_u64(struct nfacct *nfacct, enum nfacct_attr_type type, uint64_t value);
+void nfacct_attr_set_u16(struct nfacct *nfacct, enum nfacct_attr_type type, uint16_t value);
 void nfacct_attr_unset(struct nfacct *nfacct, enum nfacct_attr_type type);
 
 const void *nfacct_attr_get(struct nfacct *nfacct, enum nfacct_attr_type type);
 const char *nfacct_attr_get_str(struct nfacct *nfacct, enum nfacct_attr_type type);
 uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type);
+uint16_t nfacct_attr_get_u16(struct nfacct *nfacct, enum nfacct_attr_type type);
 
 void nfacct_option_set(struct nfacct_options *options,
 			enum nfacct_option_type type, const void *data);
diff --git a/include/linux/netfilter/nfnetlink_acct.h b/include/linux/netfilter/nfnetlink_acct.h
index c7b6269..0b65f9c 100644
--- a/include/linux/netfilter/nfnetlink_acct.h
+++ b/include/linux/netfilter/nfnetlink_acct.h
@@ -19,6 +19,7 @@ enum nfnl_acct_type {
 	NFACCT_PKTS,
 	NFACCT_BYTES,
 	NFACCT_USE,
+	NFACCT_FMT,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index 440bc0b..f7890a1 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -61,6 +61,11 @@ struct nfacct {
 	char		name[NFACCT_NAME_MAX];
 	uint64_t	pkts;
 	uint64_t	bytes;
+	/*
+	 * Structure of fmt:
+	 * <- packets format: 8 bits-><-bytes format: 8 bits ->
+	 */
+	uint16_t	fmt;
 	uint32_t	bitset;
 };
 
@@ -144,6 +149,10 @@ nfacct_attr_set(struct nfacct *nfacct, enum nfacct_attr_type type,
 		nfacct->bytes = *((uint64_t *) data);
 		nfacct->bitset |= (1 << NFACCT_ATTR_BYTES);
 		break;
+	case NFACCT_ATTR_FMT:
+		nfacct->fmt = *((uint16_t *) data);
+		nfacct->bitset |= (1 << NFACCT_ATTR_FMT);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_attr_set);
@@ -177,6 +186,20 @@ nfacct_attr_set_u64(struct nfacct *nfacct, enum nfacct_attr_type type,
 EXPORT_SYMBOL(nfacct_attr_set_u64);
 
 /**
+ * nfacct_attr_set_u16 - set one attribute the accounting object
+ * \param nfacct pointer to the accounting object
+ * \param type attribute type you want to set
+ * \param value unsigned 16-bit integer
+ */
+void
+nfacct_attr_set_u16(struct nfacct *nfacct, enum nfacct_attr_type type,
+		    uint16_t value)
+{
+	nfacct_attr_set(nfacct, type, &value);
+}
+EXPORT_SYMBOL(nfacct_attr_set_u16);
+
+/**
  * nfacct_attr_unset - unset one attribute the accounting object
  * \param nfacct pointer to the accounting object
  * \param type attribute type you want to set
@@ -194,6 +217,9 @@ nfacct_attr_unset(struct nfacct *nfacct, enum nfacct_attr_type type)
 	case NFACCT_ATTR_BYTES:
 		nfacct->bitset &= ~(1 << NFACCT_ATTR_BYTES);
 		break;
+	case NFACCT_ATTR_FMT:
+		nfacct->bitset &= ~(1 << NFACCT_ATTR_FMT);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_attr_unset);
@@ -223,6 +249,10 @@ const void *nfacct_attr_get(struct nfacct *nfacct, enum nfacct_attr_type type)
 		if (nfacct->bitset & (1 << NFACCT_ATTR_BYTES))
 			ret = &nfacct->bytes;
 		break;
+	case NFACCT_ATTR_FMT:
+		if (nfacct->bitset & (1 << NFACCT_ATTR_FMT))
+			ret = &nfacct->fmt;
+		break;
 	}
 	return ret;
 }
@@ -259,6 +289,21 @@ uint64_t nfacct_attr_get_u64(struct nfacct *nfacct, enum nfacct_attr_type type)
 EXPORT_SYMBOL(nfacct_attr_get_u64);
 
 /**
+ * nfacct_attr_get_u16 - get one attribute the accounting object
+ * \param nfacct pointer to the accounting object
+ * \param type attribute type you want to get
+ *
+ * This function returns a unsigned 16-bits integer. If the attribute is
+ * unsupported, this returns NULL.
+ */
+uint16_t nfacct_attr_get_u16(struct nfacct *nfacct, enum nfacct_attr_type type)
+{
+	const void *ret = nfacct_attr_get(nfacct, type);
+	return ret ? *((uint16_t *)ret) : 0;
+}
+EXPORT_SYMBOL(nfacct_attr_get_u16);
+
+/**
  * nfacct_option_set - set one option of the accounting options object
  * \param options pointer to the accounting options object
  * \param type option type you want to set
@@ -416,7 +461,7 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define EB ((uint64_t) PB*1000)
 
 #define NFACCT_STR_PLAIN_NUMONLY	"%s %s"
-#define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s pkts=%"PRIu64 \
+#define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s fmt=%s,%s pkts=%"PRIu64 \
 					" bytes=%"PRIu64
 #define NFACCT_STR_PLAIN_NEW		"[ pkts = %%%zus" \
 					" bytes = %%%zus ] = %%s"
@@ -695,7 +740,7 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		}
 
 		if (fmt == NFACCT_FMT_MAX)
-			fmt = NFACCT_FMT_DEFAULT;
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
 
 		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
 			init_locale();
@@ -707,11 +752,14 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 				nfacct_name);
 	} else if (flags & NFACCT_SNPRINTF_F_SAVE) {
 		/* save: format useful for 'restore' */
+		fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
 		parse_nfacct_name(nfacct_name,
 				  nfacct_attr_get_str(nfacct,
 						      NFACCT_ATTR_NAME));
 		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_SAVE_BASE,
 			       nfacct_name,
+			       NFACCT_GET_FMT(nfacct_get_pkt_fmt(fmt)),
+			       NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_PKTS),
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BYTES));
 	} else if (flags & NFACCT_SNPRINTF_F_BONLY) {
@@ -723,7 +771,7 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
 
 		if (fmt == NFACCT_FMT_MAX)
-			fmt = NFACCT_FMT_DEFAULT;
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
 
 		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
 			init_locale();
@@ -739,7 +787,7 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
 
 		if (fmt == NFACCT_FMT_MAX)
-			fmt = NFACCT_FMT_DEFAULT;
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
 
 		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
 			init_locale();
@@ -824,7 +872,7 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
 
 		if (fmt == NFACCT_FMT_MAX)
-			fmt = NFACCT_FMT_DEFAULT;
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
 
 		format_number(&bn, bytes, NFACCT_FMT_NONE);
 		ret = snprintf(buf, rem, NFACCT_STR_XML_BONLY, nfacct_name,
@@ -837,7 +885,7 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
 
 		if (fmt == NFACCT_FMT_MAX)
-			fmt = NFACCT_FMT_DEFAULT;
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
 
 		format_number(&pn, pkts, NFACCT_FMT_NONE);
 		format_number(&bn, bytes, NFACCT_FMT_NONE);
@@ -999,6 +1047,9 @@ void nfacct_nlmsg_build_payload(struct nlmsghdr *nlh, struct nfacct *nfacct)
 
 	if (nfacct->bitset & (1 << NFACCT_ATTR_BYTES))
 		mnl_attr_put_u64(nlh, NFACCT_BYTES, htobe64(nfacct->bytes));
+
+	if (nfacct->bitset & (1 << NFACCT_ATTR_FMT))
+		mnl_attr_put_u16(nlh, NFACCT_FMT, htobe16(nfacct->fmt));
 }
 EXPORT_SYMBOL(nfacct_nlmsg_build_payload);
 
@@ -1024,6 +1075,12 @@ static int nfacct_nlmsg_parse_attr_cb(const struct nlattr *attr, void *data)
 			return MNL_CB_ERROR;
 		}
 		break;
+	case NFACCT_FMT:
+		if (mnl_attr_validate(attr, MNL_TYPE_U16) < 0) {
+			perror("mnl_attr_validate");
+			return MNL_CB_ERROR;
+		}
+		break;
 	}
 	tb[type] = attr;
 	return MNL_CB_OK;
@@ -1053,6 +1110,9 @@ nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct)
 			    be64toh(mnl_attr_get_u64(tb[NFACCT_PKTS])));
 	nfacct_attr_set_u64(nfacct, NFACCT_ATTR_BYTES,
 			    be64toh(mnl_attr_get_u64(tb[NFACCT_BYTES])));
+	if (tb[NFACCT_FMT])
+		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FMT,
+				  be16toh(mnl_attr_get_u16(tb[NFACCT_FMT])));
 
 	return 0;
 }
diff --git a/src/libnetfilter_acct.map b/src/libnetfilter_acct.map
index ded60a9..6554090 100644
--- a/src/libnetfilter_acct.map
+++ b/src/libnetfilter_acct.map
@@ -21,6 +21,8 @@ local: *;
 
 LIBNETFILTER_ACCT_1.1 {
   nfacct_snprintf;
+  nfacct_attr_set_u16;
+  nfacct_attr_get_u16;
   nfacct_options_alloc;
   nfacct_options_free;
   nfacct_option_set;
-- 
1.8.3.1

[PATCH v3 nfacct 18/29] add permanent number formatting to nfacct objects

[Michael Zintakis]

* add support for permanent number formatting of nfacct objects, enhancing
the "add" and "get" commands, modifying the exisitng "save" and "restore"
commands also. That way, each nfacct object can have its own independent
formatting, which is stored permanently when the object was created with
"add". That formatting is then used in the "list" and "get" commands, unless
the "format" option is specified. More than 14 different formats can be used
to format both bytes and packets. The formatting of packets and bytes is also
independent and can be specified separately.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/linux/netfilter/nfnetlink_acct.h |  1 +
 src/nfacct.c                             | 43 +++++++++++++++++++++++++++-----
 2 files changed, 38 insertions(+), 6 deletions(-)

diff --git a/include/linux/netfilter/nfnetlink_acct.h b/include/linux/netfilter/nfnetlink_acct.h
index 7c4279b..4c4ce88 100644
--- a/include/linux/netfilter/nfnetlink_acct.h
+++ b/include/linux/netfilter/nfnetlink_acct.h
@@ -19,6 +19,7 @@ enum nfnl_acct_type {
 	NFACCT_PKTS,
 	NFACCT_BYTES,
 	NFACCT_USE,
+	NFACCT_FMT,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
diff --git a/src/nfacct.c b/src/nfacct.c
index 3f25334..e2f5a79 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -439,7 +439,9 @@ static int nfacct_cmd_add(int argc, char *argv[])
 {
 	int ret = -1;
 	bool replace = false;
+	bool b_fmt = false;
 	char *name;
+	uint16_t fmt = NFACCT_FMT_MAX;
 	struct nfacct *nfacct;
 
 	if (argc < 1 || strlen(argv[0]) == 0) {
@@ -455,6 +457,14 @@ static int nfacct_cmd_add(int argc, char *argv[])
 	while (argc > 0) {
 		if (!replace && nfacct_matches(argv[0],"replace")) {
 			replace = true;
+		} else if (!b_fmt && (nfacct_matches(argv[0],"format") ||
+			   nfacct_matches(argv[0],"fmt"))) {
+			NFACCT_GET_NEXT_ARG();
+			fmt = nfacct_parse_format_options(argv[0]);
+			if (fmt == NFACCT_FMT_MAX) {
+				NFACCT_RET_ARG_ERR();
+			}
+			b_fmt = true;
 		} else {
 			NFACCT_RET_ARG_ERR();
 		}
@@ -467,6 +477,10 @@ static int nfacct_cmd_add(int argc, char *argv[])
 	}
 
 	nfacct_attr_set(nfacct, NFACCT_ATTR_NAME, name);
+
+	if (fmt != NFACCT_FMT_MAX)
+		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FMT, fmt);
+
 	ret = _nfacct_cmd_add(nfacct, replace);
 
 err:
@@ -749,7 +763,7 @@ static const char help_msg[] =
 	"Parameters:\n"
 	"  LST_PARAMS := [ reset ] [ show bytes ] [ format FMT_SPEC ]\n"
 			"\t\t[ sort SORT_SPEC ] [ xml ]\n"
-	"  ADD_PARAMS := [ replace ]\n"
+	"  ADD_PARAMS := [ replace ] [ format FMT_SPEC ]\n"
 	"  GET_PARAMS := [ reset ] [ show bytes ] [ format FMT_SPEC ] [ xml ]\n"
 	"  RST_PARAMS := [ flush ] [ replace ]\n"
 	"  FMT_SPEC := { [FMT] | [,] | [FMT] ... }\n"
@@ -838,10 +852,10 @@ err:
 
 /*
  * Maximum number of restore tokens accepted:
- * name= pkts= bytes=
+ * name= fmt= pkts= bytes=
  *
  */
-#define NFACCT_MAX_TOKENS 3
+#define NFACCT_MAX_TOKENS 4
 
 /*
  * Maximum number of value tokens accepted:
@@ -871,7 +885,8 @@ err:
 static int nfacct_cmd_restore(int argc, char *argv[])
 {
 	bool replace = false, flush = false;
-	bool b_name = false, b_pkts = false, b_bytes = false;
+	bool b_name = false, b_fmt = false, b_pkts = false, b_bytes = false;
+	uint16_t fmt = NFACCT_FMT_DEFAULT;
 	uint64_t pkts = 0, bytes = 0;
 	char *tokens[NFACCT_MAX_TOKENS + 1];
 	char *vtokens[NFACCT_MAX_VTOKENS + 1];
@@ -904,8 +919,9 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 		}
 	}
 
-	for (; fgets(buf, ARRAY_SIZE(buf), stdin); pkts = 0, bytes = 0,
-	     b_name = false, b_pkts = false, b_bytes = false, line++) {
+	for (; fgets(buf, ARRAY_SIZE(buf), stdin); fmt = NFACCT_FMT_DEFAULT,
+	     pkts = 0, bytes = 0, b_name = false, b_fmt = false,
+	     b_pkts = false, b_bytes = false, line++) {
 		ret = nfacct_parse_tokens(buf, " \n", NFACCT_MAX_TOKENS + 1,
 					  true, tokens);
 		if (ret == 0)
@@ -939,6 +955,17 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 				nfacct_attr_set(nfacct, NFACCT_ATTR_NAME,
 						vtokens[1]);
 				b_name = true;
+			} else if (!b_fmt && strncmp(vtokens[0], "fmt",
+						    strlen("fmt") + 1) == 0) {
+				fmt = nfacct_parse_format_options(vtokens[1]);
+				if (fmt == NFACCT_FMT_MAX) {
+					NFACCT_PRINT_VERR("error on line %d: "
+						"invalid 'format' token (%s)",
+						vtokens[1]);
+				}
+				nfacct_attr_set_u16(nfacct,
+						    NFACCT_ATTR_FMT, fmt);
+				b_fmt = true;
 			} else if (!b_pkts && strncmp(vtokens[0], "pkts",
 						   strlen("pkts") + 1) == 0) {
 				if (nfacct_get_uint64_t(&pkts,
@@ -988,6 +1015,10 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 		}
 		NFACCT_FREE_TOKENS;
 
+		if (!b_fmt)
+			nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FMT,
+					    NFACCT_FMT_DEFAULT);
+
 		ret = _nfacct_cmd_add(nfacct, replace);
 		if (ret != 0) {
 			NFACCT_SNPRINTF("error on line %d: "
-- 
1.8.3.1

[PATCH v3 kernel 19/29] add byte threshold capability to nfacct

[Michael Zintakis]

* add a 'bthr' variable to each nfacct object, allowing a bytes 'threshold'
to be stored and then reported if/when traffic breaches it.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/uapi/linux/netfilter/nfnetlink_acct.h |  9 ++++++
 net/netfilter/nfnetlink_acct.c                | 43 ++++++++++++++++++++++++++-
 2 files changed, 51 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/netfilter/nfnetlink_acct.h b/include/uapi/linux/netfilter/nfnetlink_acct.h
index 0b65f9c1..e972970 100644
--- a/include/uapi/linux/netfilter/nfnetlink_acct.h
+++ b/include/uapi/linux/netfilter/nfnetlink_acct.h
@@ -19,10 +19,19 @@ enum nfnl_acct_type {
 	NFACCT_PKTS,
 	NFACCT_BYTES,
 	NFACCT_USE,
+	NFACCT_BTHR,
 	NFACCT_FMT,
+	NFACCT_FLAGS,
+	NFACCT_CMD,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
 
+enum nfnl_acct_flags {
+	NFACCT_FLAG_BIT_BTHR 	= 0,
+	NFACCT_FLAG_BTHR	= (1 << NFACCT_FLAG_BIT_BTHR),
+	NFACCT_FLAG_BIT_MAX	= 1,
+	NFACCT_FLAG_MAX		= (1 << NFACCT_FLAG_BIT_MAX),
+};
 
 #endif /* _UAPI_NFNL_ACCT_H_ */
diff --git a/net/netfilter/nfnetlink_acct.c b/net/netfilter/nfnetlink_acct.c
index 92ecad1..18cd28e 100644
--- a/net/netfilter/nfnetlink_acct.c
+++ b/net/netfilter/nfnetlink_acct.c
@@ -32,7 +32,9 @@ static LIST_HEAD(nfnl_acct_list);
 struct nf_acct {
 	atomic64_t		pkts;
 	atomic64_t		bytes;
+	u64			bthr;
 	u16			fmt;
+	u16			flags;
 	struct list_head	head;
 	atomic_t		refcnt;
 	char			name[NFACCT_NAME_MAX];
@@ -44,6 +46,7 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 	     const struct nlmsghdr *nlh, const struct nlattr * const tb[])
 {
 	struct nf_acct *nfacct, *matching = NULL;
+	unsigned int flags = 0, cmd = 0;
 	char *acct_name;
 
 	if (!tb[NFACCT_NAME] ||
@@ -51,6 +54,21 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 	    (!tb[NFACCT_BYTES] && tb[NFACCT_PKTS]))
 		return -EINVAL;
 
+	if (tb[NFACCT_CMD]) {
+		if (!tb[NFACCT_FLAGS])
+			return -EINVAL;
+
+		cmd = be16_to_cpu(nla_get_be16(tb[NFACCT_CMD]));
+		flags = be16_to_cpu(nla_get_be16(tb[NFACCT_FLAGS]));
+
+		if (cmd & NFACCT_FLAG_BTHR &&
+		    ((flags & NFACCT_FLAG_BTHR && !tb[NFACCT_BTHR]) ||
+		     (!(flags & NFACCT_FLAG_BTHR) && tb[NFACCT_BTHR])))
+			return -EINVAL;
+
+	} else if (tb[NFACCT_FLAGS])
+		return -EINVAL;
+
 	acct_name = nla_data(tb[NFACCT_NAME]);
 	if (strlen(acct_name) == 0)
 		return -EINVAL;
@@ -86,7 +104,7 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 				 * and == 0.
 				 *
 				 */
-				if (!tb[NFACCT_FMT]) {
+				if (!tb[NFACCT_FMT] && !cmd) {
 					atomic64_set(&matching->pkts, 0);
 					atomic64_set(&matching->bytes, 0);
 				}
@@ -96,6 +114,17 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 				matching->fmt =
 				be16_to_cpu(nla_get_be16(tb[NFACCT_FMT]));
 			}
+			/* ... and finally set the bytes threshold */
+			if (cmd & NFACCT_FLAG_BTHR) {
+				if (flags & NFACCT_FLAG_BTHR) {
+					matching->bthr = be64_to_cpu(
+					  nla_get_be64(tb[NFACCT_BTHR]));
+					matching->flags |= NFACCT_FLAG_BTHR;
+				} else {
+					matching->bthr = 0;
+					matching->flags &= ~NFACCT_FLAG_BTHR;
+				}
+			}
 			return 0;
 		}
 		return -EBUSY;
@@ -118,6 +147,13 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 	if (tb[NFACCT_FMT]) {
 		nfacct->fmt = be16_to_cpu(nla_get_be16(tb[NFACCT_FMT]));
 	}
+	if (cmd & NFACCT_FLAG_BTHR && flags & NFACCT_FLAG_BTHR) {
+		if (tb[NFACCT_BTHR])
+			nfacct->bthr = be64_to_cpu(
+				nla_get_be64(tb[NFACCT_BTHR]));
+
+		nfacct->flags |= flags & NFACCT_FLAG_BTHR;
+	}
 	atomic_set(&nfacct->refcnt, 1);
 	list_add_tail_rcu(&nfacct->head, &nfnl_acct_list);
 	return 0;
@@ -154,7 +190,9 @@ nfnl_acct_fill_info(struct sk_buff *skb, u32 portid, u32 seq, u32 type,
 	}
 	if (nla_put_be64(skb, NFACCT_PKTS, cpu_to_be64(pkts)) ||
 	    nla_put_be64(skb, NFACCT_BYTES, cpu_to_be64(bytes)) ||
+	    nla_put_be64(skb, NFACCT_BTHR, cpu_to_be64(acct->bthr)) ||
 	    nla_put_be16(skb, NFACCT_FMT, htons(acct->fmt)) ||
+	    nla_put_be16(skb, NFACCT_FLAGS, htons(acct->flags)) ||
 	    nla_put_be32(skb, NFACCT_USE, htonl(atomic_read(&acct->refcnt))))
 		goto nla_put_failure;
 
@@ -302,7 +340,10 @@ static const struct nla_policy nfnl_acct_policy[NFACCT_MAX+1] = {
 	[NFACCT_NAME] = { .type = NLA_NUL_STRING, .len = NFACCT_NAME_MAX-1 },
 	[NFACCT_BYTES] = { .type = NLA_U64 },
 	[NFACCT_PKTS] = { .type = NLA_U64 },
+	[NFACCT_BTHR] = { .type = NLA_U64 },
 	[NFACCT_FMT] = { .type = NLA_U16 },
+	[NFACCT_FLAGS] = { .type = NLA_U16 },
+	[NFACCT_CMD] = { .type = NLA_U16 },
 };
 
 static const struct nfnl_callback nfnl_acct_cb[NFNL_MSG_ACCT_MAX] = {
-- 
1.8.3.1

Re: [PATCH v3 kernel 19/29] add byte threshold capability to nfacct

[Florian Westphal]

Michael Zintakis <michael.zintakis@googlemail.com> wrote:
> * add a 'bthr' variable to each nfacct object, allowing a bytes 'threshold'
> to be stored and then reported if/when traffic breaches it.

Again, why is this needed?
Why is it useful?

Re: [PATCH v3 kernel 19/29] add byte threshold capability to nfacct

[Michael Zintakis]

Florian Westphal wrote:
> Michael Zintakis <michael.zintakis@googlemail.com> wrote:
>> * add a 'bthr' variable to each nfacct object, allowing a bytes 'threshold'
>> to be stored and then reported if/when traffic breaches it.
> 
> Again, why is this needed?
> Why is it useful?
This is used for measuring traffic "expectancy", i.e. allows one to be able to register what amount of traffic is "expected" to pass through this accounting object. If that traffic threshold is exceeded, this is properly indicated when the accounting object is listed or any statistics for that object are being collected by the nfacct daemon.

That traffic "expectancy" can be set/reset depending on the nature of the traffic or its source/destination etc, so it is pretty flexible. Again, there is extensive information on this in the (revised) man page if you decide to look at it.

Re: [PATCH v3 kernel 19/29] add byte threshold capability to nfacct

[Florian Westphal]

Michael Zintakis <michael.zintakis@googlemail.com> wrote:
> Florian Westphal wrote:
> > Michael Zintakis <michael.zintakis@googlemail.com> wrote:
> >> * add a 'bthr' variable to each nfacct object, allowing a bytes 'threshold'
> >> to be stored and then reported if/when traffic breaches it.
> > 
> > Again, why is this needed?
> > Why is it useful?
> This is used for measuring traffic "expectancy", i.e. allows one to be able to register what amount of traffic is "expected" to pass through this accounting object. If that traffic threshold is exceeded, this is properly indicated when the accounting object is listed or any statistics for that object are being collected by the nfacct daemon.
> 
> That traffic "expectancy" can be set/reset depending on the nature of the traffic or its source/destination etc, so it is pretty flexible. Again, there is extensive information on this in the (revised) man page if you decide to look at it.

I still don't understand why this needs to be in the kernel.
nfacct gives you the counters, how these are interpreted (e.g. 'higher
than expected' should be entirely up to userspace).

In case you need some way of reacting to excess counters, then perhaps
it makes sense to change nfacct match to allow "greater/less than"
matching expression instead?

E.g.:
-A bla -m nfacct --nfacct-name bla --nfacct-packets 1000000: -m limit
--limit 1/hour -j NFLOG --nflog-prefix 'bla packet threshold'

or something like that?

There is something similar for the conntrack accounting (-m connbytes).

RE: [PATCH v3 kernel 19/29] add byte threshold capability to nfacct

[Alexey Perevalov]

> Date: Thu, 11 Jul 2013 22:25:47 +0200
> From: fw@strlen.de
> To: michael.zintakis@googlemail.com
> CC: fw@strlen.de; netfilter-devel@vger.kernel.org; pablo@netfilter.org
> Subject: Re: [PATCH v3 kernel 19/29] add byte threshold capability to nfacct
> 
> Michael Zintakis <michael.zintakis@googlemail.com> wrote:
>> Florian Westphal wrote:
>>> Michael Zintakis <michael.zintakis@googlemail.com> wrote:
>>>> * add a 'bthr' variable to each nfacct object, allowing a bytes 'threshold'
>>>> to be stored and then reported if/when traffic breaches it.
>>> 
>>> Again, why is this needed?
>>> Why is it useful?
>> This is used for measuring traffic "expectancy", i.e. allows one to be able to register what amount of traffic is "expected" to pass through this accounting object. If that traffic threshold is exceeded, this is properly indicated when the accounting object is listed or any statistics for that object are being collected by the nfacct daemon.
>> 
>> That traffic "expectancy" can be set/reset depending on the nature of the traffic or its source/destination etc, so it is pretty flexible. Again, there is extensive information on this in the (revised) man page if you decide to look at it.
> 
> I still don't understand why this needs to be in the kernel.
> nfacct gives you the counters, how these are interpreted (e.g. 'higher
> than expected' should be entirely up to userspace).
> 
I also vote for this patch.
I'll try here to describe our use case.
We checking counter every minute, why not to check it more often? It's possible and doesn't lead to huge performance problems, but we want to save battery power and we don't want such big resolution in measurements. But also we need to restrict traffic according to predefined user quota. To not exceed such restriction we need to deligate such responsobility to kernel.
Now I talking only about wireless connection, but even on 3G it's possible to download more than 50Mb per one minute.
Also we need to inform user about quota comming beforehand. Based on given kernel couters update time interval, quota value and bandwidth I came to conclusion what it's better to have some warning threshold for informing user space from kernel.
I predict here comment about none general use case :)

I thought to add it to xt_quota, but nfacct it's better place.


> In case you need some way of reacting to excess counters, then perhaps
> it makes sense to change nfacct match to allow "greater/less than"
> matching expression instead?
> 
> E.g.:
> -A bla -m nfacct --nfacct-name bla --nfacct-packets 1000000: -m limit
> --limit 1/hour -j NFLOG --nflog-prefix 'bla packet threshold'
> 
> or something like that?
It's fit too.

P.S.  we don't use neither nfacct now, nor xt_quota due we counting based on net_cls (cls_cgroup) marks and as I understand it's not possible for incomming traffic without netfilter refactoring.
> 
> There is something similar for the conntrack accounting (-m connbytes).
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html 		 	   		  --
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v3 libnetfilter_acct 20/29] add byte threshold capability support

[Michael Zintakis]

* add support for printing/retrieving byte threshold of nfacct objects,
using the 'bthr' value stored in each such object within the kernel;

* enhance the existing API to support get/set byte threshold functions;

* adjust all existing xml and plaintext templates to support this feature;

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/libnetfilter_acct/libnetfilter_acct.h |   4 +
 include/linux/netfilter/nfnetlink_acct.h      |   9 ++
 src/libnetfilter_acct.c                       | 149 ++++++++++++++++++++++----
 3 files changed, 140 insertions(+), 22 deletions(-)

diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index 0944247..920df9d 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -14,7 +14,10 @@ enum nfacct_attr_type {
 	NFACCT_ATTR_NAME = 0,
 	NFACCT_ATTR_PKTS,
 	NFACCT_ATTR_BYTES,
+	NFACCT_ATTR_BTHR,
 	NFACCT_ATTR_FMT,
+	NFACCT_ATTR_FLAGS,
+	NFACCT_ATTR_CMD,
 };
 
 struct nfacct_options;
@@ -23,6 +26,7 @@ enum nfacct_option_type {
 	NFACCT_OPT_FMT = 0,	/* number format option 	*/
 	NFACCT_OPT_PCW,		/* packets count column width 	*/
 	NFACCT_OPT_BCW,		/* bytes count column width 	*/
+	NFACCT_OPT_BTCW,	/* bytes threshold column width */
 };
 
 enum nfacct_format {
diff --git a/include/linux/netfilter/nfnetlink_acct.h b/include/linux/netfilter/nfnetlink_acct.h
index 0b65f9c..e972970 100644
--- a/include/linux/netfilter/nfnetlink_acct.h
+++ b/include/linux/netfilter/nfnetlink_acct.h
@@ -19,10 +19,19 @@ enum nfnl_acct_type {
 	NFACCT_PKTS,
 	NFACCT_BYTES,
 	NFACCT_USE,
+	NFACCT_BTHR,
 	NFACCT_FMT,
+	NFACCT_FLAGS,
+	NFACCT_CMD,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
 
+enum nfnl_acct_flags {
+	NFACCT_FLAG_BIT_BTHR 	= 0,
+	NFACCT_FLAG_BTHR	= (1 << NFACCT_FLAG_BIT_BTHR),
+	NFACCT_FLAG_BIT_MAX	= 1,
+	NFACCT_FLAG_MAX		= (1 << NFACCT_FLAG_BIT_MAX),
+};
 
 #endif /* _UAPI_NFNL_ACCT_H_ */
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index f7890a1..db01c31 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -61,11 +61,14 @@ struct nfacct {
 	char		name[NFACCT_NAME_MAX];
 	uint64_t	pkts;
 	uint64_t	bytes;
+	uint64_t	bthr;
 	/*
 	 * Structure of fmt:
 	 * <- packets format: 8 bits-><-bytes format: 8 bits ->
 	 */
 	uint16_t	fmt;
+	uint16_t	flags;
+	uint16_t	cmd;
 	uint32_t	bitset;
 };
 
@@ -73,6 +76,7 @@ struct nfacct_options {
 	uint16_t	fmt;
 	size_t		pcw;
 	size_t		bcw;
+	size_t		btcw;
 	uint32_t	bitset;
 };
 
@@ -149,10 +153,22 @@ nfacct_attr_set(struct nfacct *nfacct, enum nfacct_attr_type type,
 		nfacct->bytes = *((uint64_t *) data);
 		nfacct->bitset |= (1 << NFACCT_ATTR_BYTES);
 		break;
+	case NFACCT_ATTR_BTHR:
+		nfacct->bthr = *((uint64_t *) data);
+		nfacct->bitset |= (1 << NFACCT_ATTR_BTHR);
+		break;
 	case NFACCT_ATTR_FMT:
 		nfacct->fmt = *((uint16_t *) data);
 		nfacct->bitset |= (1 << NFACCT_ATTR_FMT);
 		break;
+	case NFACCT_ATTR_FLAGS:
+		nfacct->flags = *((uint16_t *) data);
+		nfacct->bitset |= (1 << NFACCT_ATTR_FLAGS);
+		break;
+	case NFACCT_ATTR_CMD:
+		nfacct->cmd = *((uint16_t *) data);
+		nfacct->bitset |= (1 << NFACCT_ATTR_CMD);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_attr_set);
@@ -217,9 +233,18 @@ nfacct_attr_unset(struct nfacct *nfacct, enum nfacct_attr_type type)
 	case NFACCT_ATTR_BYTES:
 		nfacct->bitset &= ~(1 << NFACCT_ATTR_BYTES);
 		break;
+	case NFACCT_ATTR_BTHR:
+		nfacct->bitset &= ~(1 << NFACCT_ATTR_BTHR);
+		break;
 	case NFACCT_ATTR_FMT:
 		nfacct->bitset &= ~(1 << NFACCT_ATTR_FMT);
 		break;
+	case NFACCT_ATTR_FLAGS:
+		nfacct->bitset &= ~(1 << NFACCT_ATTR_FLAGS);
+		break;
+	case NFACCT_ATTR_CMD:
+		nfacct->bitset &= ~(1 << NFACCT_ATTR_CMD);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_attr_unset);
@@ -249,10 +274,22 @@ const void *nfacct_attr_get(struct nfacct *nfacct, enum nfacct_attr_type type)
 		if (nfacct->bitset & (1 << NFACCT_ATTR_BYTES))
 			ret = &nfacct->bytes;
 		break;
+	case NFACCT_ATTR_BTHR:
+		if (nfacct->bitset & (1 << NFACCT_ATTR_BTHR))
+			ret = &nfacct->bthr;
+		break;
 	case NFACCT_ATTR_FMT:
 		if (nfacct->bitset & (1 << NFACCT_ATTR_FMT))
 			ret = &nfacct->fmt;
 		break;
+	case NFACCT_ATTR_FLAGS:
+		if (nfacct->bitset & (1 << NFACCT_ATTR_FLAGS))
+			ret = &nfacct->flags;
+		break;
+	case NFACCT_ATTR_CMD:
+		if (nfacct->bitset & (1 << NFACCT_ATTR_CMD))
+			ret = &nfacct->cmd;
+		break;
 	}
 	return ret;
 }
@@ -327,6 +364,10 @@ nfacct_option_set(struct nfacct_options *options,
 		options->bcw = *((size_t *) data);
 		options->bitset |= (1 << NFACCT_OPT_BCW);
 		break;
+	case NFACCT_OPT_BTCW:
+		options->btcw = *((size_t *) data);
+		options->bitset |= (1 << NFACCT_OPT_BTCW);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_option_set);
@@ -380,6 +421,9 @@ nfacct_option_unset(struct nfacct_options *options,
 	case NFACCT_OPT_BCW:
 		options->bitset &= ~(1 << NFACCT_OPT_BCW);
 		break;
+	case NFACCT_OPT_BTCW:
+		options->bitset &= ~(1 << NFACCT_OPT_BTCW);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_option_unset);
@@ -410,6 +454,10 @@ const void *nfacct_option_get(struct nfacct_options *options,
 		if (options->bitset & (1 << NFACCT_OPT_BCW))
 			ret = &options->bcw;
 		break;
+	case NFACCT_OPT_BTCW:
+		if (options->bitset & (1 << NFACCT_OPT_BTCW))
+			ret = &options->btcw;
+		break;
 	}
 	return ret;
 }
@@ -460,17 +508,19 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define PB ((uint64_t) TB*1000)
 #define EB ((uint64_t) PB*1000)
 
-#define NFACCT_STR_PLAIN_NUMONLY	"%s %s"
+#define NFACCT_STR_PLAIN_NUMONLY	"%s %s %s"
 #define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s fmt=%s,%s pkts=%"PRIu64 \
 					" bytes=%"PRIu64
+#define NFACCT_STR_PLAIN_SAVE_BTHR	" thr=%"PRIu64
 #define NFACCT_STR_PLAIN_NEW		"[ pkts = %%%zus" \
-					" bytes = %%%zus ] = %%s"
+					" bytes = %%%zus%s ] = %%s"
 #define NFACCT_STR_PLAIN		"{ pkts = %%%zus, " \
-					"bytes = %%%zus } = %%s;"
-#define NFACCT_STR_PLAIN_BONLY		"[ bytes = %%%zus ] = %%s"
+					"bytes = %%%zus%s } = %%s;"
+#define NFACCT_STR_PLAIN_BONLY		"[ bytes = %%%zus%s ] = %%s"
 #define NFACCT_XML_NAME			"<name>%s</name>"
 #define NFACCT_XML_PKTS			"<pkts fmt=\"%s\">%s</pkts>"
-#define NFACCT_XML_BYTES		"<bytes fmt=\"%s\">%s</bytes>"
+#define NFACCT_XML_BYTES		"<bytes fmt=\"%s\" " \
+					"thr=\"%s\">%s</bytes>"
 #define NFACCT_STR_XML_BONLY		"<obj>"	NFACCT_XML_NAME \
 					NFACCT_XML_BYTES
 #define NFACCT_STR_XML_COMPAT		"<obj><name>%s</name>"	\
@@ -484,6 +534,7 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define NFACCT_STR_SI_IEC		"%'.3f%s"
 
 #define NFACCT_NUM_DEFAULT		{ .value = 0., .str = "" }
+#define NFACCT_THR_DEFAULT		{ .value = 0., .str = "-" }
 
 struct nfacct_number {
 	float value;
@@ -703,24 +754,38 @@ static void init_locale(void) {
 #define nfacct_get_bytes_fmt(x)		_nfacct_get_value(x,bytes)
 #define nfacct_get_pkt_fmt(x)		_nfacct_get_value(x,pkts)
 
+#define NFACCT_PRINT_FL(flag,x,v)	((flag && x > v) ? "+" : " ")
+#define NFACCT_PRINT_FL_XML(flag,x,v)	((flag ? ((x > v) ? \
+					"true" : "false") : ""))
+
+#define BUFFER_SIZE(ret, size, rem, offset)		\
+	size += ret;					\
+	if (ret > rem)					\
+		ret = rem;				\
+	offset += ret;					\
+	rem -= ret;
+
 static int
 nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		      uint16_t flags, struct nfacct_options *options)
 {
-	int ret = 0;
+	int ret = 0, size = 0, offset = 0;
 	bool compat = (options == NULL);
-	uint16_t fmt;
-	uint64_t pkts = 0, bytes = 0;
+	uint16_t fmt, fl;
+	uint64_t pkts = 0, bytes = 0, thr = 0;
 	char nfacct_name[NFACCT_NAME_MAX * 2 + 4];
 	char fmt_str[sizeof(NFACCT_STR_PLAIN_NEW) +
 		     sizeof(NFACCT_STR_DEFAULT) * 5 + 10];
 	struct nfacct_number pn = NFACCT_NUM_DEFAULT,
-			     bn = NFACCT_NUM_DEFAULT;
+			     bn = NFACCT_NUM_DEFAULT,
+			     tn = NFACCT_THR_DEFAULT;
 
 	if (flags & NFACCT_SNPRINTF_F_FULL) {
 		/* default: print pkts + bytes + name */
 		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
 		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		thr = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BTHR);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
 		parse_nfacct_name(nfacct_name,
 				  nfacct_attr_get_str(nfacct,
 						      NFACCT_ATTR_NAME));
@@ -728,7 +793,7 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		if (compat) {
 			fmt = NFACCT_FMT_MAX;
 			snprintf(fmt_str, sizeof(fmt_str), NFACCT_STR_PLAIN,
-				 (size_t)0, (size_t)0);
+				 (size_t)0, (size_t)0, "");
 		} else {
 			fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
 			snprintf(fmt_str, sizeof(fmt_str),
@@ -736,7 +801,9 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 				 nfacct_option_get_tsize(options,
 							 NFACCT_OPT_PCW),
 				 nfacct_option_get_tsize(options,
-							 NFACCT_OPT_BCW));
+							 NFACCT_OPT_BCW),
+				 NFACCT_PRINT_FL(fl & NFACCT_FLAG_BTHR,
+						 bytes,thr));
 		}
 
 		if (fmt == NFACCT_FMT_MAX)
@@ -762,9 +829,19 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 			       NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_PKTS),
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BYTES));
+		BUFFER_SIZE(ret, size, rem, offset);
+
+		fl = nfacct_attr_get_u16(nfacct,NFACCT_ATTR_FLAGS);
+		if (fl & NFACCT_FLAG_BTHR) {
+			ret = snprintf(buf+offset, rem, NFACCT_STR_PLAIN_SAVE_BTHR,
+			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BTHR));
+			BUFFER_SIZE(ret, size, rem, offset);
+		}
 	} else if (flags & NFACCT_SNPRINTF_F_BONLY) {
 		/* print bytes + name only */
 		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		thr = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BTHR);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
 		parse_nfacct_name(nfacct_name,
 				nfacct_attr_get_str(nfacct,
 						    NFACCT_ATTR_NAME));
@@ -778,12 +855,15 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 
 		format_number(&bn, bytes, nfacct_get_bytes_fmt(fmt));
 		snprintf(fmt_str, sizeof(fmt_str), NFACCT_STR_PLAIN_BONLY,
-			 nfacct_option_get_tsize(options,NFACCT_OPT_BCW));
+			 nfacct_option_get_tsize(options,NFACCT_OPT_BCW),
+			 NFACCT_PRINT_FL(fl & NFACCT_FLAG_BTHR, bytes, thr));
 		ret = snprintf(buf, rem, fmt_str, bn.str, nfacct_name);
 	} else if (flags & NFACCT_SNPRINTF_F_NUMONLY) {
 		/* numbers only: to determine the maximum column width */
 		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
 		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		thr = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BTHR);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
 		fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
 
 		if (fmt == NFACCT_FMT_MAX)
@@ -795,8 +875,11 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		format_number(&pn, pkts, nfacct_get_pkt_fmt(fmt));
 		format_number(&bn, bytes, nfacct_get_bytes_fmt(fmt));
 
+		if (fl & NFACCT_FLAG_BTHR)
+			format_number(&tn, thr, nfacct_get_bytes_fmt(fmt));
+
 		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_NUMONLY,
-				pn.str, bn.str);
+				pn.str, bn.str, tn.str);
 	} else {
 		/* print out name only */
 		parse_nfacct_name(nfacct_name,
@@ -808,13 +891,6 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 	return ret;
 }
 
-#define BUFFER_SIZE(ret, size, rem, offset)		\
-	size += ret;					\
-	if (ret > rem)					\
-		ret = rem;				\
-	offset += ret;					\
-	rem -= ret;
-
 static int
 nfacct_snprintf_xml_localtime(char *buf, unsigned int rem, const struct tm *tm)
 {
@@ -853,8 +929,8 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 	int ret = 0;
 	bool compat = (options == NULL);
 	unsigned int size = 0, offset = 0;
-	uint16_t fmt;
-	uint64_t pkts = 0, bytes = 0;
+	uint16_t fmt, fl;
+	uint64_t pkts = 0, bytes = 0, thr = 0;
 	char nfacct_name[NFACCT_NAME_MAX * 6 + 1];
 	struct nfacct_number pn = NFACCT_NUM_DEFAULT,
 			     bn = NFACCT_NUM_DEFAULT;
@@ -870,6 +946,8 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 	if (flags & NFACCT_SNPRINTF_F_BONLY) {
 		/* print name + bytes only */
 		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		thr = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BTHR);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
 
 		if (fmt == NFACCT_FMT_MAX)
 			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
@@ -877,12 +955,16 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 		format_number(&bn, bytes, NFACCT_FMT_NONE);
 		ret = snprintf(buf, rem, NFACCT_STR_XML_BONLY, nfacct_name,
 				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
+				NFACCT_PRINT_FL_XML(fl & NFACCT_FLAG_BTHR,
+						    bytes,thr),
 				bn.str);
 		BUFFER_SIZE(ret, size, rem, offset);
 	} else {
 		/* default/everything else: print name + pkts + bytes */
 		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
 		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		thr = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BTHR);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
 
 		if (fmt == NFACCT_FMT_MAX)
 			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
@@ -898,6 +980,8 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 				NFACCT_GET_FMT(nfacct_get_pkt_fmt(fmt)),
 				pn.str,
 				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
+				NFACCT_PRINT_FL_XML(fl & NFACCT_FLAG_BTHR,
+						    bytes,thr),
 				bn.str);
 		}
 		BUFFER_SIZE(ret, size, rem, offset);
@@ -1048,8 +1132,17 @@ void nfacct_nlmsg_build_payload(struct nlmsghdr *nlh, struct nfacct *nfacct)
 	if (nfacct->bitset & (1 << NFACCT_ATTR_BYTES))
 		mnl_attr_put_u64(nlh, NFACCT_BYTES, htobe64(nfacct->bytes));
 
+	if (nfacct->bitset & (1 << NFACCT_ATTR_BTHR))
+		mnl_attr_put_u64(nlh, NFACCT_BTHR, htobe64(nfacct->bthr));
+
 	if (nfacct->bitset & (1 << NFACCT_ATTR_FMT))
 		mnl_attr_put_u16(nlh, NFACCT_FMT, htobe16(nfacct->fmt));
+
+	if (nfacct->bitset & (1 << NFACCT_ATTR_FLAGS))
+		mnl_attr_put_u16(nlh, NFACCT_FLAGS, htobe16(nfacct->flags));
+
+	if (nfacct->bitset & (1 << NFACCT_ATTR_CMD))
+		mnl_attr_put_u16(nlh, NFACCT_CMD, htobe16(nfacct->cmd));
 }
 EXPORT_SYMBOL(nfacct_nlmsg_build_payload);
 
@@ -1070,12 +1163,15 @@ static int nfacct_nlmsg_parse_attr_cb(const struct nlattr *attr, void *data)
 		break;
 	case NFACCT_PKTS:
 	case NFACCT_BYTES:
+	case NFACCT_BTHR:
 		if (mnl_attr_validate(attr, MNL_TYPE_U64) < 0) {
 			perror("mnl_attr_validate");
 			return MNL_CB_ERROR;
 		}
 		break;
 	case NFACCT_FMT:
+	case NFACCT_FLAGS:
+	case NFACCT_CMD:
 		if (mnl_attr_validate(attr, MNL_TYPE_U16) < 0) {
 			perror("mnl_attr_validate");
 			return MNL_CB_ERROR;
@@ -1110,9 +1206,18 @@ nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct)
 			    be64toh(mnl_attr_get_u64(tb[NFACCT_PKTS])));
 	nfacct_attr_set_u64(nfacct, NFACCT_ATTR_BYTES,
 			    be64toh(mnl_attr_get_u64(tb[NFACCT_BYTES])));
+	if (tb[NFACCT_BTHR])
+		nfacct_attr_set_u64(nfacct, NFACCT_ATTR_BTHR,
+				  be64toh(mnl_attr_get_u64(tb[NFACCT_BTHR])));
 	if (tb[NFACCT_FMT])
 		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FMT,
 				  be16toh(mnl_attr_get_u16(tb[NFACCT_FMT])));
+	if (tb[NFACCT_FLAGS])
+		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FLAGS,
+				be16toh(mnl_attr_get_u16(tb[NFACCT_FLAGS])));
+	if (tb[NFACCT_CMD])
+		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_CMD,
+				be16toh(mnl_attr_get_u16(tb[NFACCT_CMD])));
 
 	return 0;
 }
-- 
1.8.3.1

[PATCH v3 nfacct 21/29] add byte threshold capabilities to nfacct objects

[Michael Zintakis]

* add support for storing byte threshold in nfacct objects permanently and
then reported if/when the value of bytes passing through the accounting object
exceeded it.

* alter the existing "add", "get" and "list" commands to take advantage of
this new feature.

* modify the existing "save" and "restore" commands to include that new
property.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/linux/netfilter/nfnetlink_acct.h | 10 +++++
 src/nfacct.c                             | 72 ++++++++++++++++++++++++++------
 2 files changed, 69 insertions(+), 13 deletions(-)

diff --git a/include/linux/netfilter/nfnetlink_acct.h b/include/linux/netfilter/nfnetlink_acct.h
index 4c4ce88..5d64afa 100644
--- a/include/linux/netfilter/nfnetlink_acct.h
+++ b/include/linux/netfilter/nfnetlink_acct.h
@@ -19,11 +19,21 @@ enum nfnl_acct_type {
 	NFACCT_PKTS,
 	NFACCT_BYTES,
 	NFACCT_USE,
+	NFACCT_BTHR,
 	NFACCT_FMT,
+	NFACCT_FLAGS,
+	NFACCT_CMD,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
 
+enum nfnl_acct_flags {
+	NFACCT_FLAG_BIT_BTHR 	= 0,
+	NFACCT_FLAG_BTHR	= (1 << NFACCT_FLAG_BIT_BTHR),
+	NFACCT_FLAG_BIT_MAX	= 1,
+	NFACCT_FLAG_MAX		= (1 << NFACCT_FLAG_BIT_MAX),
+};
+
 #ifdef __KERNEL__
 
 struct nf_acct;
diff --git a/src/nfacct.c b/src/nfacct.c
index e2f5a79..f2ece72 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -58,6 +58,7 @@ enum nfacct_sort_mode {
 	NFACCT_SORT_NAME,
 	NFACCT_SORT_PKTS,
 	NFACCT_SORT_BYTES,
+	NFACCT_SORT_BTHR,
 };
 
 static int nfacct_cmp(void *priv, struct nfacct_list_head *a, struct nfacct_list_head *b)
@@ -76,6 +77,9 @@ static int nfacct_cmp(void *priv, struct nfacct_list_head *a, struct nfacct_list
 				break;
 	case NFACCT_SORT_BYTES:
 				attr = NFACCT_ATTR_BYTES;
+				break;
+	case NFACCT_SORT_BTHR:
+				attr = NFACCT_ATTR_BTHR;
 	case NFACCT_SORT_NAME:
 				break;
 	default: /* unsorted */
@@ -104,9 +108,9 @@ static int nfacct_cmp(void *priv, struct nfacct_list_head *a, struct nfacct_list
 /*
  * maximum total of columns to be shown, except the
  * "name" column as that is not width-dependent:
- * "pkts bytes"
+ * "pkts bytes bthr"
  */
-#define NFACCT_MAX_COLUMNS 2
+#define NFACCT_MAX_COLUMNS 3
 
 /* stores nfacct options for snprintf_* and nfacct_cb functions */
 static struct nfacct_options *options;
@@ -206,6 +210,7 @@ static int nfacct_cb(const struct nlmsghdr *nlh, void *data)
 	static const enum nfacct_option_type o_num[NFACCT_MAX_COLUMNS] =
 				{ NFACCT_OPT_PCW,
 				  NFACCT_OPT_BCW,
+				  NFACCT_OPT_BTCW,
 				};
 
 	if (ignore_col_width == NULL || options == NULL) {
@@ -297,6 +302,8 @@ static int nfacct_cmd_list(int argc, char *argv[])
 				sort_mode = NFACCT_SORT_PKTS;
 			} else if (nfacct_matches(argv[0],"bytes")) {
 				sort_mode = NFACCT_SORT_BYTES;
+			} else if (nfacct_matches(argv[0],"threshold")) {
+				sort_mode = NFACCT_SORT_BTHR;
 			} else if (nfacct_matches(argv[0],"none")) {
 				sort_mode = NFACCT_SORT_NONE;
 			} else {
@@ -438,10 +445,11 @@ static int _nfacct_cmd_add(struct nfacct *nfacct, bool replace)
 static int nfacct_cmd_add(int argc, char *argv[])
 {
 	int ret = -1;
-	bool replace = false;
+	bool replace = false, bthr_set = false;
 	bool b_fmt = false;
 	char *name;
-	uint16_t fmt = NFACCT_FMT_MAX;
+	uint16_t cmd = 0, flags = 0, fmt = NFACCT_FMT_MAX;
+	uint64_t bthr = 0;
 	struct nfacct *nfacct;
 
 	if (argc < 1 || strlen(argv[0]) == 0) {
@@ -465,6 +473,18 @@ static int nfacct_cmd_add(int argc, char *argv[])
 				NFACCT_RET_ARG_ERR();
 			}
 			b_fmt = true;
+		} else if (!bthr_set &&
+			   nfacct_matches(argv[0],"threshold")) {
+			NFACCT_GET_NEXT_ARG();
+			if (strncmp(argv[0], "-", 2) != 0) {
+				if (nfacct_get_uint64_t(&bthr,
+							argv[0]) != 0) {
+					NFACCT_RET_ARG_ERR();
+				}
+				flags |= NFACCT_FLAG_BTHR;
+			}
+			cmd |= NFACCT_FLAG_BTHR;
+			bthr_set = true;
 		} else {
 			NFACCT_RET_ARG_ERR();
 		}
@@ -481,6 +501,13 @@ static int nfacct_cmd_add(int argc, char *argv[])
 	if (fmt != NFACCT_FMT_MAX)
 		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FMT, fmt);
 
+	if (bthr_set) {
+		if (flags)
+			nfacct_attr_set_u64(nfacct, NFACCT_ATTR_BTHR, bthr);
+
+		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FLAGS, flags);
+		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_CMD, cmd);
+	}
 	ret = _nfacct_cmd_add(nfacct, replace);
 
 err:
@@ -763,11 +790,12 @@ static const char help_msg[] =
 	"Parameters:\n"
 	"  LST_PARAMS := [ reset ] [ show bytes ] [ format FMT_SPEC ]\n"
 			"\t\t[ sort SORT_SPEC ] [ xml ]\n"
-	"  ADD_PARAMS := [ replace ] [ format FMT_SPEC ]\n"
+	"  ADD_PARAMS := [ replace ] [ format FMT_SPEC ] "
+			"[ threshold [NUMBER | '-'] ]\n"
 	"  GET_PARAMS := [ reset ] [ show bytes ] [ format FMT_SPEC ] [ xml ]\n"
 	"  RST_PARAMS := [ flush ] [ replace ]\n"
 	"  FMT_SPEC := { [FMT] | [,] | [FMT] ... }\n"
-	"  SORT_SPEC := { none | name | packets | bytes }"
+	"  SORT_SPEC := { none | name | packets | bytes | threshold }"
 	"  FMT := { def | raw | 3pl | iec | kib | mib | gib | tib | pib |"
 		  " eib |\n"
 	"  \t   si | kb | mb | gb | tb | pb | eb }\n";
@@ -852,10 +880,10 @@ err:
 
 /*
  * Maximum number of restore tokens accepted:
- * name= fmt= pkts= bytes=
+ * name= fmt= pkts= bytes= thr=
  *
  */
-#define NFACCT_MAX_TOKENS 4
+#define NFACCT_MAX_TOKENS 5
 
 /*
  * Maximum number of value tokens accepted:
@@ -886,8 +914,9 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 {
 	bool replace = false, flush = false;
 	bool b_name = false, b_fmt = false, b_pkts = false, b_bytes = false;
-	uint16_t fmt = NFACCT_FMT_DEFAULT;
-	uint64_t pkts = 0, bytes = 0;
+	bool b_thr = false;
+	uint16_t cmd = 0, fmt = NFACCT_FMT_DEFAULT;
+	uint64_t pkts = 0, bytes = 0, thr = 0;
 	char *tokens[NFACCT_MAX_TOKENS + 1];
 	char *vtokens[NFACCT_MAX_VTOKENS + 1];
 	char buf[MAX_TOKEN_SIZE];
@@ -920,8 +949,8 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 	}
 
 	for (; fgets(buf, ARRAY_SIZE(buf), stdin); fmt = NFACCT_FMT_DEFAULT,
-	     pkts = 0, bytes = 0, b_name = false, b_fmt = false,
-	     b_pkts = false, b_bytes = false, line++) {
+	     pkts = 0, bytes = 0, thr = 0, b_name = false, b_fmt = false,
+	     b_pkts = false, b_bytes = false, b_thr = false, line++) {
 		ret = nfacct_parse_tokens(buf, " \n", NFACCT_MAX_TOKENS + 1,
 					  true, tokens);
 		if (ret == 0)
@@ -988,6 +1017,18 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 				nfacct_attr_set_u64(nfacct,
 						    NFACCT_ATTR_BYTES, bytes);
 				b_bytes = true;
+			} else if (!b_thr && strncmp(vtokens[0], "thr",
+					   	strlen("thr") + 1) == 0) {
+				if (nfacct_get_uint64_t(&thr,
+						 vtokens[1]) != 0) {
+					NFACCT_PRINT_VERR("error on line %d: "
+						"invalid 'thr' token (%s)",
+						vtokens[1]);
+				}
+				cmd |= NFACCT_FLAG_BTHR;
+				nfacct_attr_set_u64(nfacct,
+						    NFACCT_ATTR_BTHR, thr);
+				b_thr = true;
 			} else {
 				NFACCT_PRINT_VERR("error on line %d: "
 					  "invalid token '%s'", tokens[j]);
@@ -1007,7 +1048,8 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 					"invalid 'pkts' token (%s)",
 					"not set");
 		}
-		if (nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS) &&
+		if ((nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS) ||
+		     cmd & NFACCT_FLAG_BTHR) &&
 		    !nfacct_attr_get(nfacct, NFACCT_ATTR_BYTES)) {
 			NFACCT_PRINT_CERR("error on line %d: "
 					"invalid 'bytes' token (%s)",
@@ -1015,6 +1057,10 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 		}
 		NFACCT_FREE_TOKENS;
 
+		if (cmd) {
+			nfacct_attr_set_u16(nfacct, NFACCT_ATTR_CMD, cmd);
+			nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FLAGS, cmd);
+		}
 		if (!b_fmt)
 			nfacct_attr_set_u16(nfacct, NFACCT_ATTR_FMT,
 					    NFACCT_FMT_DEFAULT);
-- 
1.8.3.1

[PATCH v3 libnetfilter_acct 22/29] add *_EXTENDED template support

[Michael Zintakis]

* add NFACCT_SNPRINTF_F_EXTENDED plaintext and xml templates, allowing
printing of all nfacct object attriutes in separate columns: name,
packet & byte counters, threshold exceeded indicator and the threshold
value. Example:

[ pkts = 30,430,924  bytes = 17.624GiB+ thr = 15.000GiB ] = "ALL WEB"

<?xml version="1.0" encoding="utf-8"?>
<nfacct>
<obj>
<name>ALL WEB</name>
<pkts fmt="3pl">30430924</pkts>
<bytes fmt="iec" thr="true">18923536384</bytes>
<threshold fmt="iec">16106127360</threshold>
</obj>
</nfacct>

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/libnetfilter_acct/libnetfilter_acct.h |  3 ++
 src/libnetfilter_acct.c                       | 72 ++++++++++++++++++++++++++-
 2 files changed, 73 insertions(+), 2 deletions(-)

diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index 920df9d..e4f21e0 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -124,6 +124,9 @@ int nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct
 /* print numbers only (formatted), useful for determining max column width */
 #define NFACCT_SNPRINTF_F_NUMONLY	(1 << 4)
 
+/* print packets, pmark, bytes, bmark, threshold and name columns */
+#define NFACCT_SNPRINTF_F_EXTENDED	(1 << 5)
+
 #define NFACCT_SNPRINTF_T_PLAIN 0
 #define NFACCT_SNPRINTF_T_XML 1
 
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index db01c31..c9f6fe6 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -512,6 +512,9 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s fmt=%s,%s pkts=%"PRIu64 \
 					" bytes=%"PRIu64
 #define NFACCT_STR_PLAIN_SAVE_BTHR	" thr=%"PRIu64
+#define NFACCT_STR_PLAIN_EXTENDED	"[ pkts = %%%zus " \
+					" bytes = %%%zus%s" \
+					" thr = %%%zus ] = %%s"
 #define NFACCT_STR_PLAIN_NEW		"[ pkts = %%%zus" \
 					" bytes = %%%zus%s ] = %%s"
 #define NFACCT_STR_PLAIN		"{ pkts = %%%zus, " \
@@ -521,6 +524,7 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define NFACCT_XML_PKTS			"<pkts fmt=\"%s\">%s</pkts>"
 #define NFACCT_XML_BYTES		"<bytes fmt=\"%s\" " \
 					"thr=\"%s\">%s</bytes>"
+#define NFACCT_XML_BTHR			"<threshold fmt=\"%s\">%s</threshold>"
 #define NFACCT_STR_XML_BONLY		"<obj>"	NFACCT_XML_NAME \
 					NFACCT_XML_BYTES
 #define NFACCT_STR_XML_COMPAT		"<obj><name>%s</name>"	\
@@ -528,6 +532,9 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 					"<bytes>%s</bytes>"
 #define NFACCT_STR_XML			"<obj>"	NFACCT_XML_NAME \
 					NFACCT_XML_PKTS NFACCT_XML_BYTES
+#define NFACCT_STR_XML_EXTENDED		"<obj>"	NFACCT_XML_NAME \
+					NFACCT_XML_PKTS NFACCT_XML_BYTES \
+					NFACCT_XML_BTHR
 #define NFACCT_STR_DEFAULT		"%020.0f%s"
 #define NFACCT_STR_NONE			"%.0f%s"
 #define NFACCT_STR_TRIPLETS		"%'.0f%s"
@@ -772,9 +779,10 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 	int ret = 0, size = 0, offset = 0;
 	bool compat = (options == NULL);
 	uint16_t fmt, fl;
+	size_t pcw, bcw, btcw;
 	uint64_t pkts = 0, bytes = 0, thr = 0;
 	char nfacct_name[NFACCT_NAME_MAX * 2 + 4];
-	char fmt_str[sizeof(NFACCT_STR_PLAIN_NEW) +
+	char fmt_str[sizeof(NFACCT_STR_PLAIN_EXTENDED) +
 		     sizeof(NFACCT_STR_DEFAULT) * 5 + 10];
 	struct nfacct_number pn = NFACCT_NUM_DEFAULT,
 			     bn = NFACCT_NUM_DEFAULT,
@@ -880,6 +888,38 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 
 		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_NUMONLY,
 				pn.str, bn.str, tn.str);
+	} else if (flags & NFACCT_SNPRINTF_F_EXTENDED) {
+		/* print pkts + pmark + bytes + bmark + threshold + name */
+		pcw = nfacct_option_get_tsize(options, NFACCT_OPT_PCW);
+		bcw = nfacct_option_get_tsize(options, NFACCT_OPT_BCW);
+		btcw = nfacct_option_get_tsize(options, NFACCT_OPT_BTCW);
+		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		thr = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BTHR);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
+		parse_nfacct_name(nfacct_name,
+				nfacct_attr_get_str(nfacct,
+						    NFACCT_ATTR_NAME));
+		fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
+
+		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
+			init_locale();
+
+		format_number(&pn, pkts, nfacct_get_pkt_fmt(fmt));
+		format_number(&bn, bytes, nfacct_get_bytes_fmt(fmt));
+
+		if (fl & NFACCT_FLAG_BTHR)
+			format_number(&tn, thr, nfacct_get_bytes_fmt(fmt));
+
+		snprintf(fmt_str, sizeof(fmt_str), NFACCT_STR_PLAIN_EXTENDED,
+			 pcw, bcw,
+			 NFACCT_PRINT_FL(fl & NFACCT_FLAG_BTHR, bytes, thr),
+			 btcw);
+		ret = snprintf(buf, rem, fmt_str, pn.str, bn.str, tn.str,
+			       nfacct_name);
 	} else {
 		/* print out name only */
 		parse_nfacct_name(nfacct_name,
@@ -933,7 +973,8 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 	uint64_t pkts = 0, bytes = 0, thr = 0;
 	char nfacct_name[NFACCT_NAME_MAX * 6 + 1];
 	struct nfacct_number pn = NFACCT_NUM_DEFAULT,
-			     bn = NFACCT_NUM_DEFAULT;
+			     bn = NFACCT_NUM_DEFAULT,
+			     tn = NFACCT_NUM_DEFAULT;
 
 	parse_nfacct_name_xml(nfacct_name,
 				nfacct_attr_get_str(nfacct,
@@ -959,6 +1000,33 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 						    bytes,thr),
 				bn.str);
 		BUFFER_SIZE(ret, size, rem, offset);
+	} else if (flags & NFACCT_SNPRINTF_F_EXTENDED) {
+		/* print name + pkts + pmark + bytes + bmark + threshold */
+		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		thr = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BTHR);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
+
+		format_number(&pn, pkts, NFACCT_FMT_NONE);
+		format_number(&bn, bytes, NFACCT_FMT_NONE);
+
+		if (fl & NFACCT_FLAG_BTHR)
+			format_number(&tn, thr, NFACCT_FMT_NONE);
+
+		ret = snprintf(buf, rem, NFACCT_STR_XML_EXTENDED,
+				nfacct_name,
+				NFACCT_GET_FMT(nfacct_get_pkt_fmt(fmt)),
+				pn.str,
+				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
+				NFACCT_PRINT_FL_XML(fl & NFACCT_FLAG_BTHR,
+						    bytes, thr),
+				bn.str,
+				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
+				tn.str);
+		BUFFER_SIZE(ret, size, rem, offset);
 	} else {
 		/* default/everything else: print name + pkts + bytes */
 		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
-- 
1.8.3.1

[PATCH v3 nfacct 23/29] add "show extended" option to "list" and "get" commands

[Michael Zintakis]

* add a separate "show extended" option to "list" and "get" commands showing
all accounting object properties in separate columns: name, packet & byte
counters, threshold exceeded indicator and the threshold value itself.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/nfacct.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/src/nfacct.c b/src/nfacct.c
index f2ece72..4504143 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -289,6 +289,8 @@ static int nfacct_cmd_list(int argc, char *argv[])
 			NFACCT_GET_NEXT_ARG();
 			if (nfacct_matches(argv[0],"bytes")) {
 				flags = NFACCT_SNPRINTF_F_BONLY;
+			} else if (nfacct_matches(argv[0],"extended")) {
+				flags = NFACCT_SNPRINTF_F_EXTENDED;
 			} else {
 				NFACCT_RET_ARG_ERR();
 			}
@@ -617,6 +619,8 @@ static int nfacct_cmd_get(int argc, char *argv[])
 			NFACCT_GET_NEXT_ARG();
 			if (nfacct_matches(argv[0],"bytes")) {
 				flags = NFACCT_SNPRINTF_F_BONLY;
+			} else if (nfacct_matches(argv[0],"extended")) {
+				flags = NFACCT_SNPRINTF_F_EXTENDED;
 			} else {
 				NFACCT_RET_ARG_ERR();
 			}
@@ -788,12 +792,13 @@ static const char help_msg[] =
 	"  version\t\tDisplay version and disclaimer\n"
 	"  help\t\t\tDisplay this help message\n\n"
 	"Parameters:\n"
-	"  LST_PARAMS := [ reset ] [ show bytes ] [ format FMT_SPEC ]\n"
+	"  LST_PARAMS := [ reset ] [ show SHOW_SPEC ] [ format FMT_SPEC ]\n"
 			"\t\t[ sort SORT_SPEC ] [ xml ]\n"
 	"  ADD_PARAMS := [ replace ] [ format FMT_SPEC ] "
 			"[ threshold [NUMBER | '-'] ]\n"
-	"  GET_PARAMS := [ reset ] [ show bytes ] [ format FMT_SPEC ] [ xml ]\n"
+	"  GET_PARAMS := [ reset ] [ show SHOW_SPEC ] [ format FMT_SPEC ] [ xml ]\n"
 	"  RST_PARAMS := [ flush ] [ replace ]\n"
+	"  SHOW_SPEC := { bytes | extended }\n"
 	"  FMT_SPEC := { [FMT] | [,] | [FMT] ... }\n"
 	"  SORT_SPEC := { none | name | packets | bytes | threshold }"
 	"  FMT := { def | raw | 3pl | iec | kib | mib | gib | tib | pib |"
-- 
1.8.3.1

[PATCH v3 kernel 24/29] add packets and bytes mark capability to nfacct

[Michael Zintakis]

* add two variables to each nfacct object - 'pmark' and 'bmark', allowing
short-term traffic accounting to be implemented by placing a "mark" against
that object.

This enables counting of traffic (both bytes and packets) since that mark has
been enabled/set, in addition to the main packet and byte counters.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/uapi/linux/netfilter/nfnetlink_acct.h |  8 +++-
 net/netfilter/nfnetlink_acct.c                | 56 +++++++++++++++++++++++++++
 2 files changed, 63 insertions(+), 1 deletion(-)

diff --git a/include/uapi/linux/netfilter/nfnetlink_acct.h b/include/uapi/linux/netfilter/nfnetlink_acct.h
index e972970..87d2615 100644
--- a/include/uapi/linux/netfilter/nfnetlink_acct.h
+++ b/include/uapi/linux/netfilter/nfnetlink_acct.h
@@ -10,6 +10,8 @@ enum nfnl_acct_msg_types {
 	NFNL_MSG_ACCT_GET,
 	NFNL_MSG_ACCT_GET_CTRZERO,
 	NFNL_MSG_ACCT_DEL,
+	NFNL_MSG_ACCT_GET_SETMARK,
+	NFNL_MSG_ACCT_GET_CLRMARK,
 	NFNL_MSG_ACCT_MAX
 };
 
@@ -23,6 +25,8 @@ enum nfnl_acct_type {
 	NFACCT_FMT,
 	NFACCT_FLAGS,
 	NFACCT_CMD,
+	NFACCT_PMARK,
+	NFACCT_BMARK,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
@@ -30,7 +34,9 @@ enum nfnl_acct_type {
 enum nfnl_acct_flags {
 	NFACCT_FLAG_BIT_BTHR 	= 0,
 	NFACCT_FLAG_BTHR	= (1 << NFACCT_FLAG_BIT_BTHR),
-	NFACCT_FLAG_BIT_MAX	= 1,
+	NFACCT_FLAG_BIT_MARK	= 1,
+	NFACCT_FLAG_MARK	= (1 << NFACCT_FLAG_BIT_MARK),
+	NFACCT_FLAG_BIT_MAX	= 2,
 	NFACCT_FLAG_MAX		= (1 << NFACCT_FLAG_BIT_MAX),
 };
 
diff --git a/net/netfilter/nfnetlink_acct.c b/net/netfilter/nfnetlink_acct.c
index 18cd28e..809fa35 100644
--- a/net/netfilter/nfnetlink_acct.c
+++ b/net/netfilter/nfnetlink_acct.c
@@ -33,6 +33,8 @@ struct nf_acct {
 	atomic64_t		pkts;
 	atomic64_t		bytes;
 	u64			bthr;
+	u64			pmark;
+	u64			bmark;
 	u16			fmt;
 	u16			flags;
 	struct list_head	head;
@@ -61,6 +63,10 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 		cmd = be16_to_cpu(nla_get_be16(tb[NFACCT_CMD]));
 		flags = be16_to_cpu(nla_get_be16(tb[NFACCT_FLAGS]));
 
+		if (!(cmd & NFACCT_FLAG_MARK) &&
+		     (tb[NFACCT_PMARK] || tb[NFACCT_BMARK]))
+			return -EINVAL;
+
 		if (cmd & NFACCT_FLAG_BTHR &&
 		    ((flags & NFACCT_FLAG_BTHR && !tb[NFACCT_BTHR]) ||
 		     (!(flags & NFACCT_FLAG_BTHR) && tb[NFACCT_BTHR])))
@@ -114,6 +120,25 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 				matching->fmt =
 				be16_to_cpu(nla_get_be16(tb[NFACCT_FMT]));
 			}
+			/* ...then set the mark flag... */
+			if (cmd & NFACCT_FLAG_MARK) {
+				if (flags & NFACCT_FLAG_MARK) {
+					matching->pmark = tb[NFACCT_PMARK] ?
+					be64_to_cpu(
+					  nla_get_be64(tb[NFACCT_PMARK])) :
+					atomic64_read(&matching->pkts);
+
+					matching->bmark = tb[NFACCT_BMARK] ?
+					be64_to_cpu(
+					  nla_get_be64(tb[NFACCT_BMARK])) :
+					atomic64_read(&matching->bytes);
+					matching->flags |= NFACCT_FLAG_MARK;
+				} else {
+					matching->pmark = 0;
+					matching->bmark = 0;
+					matching->flags &= ~NFACCT_FLAG_MARK;
+				}
+			}
 			/* ... and finally set the bytes threshold */
 			if (cmd & NFACCT_FLAG_BTHR) {
 				if (flags & NFACCT_FLAG_BTHR) {
@@ -147,6 +172,16 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
 	if (tb[NFACCT_FMT]) {
 		nfacct->fmt = be16_to_cpu(nla_get_be16(tb[NFACCT_FMT]));
 	}
+	if (cmd & NFACCT_FLAG_MARK && flags & NFACCT_FLAG_MARK) {
+		if (tb[NFACCT_PMARK])
+			nfacct->pmark = be64_to_cpu(
+				nla_get_be64(tb[NFACCT_PMARK]));
+		if (tb[NFACCT_BMARK])
+			nfacct->bmark = be64_to_cpu(
+				nla_get_be64(tb[NFACCT_BMARK]));
+
+		nfacct->flags |= NFACCT_FLAG_MARK;
+	}
 	if (cmd & NFACCT_FLAG_BTHR && flags & NFACCT_FLAG_BTHR) {
 		if (tb[NFACCT_BTHR])
 			nfacct->bthr = be64_to_cpu(
@@ -184,15 +219,28 @@ nfnl_acct_fill_info(struct sk_buff *skb, u32 portid, u32 seq, u32 type,
 	if (type == NFNL_MSG_ACCT_GET_CTRZERO) {
 		pkts = atomic64_xchg(&acct->pkts, 0);
 		bytes = atomic64_xchg(&acct->bytes, 0);
+		acct->pmark = 0;
+		acct->bmark = 0;
 	} else {
 		pkts = atomic64_read(&acct->pkts);
 		bytes = atomic64_read(&acct->bytes);
+		if (type == NFNL_MSG_ACCT_GET_SETMARK) {
+			acct->pmark = pkts;
+			acct->bmark = bytes;
+			acct->flags |= NFACCT_FLAG_MARK;
+		} else if (type == NFNL_MSG_ACCT_GET_CLRMARK) {
+			acct->pmark = 0;
+			acct->bmark = 0;
+			acct->flags &= ~NFACCT_FLAG_MARK;
+		}
 	}
 	if (nla_put_be64(skb, NFACCT_PKTS, cpu_to_be64(pkts)) ||
 	    nla_put_be64(skb, NFACCT_BYTES, cpu_to_be64(bytes)) ||
 	    nla_put_be64(skb, NFACCT_BTHR, cpu_to_be64(acct->bthr)) ||
 	    nla_put_be16(skb, NFACCT_FMT, htons(acct->fmt)) ||
 	    nla_put_be16(skb, NFACCT_FLAGS, htons(acct->flags)) ||
+	    nla_put_be64(skb, NFACCT_PMARK, cpu_to_be64(acct->pmark)) ||
+	    nla_put_be64(skb, NFACCT_BMARK, cpu_to_be64(acct->bmark)) ||
 	    nla_put_be32(skb, NFACCT_USE, htonl(atomic_read(&acct->refcnt))))
 		goto nla_put_failure;
 
@@ -344,6 +392,8 @@ static const struct nla_policy nfnl_acct_policy[NFACCT_MAX+1] = {
 	[NFACCT_FMT] = { .type = NLA_U16 },
 	[NFACCT_FLAGS] = { .type = NLA_U16 },
 	[NFACCT_CMD] = { .type = NLA_U16 },
+	[NFACCT_PMARK] = { .type = NLA_U64 },
+	[NFACCT_BMARK] = { .type = NLA_U64 },
 };
 
 static const struct nfnl_callback nfnl_acct_cb[NFNL_MSG_ACCT_MAX] = {
@@ -359,6 +409,12 @@ static const struct nfnl_callback nfnl_acct_cb[NFNL_MSG_ACCT_MAX] = {
 	[NFNL_MSG_ACCT_DEL]		= { .call = nfnl_acct_del,
 					    .attr_count = NFACCT_MAX,
 					    .policy = nfnl_acct_policy },
+	[NFNL_MSG_ACCT_GET_SETMARK] 	= { .call = nfnl_acct_get,
+					    .attr_count = NFACCT_MAX,
+					    .policy = nfnl_acct_policy },
+	[NFNL_MSG_ACCT_GET_CLRMARK] 	= { .call = nfnl_acct_get,
+					    .attr_count = NFACCT_MAX,
+					    .policy = nfnl_acct_policy },
 };
 
 static const struct nfnetlink_subsystem nfnl_acct_subsys = {
-- 
1.8.3.1

Re: [PATCH v3 kernel 24/29] add packets and bytes mark capability to nfacct

[Florian Westphal]

Michael Zintakis <michael.zintakis@googlemail.com> wrote:
> * add two variables to each nfacct object - 'pmark' and 'bmark', allowing
> short-term traffic accounting to be implemented by placing a "mark" against
> that object.
> 
> This enables counting of traffic (both bytes and packets) since that mark has
> been enabled/set, in addition to the main packet and byte counters.

And again.
Why not simply add a 2nd accounting object, and then send traffic to
it based on ruleset? (-m time, -m condition, -m set, etc.)?

Re: [PATCH v3 kernel 24/29] add packets and bytes mark capability to nfacct

[Michael Zintakis]

Florian Westphal wrote:
> Michael Zintakis <michael.zintakis@googlemail.com> wrote:
>> * add two variables to each nfacct object - 'pmark' and 'bmark', allowing
>> short-term traffic accounting to be implemented by placing a "mark" against
>> that object.
>>
>> This enables counting of traffic (both bytes and packets) since that mark has
>> been enabled/set, in addition to the main packet and byte counters.
> 
> And again.
> Why not simply add a 2nd accounting object, and then send traffic to
> it based on ruleset? (-m time, -m condition, -m set, etc.)?
We thought of that initially, but rejected that idea.

The marking feature is, at least in our case, primarily used to measure short-term traffic. What that means is for us to be able to gather information for traffic based on certain events or characteristics in short term (peak time or certain event during which we expect our traffic to rise/slow down dramatically for example) and produce the relevant statistics without affecting the main traffic counters.

Adding separate nfacct objects to the iptables rules (which is what I think you are suggesting above) isn't very efficient or practical, simply because: a) certain traffic is going to be spread over different iptables rules in different places (i.e. chains) and in order to add a "secondary" nfacct objects one needs to do that everywhere - in every rule - a certain "primary" nfacct object is used; b) a packet will have to pass through 2 nfacct objects in order to produce the desired effect (with marking, a packet passes and is counted simply once and that is the end of the story), not to mention that it would require nearly twice-as-much memory per accounting object, compared to if we use marking, so this is inefficient; and c) in order to add/remove nfacct objects for short-term traffic me
 asurements like this and deploy what you are suggesting above, one needs to have full knowledge of the entire iptables structure and the whole netfilter system used, which is not always the 
case - at least not here, so what you suggest above isn't practical.

Re: [PATCH v3 kernel 24/29] add packets and bytes mark capability to nfacct

[Pablo Neira Ayuso]

On Wed, Jul 10, 2013 at 07:25:22PM +0100, Michael Zintakis wrote:
> * add two variables to each nfacct object - 'pmark' and 'bmark', allowing
> short-term traffic accounting to be implemented by placing a "mark" against
> that object.
> 
> This enables counting of traffic (both bytes and packets) since that mark has
> been enabled/set, in addition to the main packet and byte counters.
> 
> Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
> ---
>  include/uapi/linux/netfilter/nfnetlink_acct.h |  8 +++-
>  net/netfilter/nfnetlink_acct.c                | 56 +++++++++++++++++++++++++++
>  2 files changed, 63 insertions(+), 1 deletion(-)
> 
> diff --git a/net/netfilter/nfnetlink_acct.c b/net/netfilter/nfnetlink_acct.c
> index 18cd28e..809fa35 100644
> --- a/net/netfilter/nfnetlink_acct.c
> +++ b/net/netfilter/nfnetlink_acct.c
> @@ -33,6 +33,8 @@ struct nf_acct {
>  	atomic64_t		pkts;
>  	atomic64_t		bytes;
>  	u64			bthr;
> +	u64			pmark;
> +	u64			bmark;
>  	u16			fmt;
>  	u16			flags;
>  	struct list_head	head;

Oh my...

You insist on your idea of using the kernel as a database to simplify
your user-space program. All these fields are set/unset from
userspace, they are not altered by packets at all. This does not
belong here.

> @@ -61,6 +63,10 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
>  		cmd = be16_to_cpu(nla_get_be16(tb[NFACCT_CMD]));
>  		flags = be16_to_cpu(nla_get_be16(tb[NFACCT_FLAGS]));
>  
> +		if (!(cmd & NFACCT_FLAG_MARK) &&
> +		     (tb[NFACCT_PMARK] || tb[NFACCT_BMARK]))
> +			return -EINVAL;
> +
>  		if (cmd & NFACCT_FLAG_BTHR &&
>  		    ((flags & NFACCT_FLAG_BTHR && !tb[NFACCT_BTHR]) ||
>  		     (!(flags & NFACCT_FLAG_BTHR) && tb[NFACCT_BTHR])))
> @@ -114,6 +120,25 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
>  				matching->fmt =
>  				be16_to_cpu(nla_get_be16(tb[NFACCT_FMT]));
>  			}
> +			/* ...then set the mark flag... */
> +			if (cmd & NFACCT_FLAG_MARK) {
> +				if (flags & NFACCT_FLAG_MARK) {
> +					matching->pmark = tb[NFACCT_PMARK] ?
> +					be64_to_cpu(
> +					  nla_get_be64(tb[NFACCT_PMARK])) :
> +					atomic64_read(&matching->pkts);
> +
> +					matching->bmark = tb[NFACCT_BMARK] ?
> +					be64_to_cpu(
> +					  nla_get_be64(tb[NFACCT_BMARK])) :
> +					atomic64_read(&matching->bytes);
> +					matching->flags |= NFACCT_FLAG_MARK;
> +				} else {
> +					matching->pmark = 0;
> +					matching->bmark = 0;
> +					matching->flags &= ~NFACCT_FLAG_MARK;
> +				}
> +			}
>  			/* ... and finally set the bytes threshold */
>  			if (cmd & NFACCT_FLAG_BTHR) {
>  				if (flags & NFACCT_FLAG_BTHR) {
> @@ -147,6 +172,16 @@ nfnl_acct_new(struct sock *nfnl, struct sk_buff *skb,
>  	if (tb[NFACCT_FMT]) {
>  		nfacct->fmt = be16_to_cpu(nla_get_be16(tb[NFACCT_FMT]));
>  	}
> +	if (cmd & NFACCT_FLAG_MARK && flags & NFACCT_FLAG_MARK) {
> +		if (tb[NFACCT_PMARK])
> +			nfacct->pmark = be64_to_cpu(
> +				nla_get_be64(tb[NFACCT_PMARK]));
> +		if (tb[NFACCT_BMARK])
> +			nfacct->bmark = be64_to_cpu(
> +				nla_get_be64(tb[NFACCT_BMARK]));
> +
> +		nfacct->flags |= NFACCT_FLAG_MARK;
> +	}
>  	if (cmd & NFACCT_FLAG_BTHR && flags & NFACCT_FLAG_BTHR) {
>  		if (tb[NFACCT_BTHR])
>  			nfacct->bthr = be64_to_cpu(
> @@ -184,15 +219,28 @@ nfnl_acct_fill_info(struct sk_buff *skb, u32 portid, u32 seq, u32 type,
>  	if (type == NFNL_MSG_ACCT_GET_CTRZERO) {
>  		pkts = atomic64_xchg(&acct->pkts, 0);
>  		bytes = atomic64_xchg(&acct->bytes, 0);
> +		acct->pmark = 0;
> +		acct->bmark = 0;
>  	} else {
>  		pkts = atomic64_read(&acct->pkts);
>  		bytes = atomic64_read(&acct->bytes);
> +		if (type == NFNL_MSG_ACCT_GET_SETMARK) {
> +			acct->pmark = pkts;
> +			acct->bmark = bytes;
> +			acct->flags |= NFACCT_FLAG_MARK;
> +		} else if (type == NFNL_MSG_ACCT_GET_CLRMARK) {
> +			acct->pmark = 0;
> +			acct->bmark = 0;
> +			acct->flags &= ~NFACCT_FLAG_MARK;
> +		}
>  	}
>  	if (nla_put_be64(skb, NFACCT_PKTS, cpu_to_be64(pkts)) ||
>  	    nla_put_be64(skb, NFACCT_BYTES, cpu_to_be64(bytes)) ||
>  	    nla_put_be64(skb, NFACCT_BTHR, cpu_to_be64(acct->bthr)) ||
>  	    nla_put_be16(skb, NFACCT_FMT, htons(acct->fmt)) ||
>  	    nla_put_be16(skb, NFACCT_FLAGS, htons(acct->flags)) ||
> +	    nla_put_be64(skb, NFACCT_PMARK, cpu_to_be64(acct->pmark)) ||
> +	    nla_put_be64(skb, NFACCT_BMARK, cpu_to_be64(acct->bmark)) ||
>  	    nla_put_be32(skb, NFACCT_USE, htonl(atomic_read(&acct->refcnt))))
>  		goto nla_put_failure;
>  
> @@ -344,6 +392,8 @@ static const struct nla_policy nfnl_acct_policy[NFACCT_MAX+1] = {
>  	[NFACCT_FMT] = { .type = NLA_U16 },
>  	[NFACCT_FLAGS] = { .type = NLA_U16 },
>  	[NFACCT_CMD] = { .type = NLA_U16 },
> +	[NFACCT_PMARK] = { .type = NLA_U64 },
> +	[NFACCT_BMARK] = { .type = NLA_U64 },
>  };
>  
>  static const struct nfnl_callback nfnl_acct_cb[NFNL_MSG_ACCT_MAX] = {
> @@ -359,6 +409,12 @@ static const struct nfnl_callback nfnl_acct_cb[NFNL_MSG_ACCT_MAX] = {
>  	[NFNL_MSG_ACCT_DEL]		= { .call = nfnl_acct_del,
>  					    .attr_count = NFACCT_MAX,
>  					    .policy = nfnl_acct_policy },
> +	[NFNL_MSG_ACCT_GET_SETMARK] 	= { .call = nfnl_acct_get,
> +					    .attr_count = NFACCT_MAX,
> +					    .policy = nfnl_acct_policy },
> +	[NFNL_MSG_ACCT_GET_CLRMARK] 	= { .call = nfnl_acct_get,
> +					    .attr_count = NFACCT_MAX,
> +					    .policy = nfnl_acct_policy },
>  };
>  
>  static const struct nfnetlink_subsystem nfnl_acct_subsys = {
> -- 
> 1.8.3.1
> 

Re: [PATCH v3 kernel 24/29] add packets and bytes mark capability to nfacct

[Michael Zintakis]

Pablo Neira Ayuso wrote:
> On Wed, Jul 10, 2013 at 07:25:22PM +0100, Michael Zintakis wrote:
>> * add two variables to each nfacct object - 'pmark' and 'bmark', allowing
>> short-term traffic accounting to be implemented by placing a "mark" against
>> that object.
>>
>> This enables counting of traffic (both bytes and packets) since that mark has
>> been enabled/set, in addition to the main packet and byte counters.
>>
>> Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
>> ---
>>  include/uapi/linux/netfilter/nfnetlink_acct.h |  8 +++-
>>  net/netfilter/nfnetlink_acct.c                | 56 +++++++++++++++++++++++++++
>>  2 files changed, 63 insertions(+), 1 deletion(-)
>>
>> diff --git a/net/netfilter/nfnetlink_acct.c b/net/netfilter/nfnetlink_acct.c
>> index 18cd28e..809fa35 100644
>> --- a/net/netfilter/nfnetlink_acct.c
>> +++ b/net/netfilter/nfnetlink_acct.c
>> @@ -33,6 +33,8 @@ struct nf_acct {
>>  	atomic64_t		pkts;
>>  	atomic64_t		bytes;
>>  	u64			bthr;
>> +	u64			pmark;
>> +	u64			bmark;
>>  	u16			fmt;
>>  	u16			flags;
>>  	struct list_head	head;
> 
> Oh my...
> 
> You insist on your idea of using the kernel as a database to simplify
> your user-space program. All these fields are set/unset from
> userspace, they are not altered by packets at all. This does not
> belong here.
I don't "insist" on anything Pablo and I am not using the kernel as a database either.

The way these variables are structured and information is gathered/presented is the most efficient way - both memory and performance-wise, compared to what you were suggesting we do up until now (with separate file access and so on), not to mention the security aspect of it and the fact that protecting this within the kernel is much more secure and less error-prone compared to your suggestion with file access. That, at the end of the day, is what matters here.

[PATCH v3 libnetfilter_acct 25/29] add packets/bytes mark capability support

[Michael Zintakis]

* add support for printing/retrieving packets and bytes mark of nfacct
objects, using the 'pmark' and 'bmark' values stored in each such object
within the kernel;

* enhance the existing API to support get/set pmark/bmark functions;

* adjust all existing xml and plaintext templates to support this feature;

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/libnetfilter_acct/libnetfilter_acct.h |   4 +
 include/linux/netfilter/nfnetlink_acct.h      |   8 +-
 src/libnetfilter_acct.c                       | 169 +++++++++++++++++++++++---
 3 files changed, 161 insertions(+), 20 deletions(-)

diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index e4f21e0..a58d93f 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -18,6 +18,8 @@ enum nfacct_attr_type {
 	NFACCT_ATTR_FMT,
 	NFACCT_ATTR_FLAGS,
 	NFACCT_ATTR_CMD,
+	NFACCT_ATTR_PMARK,
+	NFACCT_ATTR_BMARK,
 };
 
 struct nfacct_options;
@@ -25,7 +27,9 @@ struct nfacct_options;
 enum nfacct_option_type {
 	NFACCT_OPT_FMT = 0,	/* number format option 	*/
 	NFACCT_OPT_PCW,		/* packets count column width 	*/
+	NFACCT_OPT_PMCW,	/* packets mark column width 	*/
 	NFACCT_OPT_BCW,		/* bytes count column width 	*/
+	NFACCT_OPT_BMCW,	/* bytes mark column width 	*/
 	NFACCT_OPT_BTCW,	/* bytes threshold column width */
 };
 
diff --git a/include/linux/netfilter/nfnetlink_acct.h b/include/linux/netfilter/nfnetlink_acct.h
index e972970..87d2615 100644
--- a/include/linux/netfilter/nfnetlink_acct.h
+++ b/include/linux/netfilter/nfnetlink_acct.h
@@ -10,6 +10,8 @@ enum nfnl_acct_msg_types {
 	NFNL_MSG_ACCT_GET,
 	NFNL_MSG_ACCT_GET_CTRZERO,
 	NFNL_MSG_ACCT_DEL,
+	NFNL_MSG_ACCT_GET_SETMARK,
+	NFNL_MSG_ACCT_GET_CLRMARK,
 	NFNL_MSG_ACCT_MAX
 };
 
@@ -23,6 +25,8 @@ enum nfnl_acct_type {
 	NFACCT_FMT,
 	NFACCT_FLAGS,
 	NFACCT_CMD,
+	NFACCT_PMARK,
+	NFACCT_BMARK,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
@@ -30,7 +34,9 @@ enum nfnl_acct_type {
 enum nfnl_acct_flags {
 	NFACCT_FLAG_BIT_BTHR 	= 0,
 	NFACCT_FLAG_BTHR	= (1 << NFACCT_FLAG_BIT_BTHR),
-	NFACCT_FLAG_BIT_MAX	= 1,
+	NFACCT_FLAG_BIT_MARK	= 1,
+	NFACCT_FLAG_MARK	= (1 << NFACCT_FLAG_BIT_MARK),
+	NFACCT_FLAG_BIT_MAX	= 2,
 	NFACCT_FLAG_MAX		= (1 << NFACCT_FLAG_BIT_MAX),
 };
 
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index c9f6fe6..8773710 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -62,6 +62,8 @@ struct nfacct {
 	uint64_t	pkts;
 	uint64_t	bytes;
 	uint64_t	bthr;
+	uint64_t	pmark;
+	uint64_t	bmark;
 	/*
 	 * Structure of fmt:
 	 * <- packets format: 8 bits-><-bytes format: 8 bits ->
@@ -77,6 +79,8 @@ struct nfacct_options {
 	size_t		pcw;
 	size_t		bcw;
 	size_t		btcw;
+	size_t		pmcw;
+	size_t		bmcw;
 	uint32_t	bitset;
 };
 
@@ -169,6 +173,14 @@ nfacct_attr_set(struct nfacct *nfacct, enum nfacct_attr_type type,
 		nfacct->cmd = *((uint16_t *) data);
 		nfacct->bitset |= (1 << NFACCT_ATTR_CMD);
 		break;
+	case NFACCT_ATTR_PMARK:
+		nfacct->pmark = *((uint64_t *) data);
+		nfacct->bitset |= (1 << NFACCT_ATTR_PMARK);
+		break;
+	case NFACCT_ATTR_BMARK:
+		nfacct->bmark = *((uint64_t *) data);
+		nfacct->bitset |= (1 << NFACCT_ATTR_BMARK);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_attr_set);
@@ -245,6 +257,12 @@ nfacct_attr_unset(struct nfacct *nfacct, enum nfacct_attr_type type)
 	case NFACCT_ATTR_CMD:
 		nfacct->bitset &= ~(1 << NFACCT_ATTR_CMD);
 		break;
+	case NFACCT_ATTR_PMARK:
+		nfacct->bitset &= ~(1 << NFACCT_ATTR_PMARK);
+		break;
+	case NFACCT_ATTR_BMARK:
+		nfacct->bitset &= ~(1 << NFACCT_ATTR_BMARK);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_attr_unset);
@@ -290,6 +308,14 @@ const void *nfacct_attr_get(struct nfacct *nfacct, enum nfacct_attr_type type)
 		if (nfacct->bitset & (1 << NFACCT_ATTR_CMD))
 			ret = &nfacct->cmd;
 		break;
+	case NFACCT_ATTR_PMARK:
+		if (nfacct->bitset & (1 << NFACCT_ATTR_PMARK))
+			ret = &nfacct->pmark;
+		break;
+	case NFACCT_ATTR_BMARK:
+		if (nfacct->bitset & (1 << NFACCT_ATTR_BMARK))
+			ret = &nfacct->bmark;
+		break;
 	}
 	return ret;
 }
@@ -368,6 +394,14 @@ nfacct_option_set(struct nfacct_options *options,
 		options->btcw = *((size_t *) data);
 		options->bitset |= (1 << NFACCT_OPT_BTCW);
 		break;
+	case NFACCT_OPT_PMCW:
+		options->pmcw = *((size_t *) data);
+		options->bitset |= (1 << NFACCT_OPT_PMCW);
+		break;
+	case NFACCT_OPT_BMCW:
+		options->bmcw = *((size_t *) data);
+		options->bitset |= (1 << NFACCT_OPT_BMCW);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_option_set);
@@ -424,6 +458,12 @@ nfacct_option_unset(struct nfacct_options *options,
 	case NFACCT_OPT_BTCW:
 		options->bitset &= ~(1 << NFACCT_OPT_BTCW);
 		break;
+	case NFACCT_OPT_PMCW:
+		options->bitset &= ~(1 << NFACCT_OPT_PMCW);
+		break;
+	case NFACCT_OPT_BMCW:
+		options->bitset &= ~(1 << NFACCT_OPT_BMCW);
+		break;
 	}
 }
 EXPORT_SYMBOL(nfacct_option_unset);
@@ -458,6 +498,14 @@ const void *nfacct_option_get(struct nfacct_options *options,
 		if (options->bitset & (1 << NFACCT_OPT_BTCW))
 			ret = &options->btcw;
 		break;
+	case NFACCT_OPT_PMCW:
+		if (options->bitset & (1 << NFACCT_OPT_PMCW))
+			ret = &options->pmcw;
+		break;
+	case NFACCT_OPT_BMCW:
+		if (options->bitset & (1 << NFACCT_OPT_BMCW))
+			ret = &options->bmcw;
+		break;
 	}
 	return ret;
 }
@@ -508,22 +556,26 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define PB ((uint64_t) TB*1000)
 #define EB ((uint64_t) PB*1000)
 
-#define NFACCT_STR_PLAIN_NUMONLY	"%s %s %s"
+#define NFACCT_STR_PLAIN_NUMONLY	"%s %s %s %s %s"
 #define NFACCT_STR_PLAIN_SAVE_BASE	"name=%s fmt=%s,%s pkts=%"PRIu64 \
 					" bytes=%"PRIu64
 #define NFACCT_STR_PLAIN_SAVE_BTHR	" thr=%"PRIu64
-#define NFACCT_STR_PLAIN_EXTENDED	"[ pkts = %%%zus " \
-					" bytes = %%%zus%s" \
+#define NFACCT_STR_PLAIN_SAVE_MARK	" pmark=%"PRIu64 " bmark=%"PRIu64
+#define NFACCT_STR_PLAIN_EXTENDED	"[ pkts = %%%zus%s pmark = %%%zus " \
+					" bytes = %%%zus%s bmark = %%%zus " \
 					" thr = %%%zus ] = %%s"
-#define NFACCT_STR_PLAIN_NEW		"[ pkts = %%%zus" \
+#define NFACCT_STR_PLAIN_NEW		"[ pkts = %%%zus%s" \
 					" bytes = %%%zus%s ] = %%s"
-#define NFACCT_STR_PLAIN		"{ pkts = %%%zus, " \
+#define NFACCT_STR_PLAIN		"{ pkts = %%%zus%s, " \
 					"bytes = %%%zus%s } = %%s;"
 #define NFACCT_STR_PLAIN_BONLY		"[ bytes = %%%zus%s ] = %%s"
 #define NFACCT_XML_NAME			"<name>%s</name>"
-#define NFACCT_XML_PKTS			"<pkts fmt=\"%s\">%s</pkts>"
+#define NFACCT_XML_PKTS			"<pkts fmt=\"%s\" " \
+					"mark=\"%s\">%s</pkts>"
+#define NFACCT_XML_PMARK		"<pmark fmt=\"%s\">%s</pmark>"
 #define NFACCT_XML_BYTES		"<bytes fmt=\"%s\" " \
 					"thr=\"%s\">%s</bytes>"
+#define NFACCT_XML_BMARK		"<bmark fmt=\"%s\">%s</bmark>"
 #define NFACCT_XML_BTHR			"<threshold fmt=\"%s\">%s</threshold>"
 #define NFACCT_STR_XML_BONLY		"<obj>"	NFACCT_XML_NAME \
 					NFACCT_XML_BYTES
@@ -533,7 +585,8 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define NFACCT_STR_XML			"<obj>"	NFACCT_XML_NAME \
 					NFACCT_XML_PKTS NFACCT_XML_BYTES
 #define NFACCT_STR_XML_EXTENDED		"<obj>"	NFACCT_XML_NAME \
-					NFACCT_XML_PKTS NFACCT_XML_BYTES \
+					NFACCT_XML_PKTS NFACCT_XML_PMARK \
+					NFACCT_XML_BYTES NFACCT_XML_BMARK \
 					NFACCT_XML_BTHR
 #define NFACCT_STR_DEFAULT		"%020.0f%s"
 #define NFACCT_STR_NONE			"%.0f%s"
@@ -542,6 +595,7 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 
 #define NFACCT_NUM_DEFAULT		{ .value = 0., .str = "" }
 #define NFACCT_THR_DEFAULT		{ .value = 0., .str = "-" }
+#define NFACCT_MARK_DEFAULT		NFACCT_THR_DEFAULT
 
 struct nfacct_number {
 	float value;
@@ -779,13 +833,15 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 	int ret = 0, size = 0, offset = 0;
 	bool compat = (options == NULL);
 	uint16_t fmt, fl;
-	size_t pcw, bcw, btcw;
-	uint64_t pkts = 0, bytes = 0, thr = 0;
+	size_t pcw, bcw, btcw, pmcw, bmcw;
+	uint64_t pkts = 0, bytes = 0, thr = 0, pmark = 0, bmark = 0;
 	char nfacct_name[NFACCT_NAME_MAX * 2 + 4];
 	char fmt_str[sizeof(NFACCT_STR_PLAIN_EXTENDED) +
 		     sizeof(NFACCT_STR_DEFAULT) * 5 + 10];
 	struct nfacct_number pn = NFACCT_NUM_DEFAULT,
+			     pm = NFACCT_MARK_DEFAULT,
 			     bn = NFACCT_NUM_DEFAULT,
+			     bm = NFACCT_MARK_DEFAULT,
 			     tn = NFACCT_THR_DEFAULT;
 
 	if (flags & NFACCT_SNPRINTF_F_FULL) {
@@ -798,16 +854,22 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 				  nfacct_attr_get_str(nfacct,
 						      NFACCT_ATTR_NAME));
 
+		if (fl & NFACCT_FLAG_MARK)
+			pmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_PMARK);
+
 		if (compat) {
 			fmt = NFACCT_FMT_MAX;
 			snprintf(fmt_str, sizeof(fmt_str), NFACCT_STR_PLAIN,
-				 (size_t)0, (size_t)0, "");
+				 (size_t)0, "", (size_t)0, "");
 		} else {
 			fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
 			snprintf(fmt_str, sizeof(fmt_str),
 				NFACCT_STR_PLAIN_NEW,
 				 nfacct_option_get_tsize(options,
 							 NFACCT_OPT_PCW),
+				 NFACCT_PRINT_FL(fl & NFACCT_FLAG_MARK,
+						 pkts,pmark),
 				 nfacct_option_get_tsize(options,
 							 NFACCT_OPT_BCW),
 				 NFACCT_PRINT_FL(fl & NFACCT_FLAG_BTHR,
@@ -840,6 +902,12 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		BUFFER_SIZE(ret, size, rem, offset);
 
 		fl = nfacct_attr_get_u16(nfacct,NFACCT_ATTR_FLAGS);
+		if (fl & NFACCT_FLAG_MARK) {
+			ret = snprintf(buf+offset, rem, NFACCT_STR_PLAIN_SAVE_MARK,
+			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_PMARK),
+			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BMARK));
+			BUFFER_SIZE(ret, size, rem, offset);
+		}
 		if (fl & NFACCT_FLAG_BTHR) {
 			ret = snprintf(buf+offset, rem, NFACCT_STR_PLAIN_SAVE_BTHR,
 			       nfacct_attr_get_u64(nfacct,NFACCT_ATTR_BTHR));
@@ -883,15 +951,27 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		format_number(&pn, pkts, nfacct_get_pkt_fmt(fmt));
 		format_number(&bn, bytes, nfacct_get_bytes_fmt(fmt));
 
+		if (fl & NFACCT_FLAG_MARK) {
+			pmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_PMARK);
+			bmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_BMARK);
+			format_number(&pm, pkts-pmark < 0 ? 0 : pkts-pmark,
+				      nfacct_get_pkt_fmt(fmt));
+			format_number(&bm, bytes-bmark < 0 ? 0 : bytes-bmark,
+				      nfacct_get_bytes_fmt(fmt));
+		}
 		if (fl & NFACCT_FLAG_BTHR)
 			format_number(&tn, thr, nfacct_get_bytes_fmt(fmt));
 
 		ret = snprintf(buf, rem, NFACCT_STR_PLAIN_NUMONLY,
-				pn.str, bn.str, tn.str);
+				pn.str, pm.str,	bn.str,	bm.str,	tn.str);
 	} else if (flags & NFACCT_SNPRINTF_F_EXTENDED) {
 		/* print pkts + pmark + bytes + bmark + threshold + name */
 		pcw = nfacct_option_get_tsize(options, NFACCT_OPT_PCW);
+		pmcw = nfacct_option_get_tsize(options,	NFACCT_OPT_PMCW);
 		bcw = nfacct_option_get_tsize(options, NFACCT_OPT_BCW);
+		bmcw = nfacct_option_get_tsize(options,	NFACCT_OPT_BMCW);
 		btcw = nfacct_option_get_tsize(options, NFACCT_OPT_BTCW);
 		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
 		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
@@ -908,18 +988,31 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
 			init_locale();
 
+		if (fl & NFACCT_FLAG_MARK) {
+			pmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_PMARK);
+			bmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_BMARK);
+			format_number(&pm, pkts-pmark < 0 ? 0 : pkts-pmark,
+				      nfacct_get_pkt_fmt(fmt));
+			format_number(&bm, bytes-bmark < 0 ? 0 : bytes-bmark,
+				      nfacct_get_bytes_fmt(fmt));
+		}
+
 		format_number(&pn, pkts, nfacct_get_pkt_fmt(fmt));
 		format_number(&bn, bytes, nfacct_get_bytes_fmt(fmt));
 
 		if (fl & NFACCT_FLAG_BTHR)
 			format_number(&tn, thr, nfacct_get_bytes_fmt(fmt));
 
-		snprintf(fmt_str, sizeof(fmt_str), NFACCT_STR_PLAIN_EXTENDED,
-			 pcw, bcw,
+		snprintf(fmt_str,sizeof(fmt_str),NFACCT_STR_PLAIN_EXTENDED,
+			 pcw,
+			 NFACCT_PRINT_FL(fl & NFACCT_FLAG_MARK, pkts, pmark),
+			 pmcw, bcw,
 			 NFACCT_PRINT_FL(fl & NFACCT_FLAG_BTHR, bytes, thr),
-			 btcw);
-		ret = snprintf(buf, rem, fmt_str, pn.str, bn.str, tn.str,
-			       nfacct_name);
+			 bmcw, btcw);
+		ret = snprintf(buf, rem, fmt_str, pn.str, pm.str, bn.str,
+				bm.str, tn.str, nfacct_name);
 	} else {
 		/* print out name only */
 		parse_nfacct_name(nfacct_name,
@@ -970,10 +1063,12 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 	bool compat = (options == NULL);
 	unsigned int size = 0, offset = 0;
 	uint16_t fmt, fl;
-	uint64_t pkts = 0, bytes = 0, thr = 0;
+	uint64_t pkts = 0, bytes = 0, thr = 0, pmark = 0, bmark = 0;
 	char nfacct_name[NFACCT_NAME_MAX * 6 + 1];
 	struct nfacct_number pn = NFACCT_NUM_DEFAULT,
+			     pm = NFACCT_NUM_DEFAULT,
 			     bn = NFACCT_NUM_DEFAULT,
+			     bm = NFACCT_NUM_DEFAULT,
 			     tn = NFACCT_NUM_DEFAULT;
 
 	parse_nfacct_name_xml(nfacct_name,
@@ -1013,18 +1108,34 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 		format_number(&pn, pkts, NFACCT_FMT_NONE);
 		format_number(&bn, bytes, NFACCT_FMT_NONE);
 
+		if (fl & NFACCT_FLAG_MARK) {
+			pmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_PMARK);
+			bmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_BMARK);
+			format_number(&pm, pkts-pmark < 0 ? 0 : pkts-pmark,
+				      NFACCT_FMT_NONE);
+			format_number(&bm, bytes-bmark < 0 ? 0 : bytes-bmark,
+				      NFACCT_FMT_NONE);
+		}
 		if (fl & NFACCT_FLAG_BTHR)
 			format_number(&tn, thr, NFACCT_FMT_NONE);
 
 		ret = snprintf(buf, rem, NFACCT_STR_XML_EXTENDED,
 				nfacct_name,
 				NFACCT_GET_FMT(nfacct_get_pkt_fmt(fmt)),
+				NFACCT_PRINT_FL_XML(fl & NFACCT_FLAG_MARK, pkts,
+					     	    pmark),
 				pn.str,
+				NFACCT_GET_FMT(nfacct_get_pkt_fmt(fmt)),
+				pm.str,
 				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
-				NFACCT_PRINT_FL_XML(fl & NFACCT_FLAG_BTHR,
-						    bytes, thr),
+				NFACCT_PRINT_FL_XML(fl & NFACCT_FLAG_BTHR, bytes,
+						    thr),
 				bn.str,
 				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
+				bm.str,
+				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
 				tn.str);
 		BUFFER_SIZE(ret, size, rem, offset);
 	} else {
@@ -1034,6 +1145,10 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 		thr = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BTHR);
 		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
 
+		if (fl & NFACCT_FLAG_MARK)
+			pmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_PMARK);
+
 		if (fmt == NFACCT_FMT_MAX)
 			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
 
@@ -1046,6 +1161,8 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 		} else {
 			ret = snprintf(buf, rem, NFACCT_STR_XML, nfacct_name,
 				NFACCT_GET_FMT(nfacct_get_pkt_fmt(fmt)),
+				NFACCT_PRINT_FL_XML(fl & NFACCT_FLAG_MARK,
+						    pkts, pmark),
 				pn.str,
 				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
 				NFACCT_PRINT_FL_XML(fl & NFACCT_FLAG_BTHR,
@@ -1211,6 +1328,12 @@ void nfacct_nlmsg_build_payload(struct nlmsghdr *nlh, struct nfacct *nfacct)
 
 	if (nfacct->bitset & (1 << NFACCT_ATTR_CMD))
 		mnl_attr_put_u16(nlh, NFACCT_CMD, htobe16(nfacct->cmd));
+
+	if (nfacct->bitset & (1 << NFACCT_ATTR_PMARK))
+		mnl_attr_put_u64(nlh, NFACCT_PMARK, htobe64(nfacct->pmark));
+
+	if (nfacct->bitset & (1 << NFACCT_ATTR_BMARK))
+		mnl_attr_put_u64(nlh, NFACCT_BMARK, htobe64(nfacct->bmark));
 }
 EXPORT_SYMBOL(nfacct_nlmsg_build_payload);
 
@@ -1232,6 +1355,8 @@ static int nfacct_nlmsg_parse_attr_cb(const struct nlattr *attr, void *data)
 	case NFACCT_PKTS:
 	case NFACCT_BYTES:
 	case NFACCT_BTHR:
+	case NFACCT_PMARK:
+	case NFACCT_BMARK:
 		if (mnl_attr_validate(attr, MNL_TYPE_U64) < 0) {
 			perror("mnl_attr_validate");
 			return MNL_CB_ERROR;
@@ -1286,6 +1411,12 @@ nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct)
 	if (tb[NFACCT_CMD])
 		nfacct_attr_set_u16(nfacct, NFACCT_ATTR_CMD,
 				be16toh(mnl_attr_get_u16(tb[NFACCT_CMD])));
+	if (tb[NFACCT_PMARK])
+		nfacct_attr_set_u64(nfacct, NFACCT_ATTR_PMARK,
+				be64toh(mnl_attr_get_u64(tb[NFACCT_PMARK])));
+	if (tb[NFACCT_BMARK])
+		nfacct_attr_set_u64(nfacct, NFACCT_ATTR_BMARK,
+				be64toh(mnl_attr_get_u64(tb[NFACCT_BMARK])));
 
 	return 0;
 }
-- 
1.8.3.1

[PATCH v3 nfacct 26/29] add setmark and clrmark to "get" and "list" commands

[Michael Zintakis]

* add "setmark" (to set a mark) and "clrmark" (to cleaar existing mark, if
present) options to the "get" and "list" commands. Setting a mark allows
short-term traffic to be reported via the pmark and bmark nfacct object
properties. If *any* traffic has passed through a given accounting object
since a mark has been placed, that is indicated with a plus (+) sign next to
the packet counters. The exact value of this traffic (i.e. how much traffic
has passed through since a "mark" was placed) could be shown using
"show extended" option.

* existing "save" and "restore" commands have been modofied to take into
account the new nfacct object properties and data integrity checks have
also been implemented.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/linux/netfilter/nfnetlink_acct.h |   8 ++-
 src/nfacct.c                             | 115 ++++++++++++++++++++++++++-----
 2 files changed, 104 insertions(+), 19 deletions(-)

diff --git a/include/linux/netfilter/nfnetlink_acct.h b/include/linux/netfilter/nfnetlink_acct.h
index 5d64afa..ace3428 100644
--- a/include/linux/netfilter/nfnetlink_acct.h
+++ b/include/linux/netfilter/nfnetlink_acct.h
@@ -10,6 +10,8 @@ enum nfnl_acct_msg_types {
 	NFNL_MSG_ACCT_GET,
 	NFNL_MSG_ACCT_GET_CTRZERO,
 	NFNL_MSG_ACCT_DEL,
+	NFNL_MSG_ACCT_GET_SETMARK,
+	NFNL_MSG_ACCT_GET_CLRMARK,
 	NFNL_MSG_ACCT_MAX
 };
 
@@ -23,6 +25,8 @@ enum nfnl_acct_type {
 	NFACCT_FMT,
 	NFACCT_FLAGS,
 	NFACCT_CMD,
+	NFACCT_PMARK,
+	NFACCT_BMARK,
 	__NFACCT_MAX
 };
 #define NFACCT_MAX (__NFACCT_MAX - 1)
@@ -30,7 +34,9 @@ enum nfnl_acct_type {
 enum nfnl_acct_flags {
 	NFACCT_FLAG_BIT_BTHR 	= 0,
 	NFACCT_FLAG_BTHR	= (1 << NFACCT_FLAG_BIT_BTHR),
-	NFACCT_FLAG_BIT_MAX	= 1,
+	NFACCT_FLAG_BIT_MARK	= 1,
+	NFACCT_FLAG_MARK	= (1 << NFACCT_FLAG_BIT_MARK),
+	NFACCT_FLAG_BIT_MAX	= 2,
 	NFACCT_FLAG_MAX		= (1 << NFACCT_FLAG_BIT_MAX),
 };
 
diff --git a/src/nfacct.c b/src/nfacct.c
index 4504143..2cb0256 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -57,7 +57,9 @@ enum nfacct_sort_mode {
 	NFACCT_SORT_NONE = 0,
 	NFACCT_SORT_NAME,
 	NFACCT_SORT_PKTS,
+	NFACCT_SORT_PMARK,
 	NFACCT_SORT_BYTES,
+	NFACCT_SORT_BMARK,
 	NFACCT_SORT_BTHR,
 };
 
@@ -75,9 +77,15 @@ static int nfacct_cmp(void *priv, struct nfacct_list_head *a, struct nfacct_list
 	case NFACCT_SORT_PKTS:
 				attr = NFACCT_ATTR_PKTS;
 				break;
+	case NFACCT_SORT_PMARK:
+				attr = NFACCT_ATTR_PMARK;
+				break;
 	case NFACCT_SORT_BYTES:
 				attr = NFACCT_ATTR_BYTES;
 				break;
+	case NFACCT_SORT_BMARK:
+				attr = NFACCT_ATTR_BMARK;
+				break;
 	case NFACCT_SORT_BTHR:
 				attr = NFACCT_ATTR_BTHR;
 	case NFACCT_SORT_NAME:
@@ -108,9 +116,9 @@ static int nfacct_cmp(void *priv, struct nfacct_list_head *a, struct nfacct_list
 /*
  * maximum total of columns to be shown, except the
  * "name" column as that is not width-dependent:
- * "pkts bytes bthr"
+ * "pkts pmark bytes bmark bthr"
  */
-#define NFACCT_MAX_COLUMNS 3
+#define NFACCT_MAX_COLUMNS 5
 
 /* stores nfacct options for snprintf_* and nfacct_cb functions */
 static struct nfacct_options *options;
@@ -209,7 +217,9 @@ static int nfacct_cb(const struct nlmsghdr *nlh, void *data)
 	bool *ignore_col_width = (bool *)data;
 	static const enum nfacct_option_type o_num[NFACCT_MAX_COLUMNS] =
 				{ NFACCT_OPT_PCW,
+				  NFACCT_OPT_PMCW,
 				  NFACCT_OPT_BCW,
+				  NFACCT_OPT_BMCW,
 				  NFACCT_OPT_BTCW,
 				};
 
@@ -283,6 +293,12 @@ static int nfacct_cmd_list(int argc, char *argv[])
 		if (!nfnl_msg && nfacct_matches(argv[0],"reset")) {
 			nfnl_cmd = NFNL_MSG_ACCT_GET_CTRZERO;
 			nfnl_msg = true;
+		} else if (!nfnl_msg && nfacct_matches(argv[0],"clrmark")) {
+			nfnl_cmd = NFNL_MSG_ACCT_GET_CLRMARK;
+			nfnl_msg = true;
+		} else if (!nfnl_msg && nfacct_matches(argv[0],"setmark")) {
+			nfnl_cmd = NFNL_MSG_ACCT_GET_SETMARK;
+			nfnl_msg = true;
 		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
 		} else if (!b_sh && nfacct_matches(argv[0],"show")) {
@@ -302,8 +318,12 @@ static int nfacct_cmd_list(int argc, char *argv[])
 			} else if (nfacct_matches(argv[0],"packets") ||
 				   nfacct_matches(argv[0],"pkts")) {
 				sort_mode = NFACCT_SORT_PKTS;
+			} else if (nfacct_matches(argv[0],"pmark")) {
+				sort_mode = NFACCT_SORT_PMARK;
 			} else if (nfacct_matches(argv[0],"bytes")) {
 				sort_mode = NFACCT_SORT_BYTES;
+			} else if (nfacct_matches(argv[0],"bmark")) {
+				sort_mode = NFACCT_SORT_BMARK;
 			} else if (nfacct_matches(argv[0],"threshold")) {
 				sort_mode = NFACCT_SORT_BTHR;
 			} else if (nfacct_matches(argv[0],"none")) {
@@ -613,6 +633,12 @@ static int nfacct_cmd_get(int argc, char *argv[])
 		if (!nfnl_msg && nfacct_matches(argv[0],"reset")) {
 			nfnl_cmd = NFNL_MSG_ACCT_GET_CTRZERO;
 			nfnl_msg = true;
+		} else if (!nfnl_msg && nfacct_matches(argv[0],"clrmark")) {
+			nfnl_cmd = NFNL_MSG_ACCT_GET_CLRMARK;
+			nfnl_msg = true;
+		} else if (!nfnl_msg && nfacct_matches(argv[0],"setmark")) {
+			nfnl_cmd = NFNL_MSG_ACCT_GET_SETMARK;
+			nfnl_msg = true;
 		} else if (!xml && nfacct_matches(argv[0],"xml")) {
 			xml = true;
 		} else if (!b_sh && nfacct_matches(argv[0],"show")) {
@@ -792,15 +818,17 @@ static const char help_msg[] =
 	"  version\t\tDisplay version and disclaimer\n"
 	"  help\t\t\tDisplay this help message\n\n"
 	"Parameters:\n"
-	"  LST_PARAMS := [ reset ] [ show SHOW_SPEC ] [ format FMT_SPEC ]\n"
-			"\t\t[ sort SORT_SPEC ] [ xml ]\n"
+	"  LST_PARAMS := [ reset | setmark | clrmark ] [ show SHOW_SPEC ]\n"
+			"\t\t[ format FMT_SPEC ] [ sort SORT_SPEC ] [ xml ]\n"
 	"  ADD_PARAMS := [ replace ] [ format FMT_SPEC ] "
 			"[ threshold [NUMBER | '-'] ]\n"
-	"  GET_PARAMS := [ reset ] [ show SHOW_SPEC ] [ format FMT_SPEC ] [ xml ]\n"
+	"  GET_PARAMS := [ reset | setmark | clrmark ] [ show SHOW_SPEC ]\n"
+			"\t\t[ format FMT_SPEC ] [ xml ]\n"
 	"  RST_PARAMS := [ flush ] [ replace ]\n"
 	"  SHOW_SPEC := { bytes | extended }\n"
 	"  FMT_SPEC := { [FMT] | [,] | [FMT] ... }\n"
-	"  SORT_SPEC := { none | name | packets | bytes | threshold }"
+	"  SORT_SPEC := { none | name | packets | pmark | bytes | bmark |"
+			" threshold }\n"
 	"  FMT := { def | raw | 3pl | iec | kib | mib | gib | tib | pib |"
 		  " eib |\n"
 	"  \t   si | kb | mb | gb | tb | pb | eb }\n";
@@ -885,10 +913,10 @@ err:
 
 /*
  * Maximum number of restore tokens accepted:
- * name= fmt= pkts= bytes= thr=
+ * name= fmt= pkts= bytes= pmark= bmark= thr=
  *
  */
-#define NFACCT_MAX_TOKENS 5
+#define NFACCT_MAX_TOKENS 7
 
 /*
  * Maximum number of value tokens accepted:
@@ -919,9 +947,9 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 {
 	bool replace = false, flush = false;
 	bool b_name = false, b_fmt = false, b_pkts = false, b_bytes = false;
-	bool b_thr = false;
+	bool b_pmark = false, b_bmark = false, b_thr = false;
 	uint16_t cmd = 0, fmt = NFACCT_FMT_DEFAULT;
-	uint64_t pkts = 0, bytes = 0, thr = 0;
+	uint64_t pkts = 0, pmark = 0, bytes = 0, bmark = 0, thr = 0;
 	char *tokens[NFACCT_MAX_TOKENS + 1];
 	char *vtokens[NFACCT_MAX_VTOKENS + 1];
 	char buf[MAX_TOKEN_SIZE];
@@ -953,9 +981,11 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 		}
 	}
 
-	for (; fgets(buf, ARRAY_SIZE(buf), stdin); fmt = NFACCT_FMT_DEFAULT,
-	     pkts = 0, bytes = 0, thr = 0, b_name = false, b_fmt = false,
-	     b_pkts = false, b_bytes = false, b_thr = false, line++) {
+	for (; fgets(buf, ARRAY_SIZE(buf), stdin); cmd = 0,
+	     fmt = NFACCT_FMT_DEFAULT, pkts = 0, pmark = 0, bytes = 0,
+	     bmark = 0, thr = 0, b_name = false, b_fmt = false,
+	     b_pkts = false, b_bytes = false, b_pmark = false,
+	     b_bmark = false, b_thr = false, line++) {
 		ret = nfacct_parse_tokens(buf, " \n", NFACCT_MAX_TOKENS + 1,
 					  true, tokens);
 		if (ret == 0)
@@ -1022,6 +1052,30 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 				nfacct_attr_set_u64(nfacct,
 						    NFACCT_ATTR_BYTES, bytes);
 				b_bytes = true;
+			} else if (!b_pmark && strncmp(vtokens[0], "pmark",
+					         strlen("pmark") + 1) == 0) {
+				if (nfacct_get_uint64_t(&pmark,
+						 vtokens[1]) != 0) {
+					NFACCT_PRINT_VERR("error on line %d: "
+						"invalid 'pmark' token (%s)",
+						vtokens[1]);
+				}
+				cmd |= NFACCT_FLAG_MARK;
+				nfacct_attr_set_u64(nfacct,
+						    NFACCT_ATTR_PMARK, pmark);
+				b_pmark = true;
+			} else if (!b_bmark && strncmp(vtokens[0], "bmark",
+					         strlen("bmark") + 1) == 0) {
+				if (nfacct_get_uint64_t(&bmark,
+						 vtokens[1]) != 0) {
+					NFACCT_PRINT_VERR("error on line %d: "
+						"invalid 'bmark' token (%s)",
+						vtokens[1]);
+				}
+				cmd |= NFACCT_FLAG_MARK;
+				nfacct_attr_set_u64(nfacct,
+						    NFACCT_ATTR_BMARK, bmark);
+				b_bmark = true;
 			} else if (!b_thr && strncmp(vtokens[0], "thr",
 					   	strlen("thr") + 1) == 0) {
 				if (nfacct_get_uint64_t(&thr,
@@ -1047,19 +1101,44 @@ static int nfacct_cmd_restore(int argc, char *argv[])
 					"invalid 'name' token (%s)",
 					"not set");
 		}
-		if (nfacct_attr_get(nfacct, NFACCT_ATTR_BYTES) &&
-		    !nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS)) {
+		if ((nfacct_attr_get(nfacct, NFACCT_ATTR_BYTES) &&
+		     !nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS)) ||
+		    (cmd & NFACCT_FLAG_MARK &&
+		     !nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS))) {
 			NFACCT_PRINT_CERR("error on line %d: "
 					"invalid 'pkts' token (%s)",
 					"not set");
 		}
-		if ((nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS) ||
-		     cmd & NFACCT_FLAG_BTHR) &&
-		    !nfacct_attr_get(nfacct, NFACCT_ATTR_BYTES)) {
+		if ((nfacct_attr_get(nfacct, NFACCT_ATTR_PKTS) &&
+		    !nfacct_attr_get(nfacct, NFACCT_ATTR_BYTES)) ||
+		    ((cmd & NFACCT_FLAG_BTHR || cmd & NFACCT_FLAG_MARK) &&
+		    !nfacct_attr_get(nfacct, NFACCT_ATTR_BYTES))) {
 			NFACCT_PRINT_CERR("error on line %d: "
 					"invalid 'bytes' token (%s)",
 					"not set");
 		}
+		if (cmd & NFACCT_FLAG_MARK) {
+			if (!nfacct_attr_get(nfacct, NFACCT_ATTR_PMARK)) {
+				NFACCT_PRINT_CERR("error on line %d: "
+						"invalid 'pmark' token (%s)",
+						"not set");
+			}
+			if (!nfacct_attr_get(nfacct, NFACCT_ATTR_BMARK)) {
+				NFACCT_PRINT_CERR("error on line %d: "
+						"invalid 'bmark' token (%s)",
+						"not set");
+			}
+			if (pmark > pkts) {
+				NFACCT_PRINT_CERR("error on line %d: "
+						"invalid 'pmark' token (%s)",
+						"> 'pkts'");
+			}
+			if (bmark > bytes) {
+				NFACCT_PRINT_CERR("error on line %d: "
+						"invalid 'bmark' token (%s)",
+						"> 'bytes'");
+			}
+		}
 		NFACCT_FREE_TOKENS;
 
 		if (cmd) {
-- 
1.8.3.1

[PATCH v3 libnetfilter_acct 27/29] add *_MONLY template support

[Michael Zintakis]

* add NFACCT_SNPRINTF_F_MONLY plaintext and xml templates, allowing
printing of mark-only nfacct object attriutes - name, pmark and bmark -
showing the amount of traffic which passed through each nfacct object
since the last mark was set, allowing for short-term traffic analysis, in
addition to the main traffic. Example:

[ pmark = 8,629,324  bmark = 4.447GiB ] = "ALL WEB"

<?xml version="1.0" encoding="utf-8"?>
<nfacct>
<obj>
<name>ALL WEB</name>
<pmark fmt="3pl">8629324</pmark>
<bmark fmt="iec">4774467072</bmark>
</obj>
</nfacct>

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 include/libnetfilter_acct/libnetfilter_acct.h |  3 ++
 src/libnetfilter_acct.c                       | 62 +++++++++++++++++++++++++++
 2 files changed, 65 insertions(+)

diff --git a/include/libnetfilter_acct/libnetfilter_acct.h b/include/libnetfilter_acct/libnetfilter_acct.h
index a58d93f..9030f64 100644
--- a/include/libnetfilter_acct/libnetfilter_acct.h
+++ b/include/libnetfilter_acct/libnetfilter_acct.h
@@ -131,6 +131,9 @@ int nfacct_nlmsg_parse_payload(const struct nlmsghdr *nlh, struct nfacct *nfacct
 /* print packets, pmark, bytes, bmark, threshold and name columns */
 #define NFACCT_SNPRINTF_F_EXTENDED	(1 << 5)
 
+/* print pmark, bmark and name columns */
+#define NFACCT_SNPRINTF_F_MONLY		(1 << 6)
+
 #define NFACCT_SNPRINTF_T_PLAIN 0
 #define NFACCT_SNPRINTF_T_XML 1
 
diff --git a/src/libnetfilter_acct.c b/src/libnetfilter_acct.c
index 8773710..e57cb16 100644
--- a/src/libnetfilter_acct.c
+++ b/src/libnetfilter_acct.c
@@ -569,6 +569,8 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define NFACCT_STR_PLAIN		"{ pkts = %%%zus%s, " \
 					"bytes = %%%zus%s } = %%s;"
 #define NFACCT_STR_PLAIN_BONLY		"[ bytes = %%%zus%s ] = %%s"
+#define NFACCT_STR_PLAIN_MONLY		"[ pmark = %%%zus  bmark = %%%zus ]" \
+					" = %%s"
 #define NFACCT_XML_NAME			"<name>%s</name>"
 #define NFACCT_XML_PKTS			"<pkts fmt=\"%s\" " \
 					"mark=\"%s\">%s</pkts>"
@@ -579,6 +581,8 @@ EXPORT_SYMBOL(nfacct_option_get_tsize);
 #define NFACCT_XML_BTHR			"<threshold fmt=\"%s\">%s</threshold>"
 #define NFACCT_STR_XML_BONLY		"<obj>"	NFACCT_XML_NAME \
 					NFACCT_XML_BYTES
+#define NFACCT_STR_XML_MONLY		"<obj>"	NFACCT_XML_NAME \
+					NFACCT_XML_PMARK NFACCT_XML_BMARK
 #define NFACCT_STR_XML_COMPAT		"<obj><name>%s</name>"	\
 					"<pkts>%s</pkts>"	\
 					"<bytes>%s</bytes>"
@@ -1013,6 +1017,39 @@ nfacct_snprintf_plain(char *buf, size_t rem, struct nfacct *nfacct,
 			 bmcw, btcw);
 		ret = snprintf(buf, rem, fmt_str, pn.str, pm.str, bn.str,
 				bm.str, tn.str, nfacct_name);
+	} else if (flags & NFACCT_SNPRINTF_F_MONLY) {
+		/* print pmark + bmark + name */
+		pmcw = nfacct_option_get_tsize(options,	NFACCT_OPT_PMCW);
+		bmcw = nfacct_option_get_tsize(options,	NFACCT_OPT_BMCW);
+		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
+		parse_nfacct_name(nfacct_name,
+				nfacct_attr_get_str(nfacct,
+						    NFACCT_ATTR_NAME));
+		fmt = nfacct_option_get_u16(options, NFACCT_OPT_FMT);
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
+
+		if (nfacct_get_fmt(fmt) > NFACCT_FMT_NONE)
+			init_locale();
+
+		if (fl & NFACCT_FLAG_MARK) {
+			pmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_PMARK);
+			bmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_BMARK);
+			format_number(&pm, pkts-pmark < 0 ? 0 : pkts-pmark,
+				      nfacct_get_pkt_fmt(fmt));
+			format_number(&bm, bytes-bmark < 0 ? 0 : bytes-bmark,
+				      nfacct_get_bytes_fmt(fmt));
+		}
+
+		snprintf(fmt_str, sizeof(fmt_str), NFACCT_STR_PLAIN_MONLY,
+			 pmcw, bmcw);
+		ret = snprintf(buf, rem, fmt_str, pm.str, bm.str,
+				nfacct_name);
 	} else {
 		/* print out name only */
 		parse_nfacct_name(nfacct_name,
@@ -1138,6 +1175,31 @@ nfacct_snprintf_xml(char *buf, size_t rem, struct nfacct *nfacct,
 				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
 				tn.str);
 		BUFFER_SIZE(ret, size, rem, offset);
+	} else if (flags & NFACCT_SNPRINTF_F_MONLY) {
+		/* print pmark + bmark + name */
+		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
+		bytes = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_BYTES);
+		fl = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FLAGS);
+
+		if (fmt == NFACCT_FMT_MAX)
+			fmt = nfacct_attr_get_u16(nfacct, NFACCT_ATTR_FMT);
+
+		if (fl & NFACCT_FLAG_MARK) {
+			pmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_PMARK);
+			bmark = nfacct_attr_get_u64(nfacct,
+						    NFACCT_ATTR_BMARK);
+			format_number(&pm, pkts-pmark < 0 ? 0 : pkts-pmark,
+				      NFACCT_FMT_NONE);
+			format_number(&bm, bytes-bmark < 0 ? 0 : bytes-bmark,
+				      NFACCT_FMT_NONE);
+		}
+		ret = snprintf(buf, rem, NFACCT_STR_XML_MONLY, nfacct_name,
+				NFACCT_GET_FMT(nfacct_get_pkt_fmt(fmt)),
+				pm.str,
+				NFACCT_GET_FMT(nfacct_get_bytes_fmt(fmt)),
+				bm.str);
+		BUFFER_SIZE(ret, size, rem, offset);
 	} else {
 		/* default/everything else: print name + pkts + bytes */
 		pkts = nfacct_attr_get_u64(nfacct, NFACCT_ATTR_PKTS);
-- 
1.8.3.1

[PATCH v3 nfacct 28/29] add "show marks" option to "list" and "get" commands

[Michael Zintakis]

* add "show marks" option to "list" and "get" commands, allwoing only
mark-specific nfacct object properties to be shown - pmark (the amount of
packets passed through the accounting object since a "mark" was set), bmark
(the amount of bytes passed through the accounting object since a "mark" was
set) and name. The formatting used for showing pmark and bmark is the same as
pkts and bytes properties respectively, unless overwritten with the "format"
option.

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 src/nfacct.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/src/nfacct.c b/src/nfacct.c
index 2cb0256..4ce6854 100644
--- a/src/nfacct.c
+++ b/src/nfacct.c
@@ -305,6 +305,8 @@ static int nfacct_cmd_list(int argc, char *argv[])
 			NFACCT_GET_NEXT_ARG();
 			if (nfacct_matches(argv[0],"bytes")) {
 				flags = NFACCT_SNPRINTF_F_BONLY;
+			} else if (nfacct_matches(argv[0],"marks")) {
+				flags = NFACCT_SNPRINTF_F_MONLY;
 			} else if (nfacct_matches(argv[0],"extended")) {
 				flags = NFACCT_SNPRINTF_F_EXTENDED;
 			} else {
@@ -645,6 +647,8 @@ static int nfacct_cmd_get(int argc, char *argv[])
 			NFACCT_GET_NEXT_ARG();
 			if (nfacct_matches(argv[0],"bytes")) {
 				flags = NFACCT_SNPRINTF_F_BONLY;
+			} else if (nfacct_matches(argv[0],"marks")) {
+				flags = NFACCT_SNPRINTF_F_MONLY;
 			} else if (nfacct_matches(argv[0],"extended")) {
 				flags = NFACCT_SNPRINTF_F_EXTENDED;
 			} else {
@@ -825,7 +829,7 @@ static const char help_msg[] =
 	"  GET_PARAMS := [ reset | setmark | clrmark ] [ show SHOW_SPEC ]\n"
 			"\t\t[ format FMT_SPEC ] [ xml ]\n"
 	"  RST_PARAMS := [ flush ] [ replace ]\n"
-	"  SHOW_SPEC := { bytes | extended }\n"
+	"  SHOW_SPEC := { bytes | marks | extended }\n"
 	"  FMT_SPEC := { [FMT] | [,] | [FMT] ... }\n"
 	"  SORT_SPEC := { none | name | packets | pmark | bytes | bmark |"
 			" threshold }\n"
-- 
1.8.3.1

[PATCH v3 nfacct 29/29] change man page to describe all new features

[Michael Zintakis]

* completely redesign the existing man page to describe in great details all
new features added in this patch set

Signed-off-by: Michael Zintakis <michael.zintakis@googlemail.com>
---
 nfacct.8 | 551 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 530 insertions(+), 21 deletions(-)

diff --git a/nfacct.8 b/nfacct.8
index 0c3249c..86b4953 100644
--- a/nfacct.8
+++ b/nfacct.8
@@ -1,59 +1,568 @@
-.TH NFACCT 8 "Dec 30, 2011" "" ""
+.TH NFACCT 8 "02 June 2013" "" ""
 
 .\" Man page written by Pablo Neira Ayuso <pablo@netfilter.org> (Dec 2011)
 
 .SH NAME
-nfacct \- command line tool to create/retrieve/delete accounting objects
+nfacct \- command line tool to create/retrieve/delete and manipulate
+accounting objects.
 .SH SYNOPSIS
-.BR "nfacct command [name] [options]"
+.BR "nfacct list [reset|setmark|clrmark] [show {bytes|marks|extended}] [format FORMAT] [sort SORT] [xml]"
+
+.BR "nfacct add name [replace] [format FORMAT] [threshold [NUMBER|'\-']]"
+
+.BR "nfacct get name [reset|setmark|clrmark] [show {bytes|marks|extended}] [format FORMAT] [xml]"
+
+.BR "nfacct flush"
+
+.BR "nfacct save"
+
+.BR "nfacct restore [flush] [replace]"
+
+.BR "nfacct version"
+
+.BR "nfacct help"
+
 .SH DESCRIPTION
 .B nfacct
-is the command line tool that allows to manipulate the Netfilter's extended
+is the command line tool that allows manipulation of Netfilter's extended
 accounting infrastructure.
 .SH COMMANDS
-These commands specify the action that nfacct performs. Only one of them can be
-specified at any given time.
+These commands specify the action that nfacct performs. Only one of them can
+be specified at any given time. The following commands can be used:
 .TP
 .BI "list "
-List the existing accounting object in table.
+List the existing accounting objects in the accounting table. Depending on
+the \fBshow\fR parameter used (see \fBlist command parameters\fR below),
+this command could list up to 8 different properties for each accounting
+object:
+.RS 8
 .TP
-.BI "restore "
-Restore accounting object table by reading from stdin. Input format is the one used
-as output by the list command.
+.BI "pkts "
+Current number of packets counted by this accounting object. This number can
+be formatted independently for each accounting object (see \fBFORMAT
+OPTIONS\fR section below).
+.TP
+.BI "mark exceeded indicator "
+This is only applicable if mark has been set against this accounting object
+(see \fBMARKING\fR section below). If any traffic has
+passed through since that mark was last set, then a plus sign (\fB+\fR) is
+shown at the end of the \fBpkts\fR column.
+.TP
+.BI "pmark "
+This shows the number of packets which passed through this accounting object
+since the last time a mark was set. The format used to display this property
+is the same as the \fBpkts\fR format. If no marking has been enabled for
+this accounting object, then a dash (\fB\-\fR) is shown.
+.TP
+.BI "bytes "
+Number of bytes counted by this accounting object. Again, this number can be
+formatted independently for each accounting object.
+.TP
+.BI "bytes threshold exceeded indicator "
+This is only applicable if the bytes threshold is enabled and set. If this
+threshold has been exceeded (in other words, the number of bytes counted is
+greater than the \fBbytes threshold\fR), then a plus sign (\fB+\fR) is shown
+at the end of the bytes column for each accounting object. If either the
+bytes threshold is disabled or the current bytes counter is less than the
+bytes threshold value, then nothing is shown.
+.TP
+.BI "bmark "
+This shows the number of bytes which passed through this accounting object
+since the last time a mark was set. The format used to display this property
+is the same as the \fBbytes\fR format. If no marking has been enabled for
+this accounting object, then a dash (\fB\-\fR) is shown.
+.TP
+.BI "bytes threshold "
+Number of bytes which are expected to pass through this accounting object.
+If this threshold is exceeded, the \fBbytes threshold exceeded indicator\fR
+is triggered. If the value of this property is shown as a dash (\fB\-\fR),
+then this feature is disabled. The format used to display this property is
+the same as the \fBbytes\fR format.
+.TP
+.BI "name "
+Name of the accounting object.
+.RE
 .TP
 .BI "add "
-Add new accounting object.
+Adds new or changes the properties of existing accounting object in the
+accounting table.
 .TP
 .BI "delete "
-Delete accounting object.
+Delete existing accounting object from the accounting table.
 .TP
 .BI "get "
-Get existing accounting object.
+Get and display information about existing accounting object. The
+properties shown are the same as the \fBlist\fR command above. This command
+allows for some of the account object properties to be changed (see
+\fBget command parameters\fR below).
 .TP
 .BI "flush "
-Delete all unused accounting objects.
+Flush the entire accounting table, provided the accounting objects
+are not in use by iptables (if that is so, these accounting objects
+are not deleted).
+.TP
+.BI "save "
+Dump the entire accounting object table to \fIstdout\fR using format
+appropriate for use by the \fBrestore\fR command. The output of
+packet, byte and all other counters, if applicable, is not formatted. For
+more information on the format used, see \fBRESTORE FORMAT\fR
+section below.
+.TP
+.BI "restore "
+Restore accounting object table by reading from \fIstdin\fR. Input
+format is the one used as output by the \fBsave\fR command.
 .TP
 .BI "version "
-Displays the version information.
+Displays version information.
 .TP
 .BI "help "
-Displays the help message.
-.SH OPTIONS
+Displays help message.
+.SH PARAMETERS
+This tool also allows for a set of parameters to be specified,
+depending on the command used. They are as follows:
+.TP
+.BI "list command parameters "
+All parameters for the \fBlist\fR command shown below are optional.
+These are:
+.RS 8
+.TP
+.BI "reset "
+Resets the byte and packet counters on all accounting objects
+in the accounting table. Please note that this also resets the marks of the
+accounting objects, when applicable.
 .TP
-This tool also allows a couple of options with the get and list commands that are:
+.BI "setmark "
+Sets new or updates existing marks on all registered accounting objects in
+the accounting table. For more information on marking, see
+\fBMARKING\fR section below.
+.TP
+.BI "clrmark "
+Clears existing marks and disables marking on all registered accounting
+objects in the accounting table.
+.TP
+.BI "show "
+Specify what to display. Valid options are: \fIbytes\fR \- to display only
+the name and bytes counter for each accounting object; \fImarks\fR \- to
+display only the name, packet and byte counters since the last mark was set
+for each accounting object; \fIextended\fR \- to show all properties of the
+accounting objects: name, packet, byte and threshold counters, packet and
+byte counters since the last mark was set, if applicable, mark exceeded
+and threshold exceeded indicators. If this parameter is omitted, then name,
+packet and byte counters, as well as mark exceeded and bytes threshold
+exceeded indicators are shown.
+.TP
+.BI "format "
+Specify the format to use to display \fIall\fR accounting objects (that
+option takes precedence over the formatting registered for each
+accounting object when it was added). For more information on format
+options see \fBFORMAT OPTIONS\fR section below.
+.TP
+.BI "sort "
+Specify how to sort the accounting objects. Valid options are:
+\fIname\fR (default) \- to sort by nfacct name; \fIpackets\fR \- to sort by
+number of packets received; \fIpmark\fR \- to sort by number of packets
+received since last mark was set; \fIbytes\fR \- to sort by number of bytes
+received; \fIbmark\fR \- to sort by number of bytes received since last mark
+was set; \fIthreshold\fR \- to sort by bytes threshold value; or specify
+\fInone\fR for not sorting any accounting objects. Please note that all
+sorting is done in ascending order (low-high).
+.TP
+.BI "xml "
+Generate all output in xml format. Special xml characters are properly
+escaped to conform to the xml specification. Please note that \fBNO\fR
+number formatting of any kind is applied to the values of the various
+counters for each accounting object - they are shown "as\-is", while the
+formatting properties are shown as xml element attributes, allowing external
+aplication to format these numbers accordningly. All other indicators, like
+\fBmark exceeded indicator\fR, as well as
+\fBbytes threshold exceeded indicator\fR are also shown as xml element
+attributes.
+.RE
+.TP
+.BI "add command parameters "
+All parameters for the \fBadd\fR command shown below with the exception of
+\fIname\fR are optional. These are:
+.RS 8
+.TP
+.BI "name "
+Name of the accounting object to add or replace in the accounting table.
+This should be between 1 and 31 characters long (if it is longer than 31
+characters, it is automatically truncated). This parameter is mandatory.
+.TP
+.BI "replace "
+If specified, it replaces the accounting object if it already exists. If
+this parametter is omitted and the accounting object already exists in
+the accounting table, then an error is returned.
+.TP
+.BI "format "
+Set the format to use to display packets, bytes and bytes threshold
+numbers for this accounting object with the \fBget\fR and \fBlist\fR
+commands. For more information on format options see
+\fBFORMAT OPTIONS\fR section below.
+.TP
+.BI "threshold "
+Set the bytes threshold for this accounting object. If this threshold is
+exceeded, then the bytes threshold exceeded indicator is triggered.
+The number specified must be in bytes, regardless of the format used to
+display the bytes counter. If dash (\fB\-\fR) is specified (the default),
+then the bytes threshold feature is disabled (this could later be enabled
+with \fBadd replace\fR).
+.RE
+.TP
+.BI "get command parameters "
+All parameters for the \fBget\fR command shown below with the exception of
+\fIname\fR are optional. These are:
+.RS 8
+.TP
+.BI "name "
+Name of the accounting object to get and display information of.
+This paramenter is mandatory.
 .TP
 .BI "reset "
-That atomically obtains and resets the counters.
+Resets the byte and packet counters on this accounting objects in the
+accounting table. The byte and packet counters shown will be
+\fIbefore\fR they were reset. Please note that this does \fInot\fR
+reset the byte threshold value of the accounting object, if enabled \-
+this needs to be done with the \fBadd\fR command (using \fIreplace\fR
+parameter). Mark counters, if enabled for this accounting object are
+also reset.
+.TP
+.BI "setmark "
+Sets new or updates existing mark on this accounting object. For more
+information on marking, see \fBMARKING\fR section
+below.
+.TP
+.BI "clrmark "
+Clears any existing mark and disables marking on this accounting object.
+.TP
+.BI "show "
+Specify what properties of the accounting object to display. Valid
+options are: \fIbytes\fR \- to display only the name and bytes counter
+for this accounting object; \fImarks\fR \- to display only the name, packet
+and byte counters since the last mark was set for this accounting object;
+\fIextended\fR \- to show all properties of this accounting object: name,
+packet, byte and threshold counters, packet and byte counters since the
+last mark was set, if applicable, mark exceeded and threshold exceeded
+indicators. If this parameter is omitted, then name, packet and byte counters,
+as well as mark exceeded and bytes threshold exceeded indicators are shown.
+.TP
+.BI "format "
+Specify the format to use to display this accounting object (that option
+takes precedence over the formatting registered when the specified
+accounting object was added). For more information on format options see
+\fBFORMAT OPTIONS\fR section below.
 .TP
 .BI "xml "
-That displays the output in XML format.
+Generate all output in xml format. Special xml characters are properly
+escaped to conform to the xml specification.  Please note that \fBNO\fR
+number formatting of any kind is applied to the values of the various
+counters for each accounting object - they are shown "as\-is", while
+the formatting properties are shown as xml element attributes, allowing
+external aplication to format these numbers accordningly. All other
+indicators, like \fBmark exceeded indicator\fR, as well as
+\fBbytes threshold exceeded indicator\fR are also shown as xml element
+attributes.
+.RE
+.TP
+.BI "restore command parameters "
+All parameters for the \fBrestore\fR command are optional. These are:
+.RS 8
+.TP
+.BI "flush "
+If specified, it flushes the entire accounting table prior to restoring
+the accounting objects. If this option is omitted, then no flush is
+performed and the accounting objects are simply added to the accounting
+table if they don't already exist.
+.TP
+.BI "replace "
+If specified, it replaces the properties of existing accounting objects
+even if they are used in iptables. If this option is \fInot\fR specified
+and that accounting object is used in iptables, then the properties of
+that object are \fInot\fR updated!
+.RE
+.TP
+.BI " "
+\fBPLEASE NOTE:\fR  In order to restore the \fBentire\fR accounting table
+to the state it was in when the save command was executed, even if the
+accounting objects are currently \fBin use by iptables\fR, both
+\fIreplace\fR and \fIflush\fR parameters need to be specified. For more
+information on the restore format, see \fBRESTORE FORMAT\fR section below.
+.SH FORMAT OPTIONS
+This tool allows for a specific number formatting to be specified
+independently for each accounting object when it is added to the
+accounting table. These format options consist of two separate parts:
+\fIpackets\fR count number formatting, which also includes the packets
+count since mark was last applied, if applicable, and \fIbytes\fR count
+number formatting, which includes bytes count since mark was last applied,
+as well as bytes threshold, if applicable.
+.TP
+Currently, the following types of number formatting are available:
+.TP
+.BI "def "
+Default: default formatting to use. This is the format applied if no
+formatting option is specified. Numbers are formatted so that they
+always occupy 20 characters and have leading zeroes, e.g.
+123 is shown as 00000000000000000123.
+.TP
+.BI "raw "
+Raw: no formatting of any kind is applied. The number is shown 'as-is',
+without any leading or trailing zeroes or with any format applied to it,
+e.g. 123 is shown as 123.
+.TP
+.BI "3pl "
+Triplets: number is formatted in \fItriplets\fR so that each 3 digits are
+separated with the \fIthousand separator\fR registered with the current
+locale in the system, e.g. 1234567 is shown as 1,234,567 (assuming that ','
+is the current locale thousand separator symbol \- see notes below).
+.TP
+.BI "iec "
+IEC: apply the \fBIEC standard\fR for number formatting, depending on the
+current value, with 3 decimal places. e.g 8305798 is shown as 7.921MiB,
+while 2106468480 is shown as 1.962GiB (again, assuming the ',' and '.' are
+the current locale thousand separator and decimal symbols \- see notes
+below).
+.TP
+.BI "kib, mib, gib, tib, pib and eib "
+IEC KiB, MiB, GiB, TiB, PiB and EiB number format specifications: apply the
+specific IEC number formatting (kibibyte, mibibyte and so on) regardless of
+the current value, with 3 decimal places, e.g. assuming we would like to
+use the \fImib\fR format (in other words, lock the number output to appear
+in IEC mibibytes), then 8305798 is shown as 7.921MiB, while 2106468480 is
+shown as 2,008.885MiB.
+.TP
+.BI "si "
+SI: apply the (old) SI standard for number formatting, depending on the
+current value, with 3 decimal places, e.g. 8305798 is shown as 8.306MB,
+while 2106468480 is shown as 2.106GB.
+.TP
+.BI "kb, mb, gb, tb, pb and eb "
+SI KB, MB, GB, TB, PB and EB number format specifications: apply the
+specific SI number formatting (kilobyte, megabyte and so on) regardless
+of the current value, with 3 decimal places, e.g. assuming we would like
+to use the \fImb\fR format (in other words, lock the number output to
+appear in SI megabytes), then 8305798 is shown as 8.306MB, while
+2106468480 is shown as 2,106.469MB.
+.RE
+.TP
+.BI "PLEASE NOTE: "
+With the exception of the default (\fIdef\fR) and raw (\fIraw\fR)
+formatting options, all other formatting options are \fBlocale-specific\fR
+as they are dependent on the \fIdecimal sign\fR and
+\fIthousand separator\fR symbols set by the operating system. If the current
+locale on the machine where nfacct is executed cannot be determined,
+then \fBen_GB\fR locale is used.
+.TP
+.BI " "
+Also, if a particular formatting option is not specified, then the default
+(\fIdef\fR) format is applied as described above.
+.TP
+.BI " "
+For example: when '\fB,mib\fR' formatting option is specified, this is the
+equivalent of '\fBdef,mib\fR' (in other words, use \fIdef\fR for packets
+counter number formatting and \fImib\fR for bytes counter number
+formatting).
+.TP
+.BI " "
+Similarly when '\fB3pl,\fR' is specified, this is the equivalent
+of '\fB3pl,def\fR' (use \fI3pl\fR for packets counter number
+formatting and \fIdef\fR for bytes counter number formatting) and '\fB,\fR'
+is the equivalent of '\fBdef,def\fR', which is also the formatting used
+if \fIno\fR formatting options of any kind are specified.
+.TP
+.BI " "
+Finally, if just one formatting option is specified, then that option is
+applied to \fIboth\fR packet and byte numbers. In other words, when
+just '\fBraw\fR' is specified, this is the equivalent to '\fBraw,raw\fR'
+format option.
+.TP
+Examples:
+.TP
+.BI " "
+To \fIadd\fR accounting object called '\fBinward\fR' and use '\fB3pl\fR'
+for packet formatting and '\fBiec\fR' for bytes formatting, as well as set
+the threshold value to \fB1GiB\fR, execute the following command:
+.RS 8
+.TP
+.BI " "
+.BI "nfacct add inward format 3pl,iec threshold 1073741824 "
+.RE
+.TP
+.BI " "
+To \fIchange\fR the formatting properties of
+existing accounting object named '\fBnet 27\fR' with '\fB3pl,tib\fR' and
+disable the bytes threshold, execute the following command:
+.RS 8
+.TP
+.BI " "
+.BI "nfacct add replace 'net 27' format 3pl,tib threshold - "
+.RE
+.TP
+.BI " "
+To \fIlist\fR all accounting objects in the accounting table, display
+all their properties and generate all output in \fBxml\fR format, execute
+the following command:
+.RS 8
+.TP
+.BI " "
+.BI "nfacct list show extended xml "
+.RE
+.TP
+.BI " "
+To \fIsave\fR and then \fIrestore\fR all existing accounting objects in
+the accounting table, \fBreplacing\fR their properties even if used by
+iptables, and \fBflush\fR the entire accounting table prior to restore,
+execute the following command:
+.RS 8
+.TP
+.BI " "
+.BI "nfacct save | nfacct restore flush replace "
+.RE
+.SH RESTORE FORMAT
+The entire restore format consists of \fBname=value\fR pairs, describing
+accounting object properties to be restored. With the exception of
+\fBname\fR, all properties are optional, however, most depend on either
+the value of mark or byte threshold indicators. Comments are allowed \- each
+line needs to start with the hash symbol (\fB#\fR) in first position. Blank
+lines are also ignored. Although the position in which each property is
+specified is not important, it could only be specified once - multiple
+definitions of the same accounting object properties are not allowed.
+.TP
+.BI " "
+.RE
+The execution of the restore file is atomic - each line is read,
+various accounting properties are verified, and then the accounting
+object is added or replaced immediately (the entire accounting table is
+flushed prior to that operation if \fIflush\fR command parameter is used).
+In the event of an error midway through the contents of the restore file,
+the executuion stops and all accounting objects restored up to that point are
+\fBnot\fR deleted.
+.TP
+.BI " "
+.RE
+The following properties are allowed in the restore file: 
+.TP
+.BI "name "
+Name of the accounting object. The value may be quoted, if space (' ') or
+other "special" characters appear in the account object name. This property
+is \fImandatory\fR.
+.TP
+.BI "fmt "
+Format of the accounting object as described in the \fBFORMAT OPTIONS\fR
+section above.
+.TP
+.BI "pkts "
+Packets received counter. If this property is specified, then \fBbytes\fR
+(see below) also needs to be included. This property is also mandatory in
+case marking is used (see \fBMARKING\fR section below).
+.TP
+.BI "bytes "
+Bytes received counter. If this property is specified, then \fBpkts\fR
+(see above) also needs to be included. This property is also mandatory in
+case marking is used (see \fBMARKING\fR section below) or if byte threshold
+is also enabled (see \fBthr\fR below).
+.TP
+.BI "pmark "
+Packets received counter since the last time a mark was set/enabled for this
+accounting object. If this property is specified, then \fBbmark\fR
+(see below), as well as \fBpkts\fR and \fBbytes\fR properties must also be
+included. The value of this property should not exceed the value of
+\fBpkts\fR.
+.TP
+.BI "bmark "
+Bytes received counter since the last time a mark was set/enabled for this
+accounting object. Mandatory if \fBpmark\fR is also specified. The value of
+this property should not exceed the value of \fBbytes\fR.
+.TP
+.BI "thr "
+Bytes threshold limit. If set, \fBbytes\fR also needs to specified.
+.TP
+Examples:
+.TP
+.BI " "
+The following line restores an accounting object 
+named '\fBALL dns\-internal\fR' using \fB3pl\fR format for packet counters and
+\fBiec\fR for byte counters, setting their values to \fB1451\fR and
+\fB249437\fR respectively and also enabling the byte threshold feature,
+setting it at \fB1MiB\fR:
+.RS 8
+.TP
+.BI " "
+.BI "name='ALL dns\-internal' fmt=3pl,iec pkts=1451 bytes=249437 thr=1048576"
+.RE
+.TP
+.BI " "
+The following line restores an accounting object named \fBweb\fR using
+\fB3pl\fR format for packet counters and \fBsi\fR for byte counters, setting
+their values to \fB1058481\fR and \fB1432810595\fR respectively and also
+enabling the byte threshold and marking features, setting byte threshold to
+\fB1.5GiB\fR, while enabling the packet and byte marks to start from
+\fB148965\fR and \fB136198528\fR respectively:
+.RS 8
+.TP
+.BI " "
+.BI "name=web fmt=3pl,si pkts=1058481 bytes=1432810595 pmark=148965 bmark=136198528 thr=1572864000"
+.RE
+.SH MARKING
+This tool has one additional feature for short-term traffic accounting,
+which allows a "mark" to be placed against an accounting object, enabling
+it to count traffic (both bytes and packets) since that mark has been
+enabled/set. This is in addition to the main packet and byte counters and
+enables for a short-term traffic to be analysed. Such marks could be placed
+as soon as accounting object is created and can be set, reset or deleted for
+each object independently.
+.TP
+.BI " "
+.RE
+The \fBlist\fR command allows for marks to be set/cleared for all registered
+accounting objects, while the \fBget\fR command allows this to be done for a
+specific accounting object. These two commands also allow for only the
+mark-related properties of a given accounting object (or objects) to be
+shown, using the \fBshow marks\fR parameter.
+.TP
+.BI " "
+.RE
+Once a mark has been set, traffic is counted as normal, though 3 additional
+properties are made available, in addition to the main packet and byte
+counters:
+.TP
+.BI "mark exceeded indicator "
+This property gives an indication whether \fIany\fR traffic has passed
+through the accounting object since a mark was last set. This is shown in
+the form of a plus sign (\fB+\fR) displayed next to the \fBpkts\fR counters.
+.TP
+.BI "packet mark count "
+This shows how many packets have passed through this accounting object since
+the mark was last set. The format of this property is the same as \fBpkts\fR.
+.TP
+.BI "bytes mark count "
+This shows how many bytes have passed through this accounting object since
+the mark was last set. The format of this property is the same as
+\fBbytes\fR.
+.TP
+.BI " "
+.RE
+Once placed, if a mark is then replaced, then the above properties start
+counting traffic from the last mark being set - that is, the "old" mark
+is deleted (and with it the mark packet and byte counters) and the new
+one is set, starting to count traffic from zero.
+.TP
+.BI " "
+.RE
+It is also worth noting that although mark packet and byte counters can
+be set or reset independently of the main counters, their value is dependent
+on them. In other words, if the main packet and byte counters are reset
+via the \fBlist reset\fR or \fBget reset\fR commands, or the account
+object is deleted, the mark associated with it is also reset or deleted.
 .PP
 .SH SEE ALSO
 .BR iptables (8)
 .SH BUGS
+.BI " "
+.PP
 Please, report them to netfilter-devel@vger.kernel.org or file a bug in
 Netfilter's bugzilla (https://bugzilla.netfilter.org).
 .SH AUTHORS
 Pablo Neira Ayuso wrote and maintains the nfacct tool.
 .PP
-Man page written by Pablo Neira Ayuso <pablo@netfilter.org>.
+Man page written by Pablo Neira Ayuso <pablo@netfilter.org> and later
+ammended by Michael Zintakis <michael.zintakis@googlemail.com>.
-- 
1.8.3.1

Re: [0/29] nfacct changes and additions

[Pablo Neira Ayuso]

Hi,

I'm going to review several of your patches from this email to avoid
spamming the mailing list. I will reply some of them from the original
mail as my comments need some context.

This big batch contains patches for many different things, which is
not a good practise. Please, split patchsets into logical pieces, and
send them progressively.

On Wed, Jul 10, 2013 at 07:24:58PM +0100, Michael Zintakis wrote:
> The following patch set fixes bugs and introduces a variety of new features
> to the 3 nfacct components: kernel, libnetfilter_acct and nfacct executable.
> 
> All of the patches need to be applied in the order specified in this patch
> set (1-29) as they are interdependent. The full list of bugfixes and
> features added for each component are:

[kernel 1/29] bugfix: pkts/bytes need to be specified simultaneously

As Florian mentioned, the kernel validates that we don't crash on
incorrect configurations. We don't care if what you load does not
makes sense from the user perspective.

[kernel 2/29] bugfix: restore pkts/bytes counters in NLM_F_REPLACE

We already discussed this. You can atomically dump and reset counters
via the GET command with the NLM_F_DUMP flag set. Restoring counters
is allowed, but for unused rules. In case the system just started up,
you have to make sure that counters are restored via nfacct before
they are used by the rules.

[libnetfilter_acct 3/29] bugfix: correct xml name parsing
[libnetfilter_acct 4/29] bugfix: correct (plain) name parsing

Will reply to this in a separated email.

[nfacct 5/29] bugfix: prevent 0-sized parameter being accepted

I don't see what situation you're trying to catch with this patch.

[nfacct 6/29] bugfix: prevent 0-sized nfacct name being accepted

We already have a patch to validate this from the kernel, it returns
-EINVAL.

[nfacct 7/29] code-refactoring changes to the "command menu"

Will reply in separated email.

[nfacct 8/29] add 2 new options: "replace" and "flush"
[libnetfilter_acct 9/29] add *_SAVE template allowing save/restore

For 8/29 and 9/29, same reply as for 2/29.

[libnetfilter_acct 10/29] add *_BONLY template to show bytes-only
[libnetfilter_acct 11/29] add variable width and on-the-fly formatting
[nfacct 12/29] add variable width and on-the-fly number formatting
[nfacct 13/29] add new "save" and correct existing "restore" commands
[nfacct 14/29] add sort option to the "list" command
[nfacct 15/29] add "show bytes" option to "list" and "get" commands

>From 10/29 to 15/29, I will reply to this in separated emails as I
need the source code context for my comments.

[kernel 16/29] add permanent byte/packet format capability to nfacct

>From 16/29 to 29/29 this requires kernel changes that cannot get
into mainstream, the arguments you provided to get this mainstream are
bogus:

* From the memory side, this is adding new fields that are not used in
  the packet path, they only have a meaning from user-space.

* You mention that it's more secure to put this in the kernel. That's
  wrong since that provides a false feeling of security: Anyone could
  write a netlink socket to mangle those counters with your replace
  feature.

[libnetfilter_acct 17/29] add *permanent* number formatting support
[nfacct 18/29] add permanent number formatting to nfacct objects
[kernel 19/29] add byte threshold capability to nfacct
[libnetfilter_acct 20/29] add byte threshold capability support
[nfacct 21/29] add byte threshold capabilities to nfacct objects
[libnetfilter_acct 22/29] add *_EXTENDED template support
[nfacct 23/29] add "show extended" option to "list" and "get" commands
[kernel 24/29] add packets and bytes mark capability to nfacct
[libnetfilter_acct 25/29] add packets/bytes mark capability support
[nfacct 26/29] add setmark and clrmark to "get" and "list" commands
[libnetfilter_acct 27/29] add *_MONLY template support
[nfacct 28/29] add "show marks" option to "list" and "get" commands
[nfacct 29/29] change man page to describe all new features

Regards.