Netfilter: New target: RLOG

Netfilter: New target: RLOG

[Richard Weinberger]

RLOG is a new log target, it works like LOG with the exception that it writes to ring buffers.
It makes use of Steven Rostedt's ring_buffer subsystem. 
I've used Steve's ring buffer because it allows concurrent writes. IOW it's very fast.
For more details see: Documentation/trace/ring-buffer-design.txt.

Each ring buffer is represented as a pipe-like file in /proc/net/netfilter/xt_RLOG/.
You can read from it with and program you like (cat, syslog, etc...).
The default size is 1MiB. With this size it can store approximately 5000 messages.

- Why not LOG?
I like the LOG target a lot but I really hat it when it floods my kernel syslog.
dmesg becomes useless.
Writing all log messages to a file using syslogd also not always the best solution.
Most of the time my firewall logs just waste disk space.

Compared with Steve's ring_buffer, the kernel syslog is rather slow.
Especially when the firewall logs very much syslog becomes a bottleneck.
As we all know printk() is not fast.

- Why not ULOG/NFLOG?
Because it cannot replace LOG.
Details like PHYSIN and PHYSOUT are not available form the packet headers.
Also on many Linux systems ulogd is not available/supported.  

- Why RLOG?
Using RLOG you can have many ring buffers with all kind of logs.
If your firewall goes nuts you don't have to mess you rule-set with adding
new LOG rules to find out what's going on.
Just install a few RLOG rules with small buffer sized and read them if you don't
know what's going on.
If you make you firewall rule-set per default verbose using LOG or NFLOG it will 
generate lot's of useless messages which you'll never ever read.
With RLOG you can bypass this problem.
On my firewall I record only useful data to the disk. Everything else goes into RLOG.

If your firewall is really busy and you want to log nearly everything, c
reate a big ring buffer and read from is using your favorite userspace tool.
In case the buffer fills faster than the userspace consumes it, RLOG will warn you.
I'd also possible to resize the buffer.

- Example

You can use RLOG like LOG.
E.g:
iptables -A INPUT -p icmp -j RLOG
The default name of the ring buffer is "default".
Now you can read your icmp logs from /proc/net/netfilter/default

iptables -A INPUT -p icmp -j RLOG --ring foo
iptables -A INPUT -j RLOG --ring bar --size 4096

This will create two new ring buffers "foo" and "bar".
"bar" has a size of 4MiB.

ICMP logs go now into three ring buffers, "default" and "foo".
Everything else (including ICMP) goes into "bar".
If you don't want to use syslog add "--add-timestamp" to have timestamps in your logs.

Different iptables rules with different RLOG flags can use the same ring buffer.
iptables -A INPUT -p icmp -j RLOG --ring xxx --add-timestamp
iptables -A INPUT -p icmp -j RLOG --ring xxx

Every ICMP packet will show up in "xxx" twice. But the first entry will have a timestamp.


[PATCH 1/5] ring_buffer: Export for_each_buffer_cpu()
Steve, is this path ok for you?

[PATCH 2/5] xt_log: Make printk() in sb_close() optional
[PATCH 3/5] nf_log: Export dump_packet() and dump_mac_header()
RLOG reuses as much code from LOG are possible.
These two patches are needed to export some code to RLOG.
Currently RLOG depends on both ipt_LOG and ip6t_LOG.

[PATCH 4/5] netfilter: Implement xt_RLOG
[PATCH 5/5] xt_RLOG: add userspace-components

Any feedback is welcome!

[PATCH 1/5] ring_buffer: Export for_each_buffer_cpu()

[Richard Weinberger]

From: Richard Weinberger <richard@nod.at>


Signed-off-by: Richard Weinberger <richard@nod.at>
---
 include/linux/ring_buffer.h |    5 +++++
 kernel/trace/ring_buffer.c  |    7 +++++++
 2 files changed, 12 insertions(+), 0 deletions(-)

diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h
index 67be037..5c1c29f 100644
--- a/include/linux/ring_buffer.h
+++ b/include/linux/ring_buffer.h
@@ -185,4 +185,9 @@ enum ring_buffer_flags {
 	RB_FL_OVERWRITE		= 1 << 0,
 };
 
+cpumask_var_t *ring_buffer_mask(struct ring_buffer *buffer);
+
+#define for_each_buffer_cpu(buffer, cpu) \
+	for_each_cpu(cpu, *(ring_buffer_mask(buffer)))
+
 #endif /* _LINUX_RING_BUFFER_H */
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index f5b7b5c..a4117c5 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -348,6 +348,7 @@ void *ring_buffer_event_data(struct ring_buffer_event *event)
 }
 EXPORT_SYMBOL_GPL(ring_buffer_event_data);
 
+#undef for_each_buffer_cpu
 #define for_each_buffer_cpu(buffer, cpu)		\
 	for_each_cpu(cpu, buffer->cpumask)
 
@@ -4139,3 +4140,9 @@ static int rb_cpu_notify(struct notifier_block *self,
 	return NOTIFY_OK;
 }
 #endif
+
+cpumask_var_t *ring_buffer_mask(struct ring_buffer *buffer)
+{
+	return &buffer->cpumask;
+}
+EXPORT_SYMBOL_GPL(ring_buffer_mask);
-- 
1.7.7

[PATCH 2/5] xt_log: Make printk() in sb_close() optional

[Richard Weinberger]

From: Richard Weinberger <richard@nod.at>


Signed-off-by: Richard Weinberger <richard@nod.at>
---
 include/net/netfilter/xt_log.h |    9 ++++++---
 1 files changed, 6 insertions(+), 3 deletions(-)

diff --git a/include/net/netfilter/xt_log.h b/include/net/netfilter/xt_log.h
index 0dfb34a..767e08d 100644
--- a/include/net/netfilter/xt_log.h
+++ b/include/net/netfilter/xt_log.h
@@ -39,10 +39,12 @@ static struct sbuff *sb_open(void)
 	return m;
 }
 
-static void sb_close(struct sbuff *m)
+static void __sb_close(struct sbuff *m, int print)
 {
-	m->buf[m->count] = 0;
-	printk("%s\n", m->buf);
+	if (print) {
+		m->buf[m->count] = 0;
+		printk("%s\n", m->buf);
+	}
 
 	if (likely(m != &emergency))
 		kfree(m);
@@ -52,3 +54,4 @@ static void sb_close(struct sbuff *m)
 	}
 }
 
+#define sb_close(m)	__sb_close(m, 1)
-- 
1.7.7

[PATCH 3/5] nf_log: Export dump_packet() and dump_mac_header() functions

[Richard Weinberger]

From: Richard Weinberger <richard@nod.at>


Signed-off-by: Richard Weinberger <richard@nod.at>
---
 include/net/netfilter/nf_log.h |   19 +++++++++++++++++++
 net/ipv4/netfilter/ipt_LOG.c   |   12 +++++++-----
 net/ipv6/netfilter/ip6t_LOG.c  |   12 +++++++-----
 3 files changed, 33 insertions(+), 10 deletions(-)

diff --git a/include/net/netfilter/nf_log.h b/include/net/netfilter/nf_log.h
index e991bd0..c0f2b27 100644
--- a/include/net/netfilter/nf_log.h
+++ b/include/net/netfilter/nf_log.h
@@ -61,5 +61,24 @@ void nf_log_packet(u_int8_t pf,
 		   const struct net_device *out,
 		   const struct nf_loginfo *li,
 		   const char *fmt, ...);
+struct sbuff;
+void ipt_dump_packet(struct sbuff *m,
+	const struct nf_loginfo *info,
+	const struct sk_buff *skb,
+	unsigned int iphoff);
+
+void ipt_dump_mac_header(struct sbuff *m,
+	const struct nf_loginfo *info,
+	const struct sk_buff *skb);
+
+void ip6t_dump_packet(struct sbuff *m,
+	const struct nf_loginfo *info,
+	const struct sk_buff *skb,
+	unsigned int iphoff,
+	int recurse);
+
+void ip6t_dump_mac_header(struct sbuff *m,
+	const struct nf_loginfo *info,
+	const struct sk_buff *skb);
 
 #endif /* _NF_LOG_H */
diff --git a/net/ipv4/netfilter/ipt_LOG.c b/net/ipv4/netfilter/ipt_LOG.c
index d76d6c9..8ec9d2b 100644
--- a/net/ipv4/netfilter/ipt_LOG.c
+++ b/net/ipv4/netfilter/ipt_LOG.c
@@ -31,7 +31,7 @@ MODULE_AUTHOR("Netfilter Core Team <coreteam@netfilter.org>");
 MODULE_DESCRIPTION("Xtables: IPv4 packet logging to syslog");
 
 /* One level of recursion won't kill us */
-static void dump_packet(struct sbuff *m,
+void ipt_dump_packet(struct sbuff *m,
 			const struct nf_loginfo *info,
 			const struct sk_buff *skb,
 			unsigned int iphoff)
@@ -270,7 +270,7 @@ static void dump_packet(struct sbuff *m,
 			/* Max length: 3+maxlen */
 			if (!iphoff) { /* Only recurse once. */
 				sb_add(m, "[");
-				dump_packet(m, info, skb,
+				ipt_dump_packet(m, info, skb,
 					    iphoff + ih->ihl*4+sizeof(_icmph));
 				sb_add(m, "] ");
 			}
@@ -362,8 +362,9 @@ static void dump_packet(struct sbuff *m,
 	/* maxlen =  IP + ICMP +  IP + max(TCP,UDP,ICMP,unknown) */
 	/* maxlen = 230+   91  + 230 + 252 = 803 */
 }
+EXPORT_SYMBOL_GPL(ipt_dump_packet);
 
-static void dump_mac_header(struct sbuff *m,
+void ipt_dump_mac_header(struct sbuff *m,
 			    const struct nf_loginfo *info,
 			    const struct sk_buff *skb)
 {
@@ -399,6 +400,7 @@ fallback:
 	}
 	sb_add(m, " ");
 }
+EXPORT_SYMBOL_GPL(ipt_dump_mac_header);
 
 static struct nf_loginfo default_loginfo = {
 	.type	= NF_LOG_TYPE_LOG,
@@ -443,9 +445,9 @@ ipt_log_packet(u_int8_t pf,
 #endif
 
 	if (in != NULL)
-		dump_mac_header(m, loginfo, skb);
+		ipt_dump_mac_header(m, loginfo, skb);
 
-	dump_packet(m, loginfo, skb, 0);
+	ipt_dump_packet(m, loginfo, skb, 0);
 
 	sb_close(m);
 }
diff --git a/net/ipv6/netfilter/ip6t_LOG.c b/net/ipv6/netfilter/ip6t_LOG.c
index e6af8d7..c6abacb 100644
--- a/net/ipv6/netfilter/ip6t_LOG.c
+++ b/net/ipv6/netfilter/ip6t_LOG.c
@@ -34,7 +34,7 @@ struct in_device;
 #include <linux/netfilter_ipv6/ip6t_LOG.h>
 
 /* One level of recursion won't kill us */
-static void dump_packet(struct sbuff *m,
+void ip6t_dump_packet(struct sbuff *m,
 			const struct nf_loginfo *info,
 			const struct sk_buff *skb, unsigned int ip6hoff,
 			int recurse)
@@ -341,7 +341,7 @@ static void dump_packet(struct sbuff *m,
 			/* Max length: 3+maxlen */
 			if (recurse) {
 				sb_add(m, "[");
-				dump_packet(m, info, skb,
+				ip6t_dump_packet(m, info, skb,
 					    ptr + sizeof(_icmp6h), 0);
 				sb_add(m, "] ");
 			}
@@ -371,8 +371,9 @@ static void dump_packet(struct sbuff *m,
 	if (!recurse && skb->mark)
 		sb_add(m, "MARK=0x%x ", skb->mark);
 }
+EXPORT_SYMBOL(ip6t_dump_packet);
 
-static void dump_mac_header(struct sbuff *m,
+void ip6t_dump_mac_header(struct sbuff *m,
 			    const struct nf_loginfo *info,
 			    const struct sk_buff *skb)
 {
@@ -422,6 +423,7 @@ fallback:
 	} else
 		sb_add(m, " ");
 }
+EXPORT_SYMBOL(ip6t_dump_mac_header);
 
 static struct nf_loginfo default_loginfo = {
 	.type	= NF_LOG_TYPE_LOG,
@@ -453,9 +455,9 @@ ip6t_log_packet(u_int8_t pf,
 	       out ? out->name : "");
 
 	if (in != NULL)
-		dump_mac_header(m, loginfo, skb);
+		ip6t_dump_mac_header(m, loginfo, skb);
 
-	dump_packet(m, loginfo, skb, skb_network_offset(skb), 1);
+	ip6t_dump_packet(m, loginfo, skb, skb_network_offset(skb), 1);
 
 	sb_close(m);
 }
-- 
1.7.7

[PATCH 4/5] netfilter: Implement xt_RLOG

[Richard Weinberger]

From: Richard Weinberger <richard@nod.at>

xt_RLOG is a new logging target, it produces log messages like LOG
but it does not write them into the kernel syslog.
All messages are written into ring buffers.
The user can create arbitrary many ring buffer of any size.
Each ring buffer is represented as file in /proc/net/netfilter/xt_RLOG/

Signed-off-by: Richard Weinberger <richard@nod.at>
---
 net/netfilter/Kconfig   |   17 ++
 net/netfilter/Makefile  |    1 +
 net/netfilter/xt_RLOG.c |  622 +++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 640 insertions(+), 0 deletions(-)
 create mode 100644 net/netfilter/xt_RLOG.c

diff --git a/net/netfilter/Kconfig b/net/netfilter/Kconfig
index f8ac4ef..5810ed3 100644
--- a/net/netfilter/Kconfig
+++ b/net/netfilter/Kconfig
@@ -578,6 +578,23 @@ config NETFILTER_XT_TARGET_RATEEST
 
 	  To compile it as a module, choose M here.  If unsure, say N.
 
+config NETFILTER_XT_TARGET_RLOG
+	tristate '"RLOG" target support (EXPERIMENTAL)'
+	depends on EXPERIMENTAL
+	depends on NETFILTER_ADVANCED
+	depends on RING_BUFFER
+	select IP_NF_TARGET_LOG
+	select IP6_NF_TARGET_LOG if IP6_NF_IPTABLES
+	help
+	  This target records packet headers into one or more ring buffers.
+	  Each ring buffer is represented as a file in
+	  /proc/net/netfilter/xt_RLOG/.
+	  It produces the same output as LOG.
+	  Select this target if you like LOG but don't want your kernel syslog
+	  flooded.
+
+	  To compile it as a module, choose M here.  If unsure, say N.
+
 config NETFILTER_XT_TARGET_TEE
 	tristate '"TEE" - packet cloning to alternate destination'
 	depends on NETFILTER_ADVANCED
diff --git a/net/netfilter/Makefile b/net/netfilter/Makefile
index 40f4c3d..447b58d 100644
--- a/net/netfilter/Makefile
+++ b/net/netfilter/Makefile
@@ -62,6 +62,7 @@ obj-$(CONFIG_NETFILTER_XT_TARGET_NFLOG) += xt_NFLOG.o
 obj-$(CONFIG_NETFILTER_XT_TARGET_NFQUEUE) += xt_NFQUEUE.o
 obj-$(CONFIG_NETFILTER_XT_TARGET_NOTRACK) += xt_NOTRACK.o
 obj-$(CONFIG_NETFILTER_XT_TARGET_RATEEST) += xt_RATEEST.o
+obj-$(CONFIG_NETFILTER_XT_TARGET_RLOG) += xt_RLOG.o
 obj-$(CONFIG_NETFILTER_XT_TARGET_SECMARK) += xt_SECMARK.o
 obj-$(CONFIG_NETFILTER_XT_TARGET_TPROXY) += xt_TPROXY.o
 obj-$(CONFIG_NETFILTER_XT_TARGET_TCPMSS) += xt_TCPMSS.o
diff --git a/net/netfilter/xt_RLOG.c b/net/netfilter/xt_RLOG.c
new file mode 100644
index 0000000..b2b53d9
--- /dev/null
+++ b/net/netfilter/xt_RLOG.c
@@ -0,0 +1,622 @@
+/*
+ * Copyright (c) 2012 Richard Weinberger <richard@nod.at>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <linux/types.h>
+#include <linux/kernel.h>
+#include <linux/module.h>
+#include <linux/ring_buffer.h>
+#include <linux/cpu.h>
+#include <linux/slab.h>
+#include <linux/uaccess.h>
+#include <linux/fs.h>
+#include <linux/poll.h>
+#include <linux/proc_fs.h>
+#include <linux/wait.h>
+#include <linux/time.h>
+#include <linux/sched.h>
+#include <linux/list.h>
+#include <linux/atomic.h>
+#include <linux/spinlock.h>
+#include <linux/mutex.h>
+#include <linux/netfilter.h>
+#include <linux/netfilter/x_tables.h>
+
+#include <net/netfilter/xt_log.h>
+#include <net/netfilter/nf_log.h>
+
+#define TARGETNAME_LEN		32
+#define DEFAULT_RING_SIZE	1024 /* in KiB */
+
+#if defined(CONFIG_IPV6) || defined(CONFIG_IPV6_MODULE)
+#	define WITH_IPV6 1
+#endif
+
+struct rlog_target {
+	char name[TARGETNAME_LEN];
+	struct ring_buffer *buffer;
+	atomic_t pipe_in_use;
+	atomic_t refcnt;
+
+	struct list_head list;
+};
+
+struct rlog_entry {
+	size_t count;
+	char msg[0];
+};
+
+struct rlog_iter {
+	struct ring_buffer *buffer;
+	const char *buffer_name;
+	struct rlog_entry *ent;
+
+	char print_buf[PAGE_SIZE];
+	size_t print_buf_len;
+	size_t print_buf_pos;
+
+	unsigned long lost_events;
+	int cpu;
+
+	struct mutex lock;
+};
+
+struct rlog_tg_info {
+	char name[TARGETNAME_LEN];
+	unsigned int size;
+	unsigned char logflags;
+	bool add_timestamp;
+	struct rlog_target *log_target;
+};
+
+static DEFINE_SPINLOCK(target_list_lock);
+static LIST_HEAD(target_list);
+static DECLARE_WAIT_QUEUE_HEAD(rlog_wait);
+static struct proc_dir_entry *prlog;
+
+static struct nf_loginfo default_loginfo = {
+	.type   = NF_LOG_TYPE_LOG,
+	.u = {
+		.log = {
+			.logflags = NF_LOG_MASK,
+		},
+	},
+};
+
+static void wakeup_work_handler(struct work_struct *work)
+{
+	wake_up(&rlog_wait);
+}
+
+static DECLARE_WORK(wakeup_work, wakeup_work_handler);
+
+static void rlog_wake_up(void)
+{
+	schedule_work(&wakeup_work);
+}
+
+static int add_record(struct ring_buffer *buffer, struct sbuff *m)
+{
+	struct rlog_entry *entry;
+	struct ring_buffer_event *event;
+
+	event = ring_buffer_lock_reserve(buffer, sizeof(*entry) + m->count);
+	if (!event)
+		return 1;
+
+	entry = ring_buffer_event_data(event);
+	memcpy(entry->msg, m->buf, m->count);
+	entry->count = m->count;
+
+	ring_buffer_unlock_commit(buffer, event);
+	rlog_wake_up();
+
+	return 0;
+}
+
+static struct rlog_entry *peek_next_entry(struct rlog_iter *iter, int cpu,
+					  unsigned long long *ts)
+{
+	struct ring_buffer_event *event;
+
+	event = ring_buffer_peek(iter->buffer, cpu, ts, &iter->lost_events);
+
+	if (event)
+		return ring_buffer_event_data(event);
+
+	return NULL;
+}
+
+static struct rlog_entry *find_next_entry(struct rlog_iter *iter)
+{
+	struct rlog_entry *ent, *next = NULL;
+	unsigned long long next_ts = 0, ts;
+	int cpu, next_cpu = -1;
+
+	for_each_buffer_cpu (iter->buffer, cpu) {
+		if (ring_buffer_empty_cpu(iter->buffer, cpu))
+			continue;
+
+		ent = peek_next_entry(iter, cpu, &ts);
+
+		if (ent && (!next || ts < next_ts)) {
+			next = ent;
+			next_cpu = cpu;
+			next_ts = ts;
+		}
+	}
+
+	iter->cpu = next_cpu;
+
+	return next;
+}
+
+static struct rlog_iter *find_next_entry_inc(struct rlog_iter *iter)
+{
+	iter->ent = find_next_entry(iter);
+
+	if (iter->ent)
+		return iter;
+
+	return NULL;
+}
+
+static int buffer_empty(struct rlog_iter *iter)
+{
+	int cpu;
+
+	for_each_buffer_cpu (iter->buffer, cpu) {
+		if (!ring_buffer_empty_cpu(iter->buffer, cpu))
+			return 0;
+	}
+
+	return 1;
+}
+
+static ssize_t rlog_to_user(struct rlog_iter *iter, char __user *ubuf,
+			    size_t cnt)
+{
+	int ret;
+	int len;
+
+	if (!cnt)
+		goto out;
+
+	len = iter->print_buf_len - iter->print_buf_pos;
+	if (len < 1)
+		return -EBUSY;
+
+	if (cnt > len)
+		cnt = len;
+
+	ret = copy_to_user(ubuf, iter->print_buf + iter->print_buf_pos, cnt);
+	if (ret == cnt)
+		return -EFAULT;
+
+	cnt -= ret;
+	iter->print_buf_pos += cnt;
+
+out:
+	return cnt;
+}
+
+static int rlog_open_pipe(struct inode *inode, struct file *file)
+{
+	struct rlog_iter *iter;
+	struct rlog_target *tgt = PDE(inode)->data;
+	int ret = 0;
+
+	/* only one consuming reader is allowed */
+	if (atomic_cmpxchg(&tgt->pipe_in_use, 0, 1)) {
+		ret = -EBUSY;
+		goto out;
+	}
+
+	iter = kzalloc(sizeof(*iter), GFP_KERNEL);
+	if (!iter) {
+		ret = -ENOMEM;
+		goto out;
+	}
+
+	mutex_init(&iter->lock);
+	iter->buffer = tgt->buffer;
+	iter->buffer_name = tgt->name;
+
+	file->private_data = iter;
+out:
+	return ret;
+}
+
+static unsigned int rlog_poll_pipe(struct file *file, poll_table *poll_table)
+{
+	struct rlog_iter *iter = file->private_data;
+
+	if (!buffer_empty(iter))
+		return POLLIN | POLLRDNORM;
+
+	poll_wait(file, &rlog_wait, poll_table);
+
+	if (!buffer_empty(iter))
+		return POLLIN | POLLRDNORM;
+
+	return 0;
+}
+
+static int rlog_release_pipe(struct inode *inode, struct file *file)
+{
+	struct rlog_iter *iter = file->private_data;
+	struct rlog_target *tgt = PDE(inode)->data;
+
+	mutex_destroy(&iter->lock);
+	kfree(iter);
+	atomic_set(&tgt->pipe_in_use, 0);
+
+	return 0;
+}
+
+static void wait_pipe(struct rlog_iter *iter)
+{
+	DEFINE_WAIT(wait);
+
+	prepare_to_wait(&rlog_wait, &wait, TASK_INTERRUPTIBLE);
+
+	if (buffer_empty(iter))
+		schedule();
+
+	finish_wait(&rlog_wait, &wait);
+}
+
+static int rlog_wait_pipe(struct file *file)
+{
+	struct rlog_iter *iter = file->private_data;
+
+	while (buffer_empty(iter)) {
+		if (file->f_flags & O_NONBLOCK)
+			return -EAGAIN;
+
+		mutex_unlock(&iter->lock);
+
+		wait_pipe(iter);
+
+		mutex_lock(&iter->lock);
+
+		if (signal_pending(current))
+			return -EINTR;
+	}
+
+	return 1;
+}
+
+static ssize_t rlog_read_pipe(struct file *file, char __user *ubuf,
+			      size_t cnt, loff_t *ppos)
+{
+	struct rlog_iter *iter = file->private_data;
+	ssize_t ret;
+
+	ret = rlog_to_user(iter, ubuf, cnt);
+	if (ret != -EBUSY)
+		goto out;
+
+	iter->print_buf_pos = 0;
+	iter->print_buf_len = 0;
+
+	if (cnt >= PAGE_SIZE)
+		cnt = PAGE_SIZE - 1;
+
+	mutex_lock(&iter->lock);
+again:
+	ret = rlog_wait_pipe(file);
+	if (ret <= 0)
+		goto out_unlock;
+
+	while (find_next_entry_inc(iter) != NULL) {
+		struct rlog_entry *ent;
+		ent = iter->ent;
+
+		if (ent->count >= PAGE_SIZE - iter->print_buf_len)
+			break;
+
+		memcpy(iter->print_buf + iter->print_buf_len, ent->msg,
+			ent->count);
+		iter->print_buf_len += ent->count;
+
+		ring_buffer_consume(iter->buffer, iter->cpu, NULL,
+			&iter->lost_events);
+		if (iter->lost_events)
+			printk(KERN_WARNING KBUILD_MODNAME ": Ring %s "
+				"lost %lu events\n", iter->buffer_name,
+				iter->lost_events);
+
+		if (iter->print_buf_len >= cnt)
+			break;
+	}
+
+	ret = rlog_to_user(iter, ubuf, cnt);
+
+	if (iter->print_buf_pos >= iter->print_buf_len) {
+		iter->print_buf_pos = 0;
+		iter->print_buf_len = 0;
+	}
+
+	if (ret == -EBUSY)
+		goto again;
+out_unlock:
+	mutex_unlock(&iter->lock);
+out:
+	return ret;
+}
+
+static const struct file_operations rlog_pipe_fops = {
+	.open		= rlog_open_pipe,
+	.poll		= rlog_poll_pipe,
+	.read		= rlog_read_pipe,
+	.release	= rlog_release_pipe,
+	.llseek		= no_llseek,
+};
+
+static struct rlog_target *new_rlog_target(const char *name, size_t rb_size)
+{
+	struct rlog_target *new;
+
+	new = kmalloc(sizeof(*new), GFP_KERNEL);
+	if (!new) {
+		new = ERR_PTR(-ENOMEM);
+
+		goto out;
+	}
+
+	new->buffer = ring_buffer_alloc(rb_size, RB_FL_OVERWRITE);
+	if (!new->buffer) {
+		kfree(new);
+		new = ERR_PTR(-ENOMEM);
+
+		goto out;
+	}
+
+	strlcpy(new->name, name, TARGETNAME_LEN);
+
+	if (!proc_create_data(name, 0400, prlog, &rlog_pipe_fops, new)) {
+		ring_buffer_free(new->buffer);
+		kfree(new);
+		new = ERR_PTR(-ENOMEM);
+
+		goto out;
+	}
+
+	atomic_set(&new->pipe_in_use, 0);
+	atomic_set(&new->refcnt, 0);
+
+	spin_lock(&target_list_lock);
+	list_add(&new->list, &target_list);
+	spin_unlock(&target_list_lock);
+out:
+	return new;
+}
+
+static void free_rlog_target(struct rlog_target *target)
+{
+	remove_proc_entry(target->name, prlog);
+	ring_buffer_free(target->buffer);
+	list_del(&target->list);
+	kfree(target);
+}
+
+static struct rlog_target *find_taget(const char *name)
+{
+	struct list_head *e;
+	struct rlog_target *tmp, *victim = NULL;
+
+	spin_lock(&target_list_lock);
+
+	list_for_each(e, &target_list) {
+		tmp = list_entry(e, struct rlog_target, list);
+		if (strcmp(tmp->name, name) == 0) {
+			victim = tmp;
+
+			goto out;
+		}
+	}
+
+out:
+	spin_unlock(&target_list_lock);
+
+	return victim;
+}
+
+static void rlog_log_common(struct sbuff *m,
+			    u_int8_t pf,
+			    unsigned int hooknum,
+			    const struct sk_buff *skb,
+			    const struct net_device *in,
+			    const struct net_device *out,
+			    bool add_timestamp)
+{
+	if (add_timestamp) {
+		static struct timespec ts;
+		getnstimeofday(&ts);
+		sb_add(m, "[%li.%li] ", ts.tv_sec, ts.tv_nsec);
+	}
+
+	sb_add(m, "IN=%s OUT=%s ", in ? in->name : "", out ? out->name : "");
+
+#ifdef CONFIG_BRIDGE_NETFILTER
+	if (skb->nf_bridge) {
+		const struct net_device *physindev;
+		const struct net_device *physoutdev;
+
+		physindev = skb->nf_bridge->physindev;
+		if (physindev && in != physindev)
+			sb_add(m, "PHYSIN=%s ", physindev->name);
+		physoutdev = skb->nf_bridge->physoutdev;
+		if (physoutdev && out != physoutdev)
+			sb_add(m, "PHYSOUT=%s ", physoutdev->name);
+	}
+#endif
+}
+
+static struct sbuff *rlog_log_ip_packet(u_int8_t pf,
+			    unsigned int hooknum,
+			    const struct sk_buff *skb,
+			    const struct net_device *in,
+			    const struct net_device *out,
+			    const struct rlog_tg_info *info)
+{
+	struct nf_loginfo loginfo = default_loginfo;
+	struct sbuff *m = sb_open();
+
+	loginfo.u.log.logflags = info->logflags;
+	rlog_log_common(m, pf, hooknum, skb, in, out, info->add_timestamp);
+
+	if (in != NULL)
+		ipt_dump_mac_header(m, &loginfo, skb);
+
+	ipt_dump_packet(m, &loginfo, skb, 0);
+
+	return m;
+}
+
+#ifdef WITH_IPV6
+static struct sbuff *rlog_log_ip6_packet(u_int8_t pf,
+			    unsigned int hooknum,
+			    const struct sk_buff *skb,
+			    const struct net_device *in,
+			    const struct net_device *out,
+			    const struct rlog_tg_info *info)
+{
+	struct nf_loginfo loginfo = default_loginfo;
+	struct sbuff *m = sb_open();
+
+	loginfo.u.log.logflags = info->logflags;
+	rlog_log_common(m, pf, hooknum, skb, in, out, info->add_timestamp);
+
+	if (in != NULL)
+		ip6t_dump_mac_header(m, &default_loginfo, skb);
+
+	ip6t_dump_packet(m, &default_loginfo, skb, skb_network_offset(skb), 1);
+
+	return m;
+}
+#endif
+
+static unsigned int
+rlog_tg(struct sk_buff *skb, const struct xt_action_param *par)
+{
+	const struct rlog_tg_info *info = par->targinfo;
+	struct rlog_target *t = info->log_target;
+
+	struct sbuff *m;
+
+	if (par->family == NFPROTO_IPV4)
+		m = rlog_log_ip_packet(NFPROTO_IPV4, par->hooknum, skb,
+			par->in, par->out, info);
+#ifdef WITH_IPV6
+	else if (par->family == NFPROTO_IPV6)
+		m = rlog_log_ip6_packet(NFPROTO_IPV6, par->hooknum, skb,
+			par->in, par->out, info);
+#endif
+	else
+		BUG();
+
+	sb_add(m, "\n");
+	add_record(t->buffer, m);
+
+	__sb_close(m, 0);
+
+	return XT_CONTINUE;
+}
+
+static int rlog_tg_check(const struct xt_tgchk_param *par)
+{
+	struct rlog_tg_info *info = par->targinfo;
+	struct rlog_target *t;
+	int ret = 0;
+
+	if (info->name[sizeof(info->name) - 1] != '\0') {
+		ret = -EINVAL;
+
+		goto out;
+	}
+
+	t = find_taget(info->name);
+	if (!t) {
+		if (!info->size)
+			info->size = DEFAULT_RING_SIZE;
+
+		t = new_rlog_target(info->name, info->size << 10);
+		if (IS_ERR(t)) {
+			ret = PTR_ERR(t);
+
+			goto out;
+		}
+	}
+
+	atomic_inc(&t->refcnt);
+	info->log_target = t;
+out:
+	return ret;
+}
+
+static void rlog_tg_destroy(const struct xt_tgdtor_param *par)
+{
+	struct rlog_tg_info *info = par->targinfo;
+	struct rlog_target *t = info->log_target;
+
+	if (atomic_dec_and_test(&t->refcnt))
+		free_rlog_target(t);
+}
+
+static struct xt_target rlog_tg_regs[] __read_mostly = {
+	{
+		.name		= "RLOG",
+		.family		= NFPROTO_IPV4,
+		.target		= rlog_tg,
+		.targetsize	= sizeof(struct rlog_tg_info),
+		.checkentry	= rlog_tg_check,
+		.destroy	= rlog_tg_destroy,
+		.me		= THIS_MODULE,
+	},
+#ifdef WITH_IPV6
+	{
+		.name		= "RLOG",
+		.family		= NFPROTO_IPV6,
+		.target		= rlog_tg,
+		.targetsize	= sizeof(struct rlog_tg_info),
+		.checkentry	= rlog_tg_check,
+		.destroy	= rlog_tg_destroy,
+		.me		= THIS_MODULE,
+	}
+#endif
+};
+
+static void __exit rlog_exit(void)
+{
+	BUG_ON(!list_empty(&target_list));
+
+	xt_unregister_targets(rlog_tg_regs, ARRAY_SIZE(rlog_tg_regs));
+	remove_proc_entry(KBUILD_MODNAME, proc_net_netfilter);
+}
+module_exit(rlog_exit);
+
+static int __init rlog_init(void)
+{
+	prlog = proc_mkdir(KBUILD_MODNAME, proc_net_netfilter);
+	if (!prlog)
+		return -ENOMEM;
+
+	if (xt_register_targets(rlog_tg_regs, ARRAY_SIZE(rlog_tg_regs)) < 0) {
+		remove_proc_entry(KBUILD_MODNAME, proc_net_netfilter);
+
+		return -EINVAL;
+	}
+
+	return 0;
+}
+module_init(rlog_init);
+
+MODULE_AUTHOR("Richard Weinberger");
+MODULE_LICENSE("GPL");
-- 
1.7.7

[PATCH 5/5] xt_RLOG: add userspace-components

[Richard Weinberger]

From: Richard Weinberger <richard@nod.at>


Signed-off-by: Richard Weinberger <richard@nod.at>
---
 extensions/libxt_RLOG.c   |  171 +++++++++++++++++++++++++++++++++++++++++++++
 extensions/libxt_RLOG.man |   17 +++++
 2 files changed, 188 insertions(+), 0 deletions(-)
 create mode 100644 extensions/libxt_RLOG.c
 create mode 100644 extensions/libxt_RLOG.man

diff --git a/extensions/libxt_RLOG.c b/extensions/libxt_RLOG.c
new file mode 100644
index 0000000..1a7d697
--- /dev/null
+++ b/extensions/libxt_RLOG.c
@@ -0,0 +1,171 @@
+#include <stdio.h>
+#include <string.h>
+#include <ctype.h>
+#include <xtables.h>
+#include <linux/netfilter_ipv4/ipt_LOG.h>
+
+#ifndef IPT_LOG_UID /* Old kernel */
+#define IPT_LOG_UID		0x08    /* Log UID owning local socket */
+#undef  IPT_LOG_MASK
+#define IPT_LOG_MASK		0x0f
+#endif
+
+#define RING_NAME_LEN		32
+#define DEFAULT_RING_SIZE       1024 /* in KiB */
+#define DEFAULT_RING_NAME	"default"
+
+struct rlog_tg_info {
+	char name[RING_NAME_LEN];
+	unsigned int size;
+	unsigned char logflags;
+	bool add_timestamp;
+};
+
+enum {
+	O_RLOG_RING_SIZE = 0,
+	O_RLOG_RING,
+	O_LOG_TCPSEQ,
+	O_LOG_TCPOPTS,
+	O_LOG_IPOPTS,
+	O_LOG_UID,
+	O_LOG_MAC,
+	O_ADD_TS
+};
+
+static void RLOG_help(void)
+{
+	printf(
+"RLOG target options:\n"
+" --ring NAME			Log messages into NAME.\n\n"
+" --log-tcp-sequence            Log TCP sequence numbers.\n\n"
+" --log-tcp-options             Log TCP options.\n\n"
+" --log-ip-options              Log IP options.\n\n"
+" --log-uid                     Log UID owning the local socket.\n\n"
+" --log-macdecode               Decode MAC addresses and protocol.\n\n"
+" --add-timestamp		Add a timestamp to each log message.\n\n");
+}
+
+#define s struct rlog_tg_info
+static const struct xt_option_entry RLOG_opts[] = {
+	{.name = "size", .id = O_RLOG_RING_SIZE, .type = XTTYPE_UINT32,
+	 .flags = XTOPT_PUT, XTOPT_POINTER(s, size)},
+	{.name = "ring", .id = O_RLOG_RING, .type = XTTYPE_STRING,
+	 .flags = XTOPT_PUT, XTOPT_POINTER(s, name), .min = 1, .max = RING_NAME_LEN},
+	{.name = "log-tcp-sequence", .id = O_LOG_TCPSEQ, .type = XTTYPE_NONE},
+	{.name = "log-tcp-options", .id = O_LOG_TCPOPTS, .type = XTTYPE_NONE},
+	{.name = "log-ip-options", .id = O_LOG_IPOPTS, .type = XTTYPE_NONE},
+	{.name = "log-uid", .id = O_LOG_UID, .type = XTTYPE_NONE},
+	{.name = "log-macdecode", .id = O_LOG_MAC, .type = XTTYPE_NONE},
+	{.name = "add-timestamp", .id = O_ADD_TS, .type = XTTYPE_NONE},
+	XTOPT_TABLEEND,
+};
+#undef s
+
+static void RLOG_init(struct xt_entry_target *t)
+{
+	struct rlog_tg_info *info = (struct rlog_tg_info *)t->data;
+
+	info->size = DEFAULT_RING_SIZE;
+	strcpy(info->name, DEFAULT_RING_NAME);
+	info->add_timestamp = false;
+	info->logflags = 0;
+}
+
+static void RLOG_parse(struct xt_option_call *cb)
+{
+	struct rlog_tg_info *info = cb->data;
+	int len, i;
+
+	xtables_option_parse(cb);
+	switch (cb->entry->id) {
+	case O_RLOG_RING:
+		len = strlen(cb->arg);
+
+		for (i = 0; i < len; i++)
+			if (!isalnum(cb->arg[i]))
+				xtables_error(PARAMETER_PROBLEM, "--ring allows only alphanumeric names!\n");
+		break;
+	case O_LOG_TCPSEQ:
+		info->logflags |= IPT_LOG_TCPSEQ;
+		break;
+	case O_LOG_TCPOPTS:
+		info->logflags |= IPT_LOG_TCPOPT;
+		break;
+	case O_LOG_IPOPTS:
+		info->logflags |= IPT_LOG_IPOPT;
+		break;
+	case O_LOG_UID:
+		info->logflags |= IPT_LOG_UID;
+		break;
+	case O_LOG_MAC:
+		info->logflags |= IPT_LOG_MACDECODE;
+		break;
+	case O_ADD_TS:
+		info->add_timestamp = true;
+		break;
+	}
+}
+
+static void RLOG_save(const void *ip, const struct xt_entry_target *target)
+{
+	const struct rlog_tg_info *info
+		= (const struct rlog_tg_info *)target->data;
+
+	if (strcmp(info->name, DEFAULT_RING_NAME) != 0)
+		printf(" --ring %s", info->name);
+	if (info->size != DEFAULT_RING_SIZE)
+		printf(" --size %u", info->size);
+	if (info->logflags & IPT_LOG_TCPSEQ)
+		printf(" --log-tcp-sequence");
+	if (info->logflags & IPT_LOG_TCPOPT)
+		printf(" --log-tcp-options");
+	if (info->logflags & IPT_LOG_IPOPT)
+		printf(" --log-ip-options");
+	if (info->logflags & IPT_LOG_UID)
+		printf(" --log-uid");
+	if (info->logflags & IPT_LOG_MACDECODE)
+		printf(" --log-macdecode");
+	if (info->add_timestamp)
+		printf(" --add-timestamp");
+}
+
+static void RLOG_print(const void *ip, const struct xt_entry_target *target,
+                      int numeric)
+{
+	printf(" RLOG");
+	RLOG_save(ip, target);
+}
+
+static struct xtables_target rlog_tg_regs[] = {
+	{
+		.name          = "RLOG",
+		.version       = XTABLES_VERSION,
+		.family        = NFPROTO_IPV4,
+		.size          = XT_ALIGN(sizeof(struct rlog_tg_info) + sizeof (void *)),
+		.userspacesize = XT_ALIGN(sizeof(struct rlog_tg_info)),
+		.help          = RLOG_help,
+		.init          = RLOG_init,
+		.print         = RLOG_print,
+		.save          = RLOG_save,
+		.x6_parse      = RLOG_parse,
+		.x6_options    = RLOG_opts,
+	},
+	{
+		.name          = "RLOG",
+		.version       = XTABLES_VERSION,
+		.family        = NFPROTO_IPV6,
+		.size          = XT_ALIGN(sizeof(struct rlog_tg_info) + sizeof (void *)),
+		.userspacesize = XT_ALIGN(sizeof(struct rlog_tg_info)),
+		.help          = RLOG_help,
+		.init          = RLOG_init,
+		.print         = RLOG_print,
+		.save          = RLOG_save,
+		.x6_parse      = RLOG_parse,
+		.x6_options    = RLOG_opts,
+	}
+};
+
+void _init(void)
+{
+	xtables_register_targets(rlog_tg_regs, ARRAY_SIZE(rlog_tg_regs));
+}
diff --git a/extensions/libxt_RLOG.man b/extensions/libxt_RLOG.man
new file mode 100644
index 0000000..184c779
--- /dev/null
+++ b/extensions/libxt_RLOG.man
@@ -0,0 +1,17 @@
+Turn on kernel logging of matching packets. This target generates
+messages like LOG but instead of writing them into the kernel 
+syslog it writes them info one or more ring buffers
+Each ring buffer is represented as a file in 
+/proc/net/netfilter/xt_RLOG/.
+.TP
+\fB\-\-ring\fP \fIname\fP
+Write messages into ring buffer \fIname\fP.
+.TP
+\fB\-\-ring\fP \fIN\fP
+Make the ring buffer \fIN\fP KiB long.
+.TP
+\fB\-\-log\-tcp\-sequence\fP
+\fB\-\-log\-tcp\-options\fP
+\fB\-\-log\-ip\-options\fP
+\fB\-\-log\-uid\fP
+See LOG.
-- 
1.7.7

Re: [PATCH 1/5] ring_buffer: Export for_each_buffer_cpu()

[Steven Rostedt]

On Wed, 2012-01-18 at 23:43 +0100, Richard Weinberger wrote:
> From: Richard Weinberger <richard@nod.at>
> 

Missing change log.


Otherwise,

Acked-by: Steven Rostedt <rostedt@goodmis.org>

-- Steve

> 
> Signed-off-by: Richard Weinberger <richard@nod.at>
> ---
>  include/linux/ring_buffer.h |    5 +++++
>  kernel/trace/ring_buffer.c  |    7 +++++++
>  2 files changed, 12 insertions(+), 0 deletions(-)
> 
> diff --git a/include/linux/ring_buffer.h b/include/linux/ring_buffer.h
> index 67be037..5c1c29f 100644
> --- a/include/linux/ring_buffer.h
> +++ b/include/linux/ring_buffer.h
> @@ -185,4 +185,9 @@ enum ring_buffer_flags {
>  	RB_FL_OVERWRITE		= 1 << 0,
>  };
>  
> +cpumask_var_t *ring_buffer_mask(struct ring_buffer *buffer);
> +
> +#define for_each_buffer_cpu(buffer, cpu) \
> +	for_each_cpu(cpu, *(ring_buffer_mask(buffer)))
> +
>  #endif /* _LINUX_RING_BUFFER_H */
> diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
> index f5b7b5c..a4117c5 100644
> --- a/kernel/trace/ring_buffer.c
> +++ b/kernel/trace/ring_buffer.c
> @@ -348,6 +348,7 @@ void *ring_buffer_event_data(struct ring_buffer_event *event)
>  }
>  EXPORT_SYMBOL_GPL(ring_buffer_event_data);
>  
> +#undef for_each_buffer_cpu
>  #define for_each_buffer_cpu(buffer, cpu)		\
>  	for_each_cpu(cpu, buffer->cpumask)
>  
> @@ -4139,3 +4140,9 @@ static int rb_cpu_notify(struct notifier_block *self,
>  	return NOTIFY_OK;
>  }
>  #endif
> +
> +cpumask_var_t *ring_buffer_mask(struct ring_buffer *buffer)
> +{
> +	return &buffer->cpumask;
> +}
> +EXPORT_SYMBOL_GPL(ring_buffer_mask);

Re: [PATCH 2/5] xt_log: Make printk() in sb_close() optional

[Steven Rostedt]

On Wed, 2012-01-18 at 23:43 +0100, Richard Weinberger wrote:
> From: Richard Weinberger <richard@nod.at>
> 
> 

Missing change log. Note, the subject is not good enough of a
description.

-- Steve

> Signed-off-by: Richard Weinberger <richard@nod.at>
> ---
>  include/net/netfilter/xt_log.h |    9 ++++++---
>  1 files changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/include/net/netfilter/xt_log.h b/include/net/netfilter/xt_log.h
> index 0dfb34a..767e08d 100644
> --- a/include/net/netfilter/xt_log.h
> +++ b/include/net/netfilter/xt_log.h
> @@ -39,10 +39,12 @@ static struct sbuff *sb_open(void)
>  	return m;
>  }
>  
> -static void sb_close(struct sbuff *m)
> +static void __sb_close(struct sbuff *m, int print)
>  {
> -	m->buf[m->count] = 0;
> -	printk("%s\n", m->buf);
> +	if (print) {
> +		m->buf[m->count] = 0;
> +		printk("%s\n", m->buf);
> +	}
>  
>  	if (likely(m != &emergency))
>  		kfree(m);
> @@ -52,3 +54,4 @@ static void sb_close(struct sbuff *m)
>  	}
>  }
>  
> +#define sb_close(m)	__sb_close(m, 1)

Re: [PATCH 3/5] nf_log: Export dump_packet() and dump_mac_header() functions

[Steven Rostedt]

On Wed, 2012-01-18 at 23:43 +0100, Richard Weinberger wrote:
> From: Richard Weinberger <richard@nod.at>
> 

Let me guess. You left out the change logs in protest to SOPA and PIPA?

-- Steve

> 
> Signed-off-by: Richard Weinberger <richard@nod.at>
> ---
>  include/net/netfilter/nf_log.h |   19 +++++++++++++++++++
>  net/ipv4/netfilter/ipt_LOG.c   |   12 +++++++-----
>  net/ipv6/netfilter/ip6t_LOG.c  |   12 +++++++-----
>  3 files changed, 33 insertions(+), 10 deletions(-)
> 
> diff --git a/include/net/netfilter/nf_log.h b/include/net/netfilter/nf_log.h
> index e991bd0..c0f2b27 100644
> --- a/include/net/netfilter/nf_log.h
> +++ b/include/net/netfilter/nf_log.h
> @@ -61,5 +61,24 @@ void nf_log_packet(u_int8_t pf,
>  		   const struct net_device *out,
>  		   const struct nf_loginfo *li,
>  		   const char *fmt, ...);
> +struct sbuff;
> +void ipt_dump_packet(struct sbuff *m,
> +	const struct nf_loginfo *info,
> +	const struct sk_buff *skb,
> +	unsigned int iphoff);
> +
> +void ipt_dump_mac_header(struct sbuff *m,
> +	const struct nf_loginfo *info,
> +	const struct sk_buff *skb);
> +
> +void ip6t_dump_packet(struct sbuff *m,
> +	const struct nf_loginfo *info,
> +	const struct sk_buff *skb,
> +	unsigned int iphoff,
> +	int recurse);
> +
> +void ip6t_dump_mac_header(struct sbuff *m,
> +	const struct nf_loginfo *info,
> +	const struct sk_buff *skb);
>  
>  #endif /* _NF_LOG_H */
> diff --git a/net/ipv4/netfilter/ipt_LOG.c b/net/ipv4/netfilter/ipt_LOG.c
> index d76d6c9..8ec9d2b 100644
> --- a/net/ipv4/netfilter/ipt_LOG.c
> +++ b/net/ipv4/netfilter/ipt_LOG.c
> @@ -31,7 +31,7 @@ MODULE_AUTHOR("Netfilter Core Team <coreteam@netfilter.org>");
>  MODULE_DESCRIPTION("Xtables: IPv4 packet logging to syslog");
>  
>  /* One level of recursion won't kill us */
> -static void dump_packet(struct sbuff *m,
> +void ipt_dump_packet(struct sbuff *m,
>  			const struct nf_loginfo *info,
>  			const struct sk_buff *skb,
>  			unsigned int iphoff)
> @@ -270,7 +270,7 @@ static void dump_packet(struct sbuff *m,
>  			/* Max length: 3+maxlen */
>  			if (!iphoff) { /* Only recurse once. */
>  				sb_add(m, "[");
> -				dump_packet(m, info, skb,
> +				ipt_dump_packet(m, info, skb,
>  					    iphoff + ih->ihl*4+sizeof(_icmph));
>  				sb_add(m, "] ");
>  			}
> @@ -362,8 +362,9 @@ static void dump_packet(struct sbuff *m,
>  	/* maxlen =  IP + ICMP +  IP + max(TCP,UDP,ICMP,unknown) */
>  	/* maxlen = 230+   91  + 230 + 252 = 803 */
>  }
> +EXPORT_SYMBOL_GPL(ipt_dump_packet);
>  
> -static void dump_mac_header(struct sbuff *m,
> +void ipt_dump_mac_header(struct sbuff *m,
>  			    const struct nf_loginfo *info,
>  			    const struct sk_buff *skb)
>  {
> @@ -399,6 +400,7 @@ fallback:
>  	}
>  	sb_add(m, " ");
>  }
> +EXPORT_SYMBOL_GPL(ipt_dump_mac_header);
>  
>  static struct nf_loginfo default_loginfo = {
>  	.type	= NF_LOG_TYPE_LOG,
> @@ -443,9 +445,9 @@ ipt_log_packet(u_int8_t pf,
>  #endif
>  
>  	if (in != NULL)
> -		dump_mac_header(m, loginfo, skb);
> +		ipt_dump_mac_header(m, loginfo, skb);
>  
> -	dump_packet(m, loginfo, skb, 0);
> +	ipt_dump_packet(m, loginfo, skb, 0);
>  
>  	sb_close(m);
>  }
> diff --git a/net/ipv6/netfilter/ip6t_LOG.c b/net/ipv6/netfilter/ip6t_LOG.c
> index e6af8d7..c6abacb 100644
> --- a/net/ipv6/netfilter/ip6t_LOG.c
> +++ b/net/ipv6/netfilter/ip6t_LOG.c
> @@ -34,7 +34,7 @@ struct in_device;
>  #include <linux/netfilter_ipv6/ip6t_LOG.h>
>  
>  /* One level of recursion won't kill us */
> -static void dump_packet(struct sbuff *m,
> +void ip6t_dump_packet(struct sbuff *m,
>  			const struct nf_loginfo *info,
>  			const struct sk_buff *skb, unsigned int ip6hoff,
>  			int recurse)
> @@ -341,7 +341,7 @@ static void dump_packet(struct sbuff *m,
>  			/* Max length: 3+maxlen */
>  			if (recurse) {
>  				sb_add(m, "[");
> -				dump_packet(m, info, skb,
> +				ip6t_dump_packet(m, info, skb,
>  					    ptr + sizeof(_icmp6h), 0);
>  				sb_add(m, "] ");
>  			}
> @@ -371,8 +371,9 @@ static void dump_packet(struct sbuff *m,
>  	if (!recurse && skb->mark)
>  		sb_add(m, "MARK=0x%x ", skb->mark);
>  }
> +EXPORT_SYMBOL(ip6t_dump_packet);
>  
> -static void dump_mac_header(struct sbuff *m,
> +void ip6t_dump_mac_header(struct sbuff *m,
>  			    const struct nf_loginfo *info,
>  			    const struct sk_buff *skb)
>  {
> @@ -422,6 +423,7 @@ fallback:
>  	} else
>  		sb_add(m, " ");
>  }
> +EXPORT_SYMBOL(ip6t_dump_mac_header);
>  
>  static struct nf_loginfo default_loginfo = {
>  	.type	= NF_LOG_TYPE_LOG,
> @@ -453,9 +455,9 @@ ip6t_log_packet(u_int8_t pf,
>  	       out ? out->name : "");
>  
>  	if (in != NULL)
> -		dump_mac_header(m, loginfo, skb);
> +		ip6t_dump_mac_header(m, loginfo, skb);
>  
> -	dump_packet(m, loginfo, skb, skb_network_offset(skb), 1);
> +	ip6t_dump_packet(m, loginfo, skb, skb_network_offset(skb), 1);
>  
>  	sb_close(m);
>  }

Re: Netfilter: New target: RLOG

[Pablo Neira Ayuso]

Hi Richard,

On Wed, Jan 18, 2012 at 11:43:25PM +0100, Richard Weinberger wrote:
> RLOG is a new log target, it works like LOG with the exception that it writes to ring buffers.
> It makes use of Steven Rostedt's ring_buffer subsystem. 
> I've used Steve's ring buffer because it allows concurrent writes. IOW it's very fast.
> For more details see: Documentation/trace/ring-buffer-design.txt.
> 
> Each ring buffer is represented as a pipe-like file in /proc/net/netfilter/xt_RLOG/.
> You can read from it with and program you like (cat, syslog, etc...).
> The default size is 1MiB. With this size it can store approximately 5000 messages.
> 
> - Why not LOG?
> I like the LOG target a lot but I really hat it when it floods my kernel syslog.
> dmesg becomes useless.
> Writing all log messages to a file using syslogd also not always the best solution.
> Most of the time my firewall logs just waste disk space.
> 
> Compared with Steve's ring_buffer, the kernel syslog is rather slow.
> Especially when the firewall logs very much syslog becomes a bottleneck.
> As we all know printk() is not fast.
> 
> - Why not ULOG/NFLOG?
> Because it cannot replace LOG.
> Details like PHYSIN and PHYSOUT are not available form the packet headers.
> Also on many Linux systems ulogd is not available/supported.  

We only include physin and phyout if netfilter bridge is enabled. I
may be missing anything but, why can these be useful if bridging is not
enabled?

> - Why RLOG?
> Using RLOG you can have many ring buffers with all kind of logs.
> If your firewall goes nuts you don't have to mess you rule-set with adding
> new LOG rules to find out what's going on.
> Just install a few RLOG rules with small buffer sized and read them if you don't
> know what's going on.
> If you make you firewall rule-set per default verbose using LOG or NFLOG it will 
> generate lot's of useless messages which you'll never ever read.
> With RLOG you can bypass this problem.
> On my firewall I record only useful data to the disk. Everything else goes into RLOG.
> If your firewall is really busy and you want to log nearly everything, c
> reate a big ring buffer and read from is using your favorite userspace tool.
> In case the buffer fills faster than the userspace consumes it, RLOG will warn you.
> I'd also possible to resize the buffer.

I still think this can be useful.

But, why don't you add this to the LOG target as an extension instead
of yet another target?

Re: Netfilter: New target: RLOG

[Richard Weinberger]

On 19.01.2012 10:12, Pablo Neira Ayuso wrote:
> Hi Richard,
>
> On Wed, Jan 18, 2012 at 11:43:25PM +0100, Richard Weinberger wrote:
>> RLOG is a new log target, it works like LOG with the exception that it writes to ring buffers.
>> It makes use of Steven Rostedt's ring_buffer subsystem.
>> I've used Steve's ring buffer because it allows concurrent writes. IOW it's very fast.
>> For more details see: Documentation/trace/ring-buffer-design.txt.
>>
>> Each ring buffer is represented as a pipe-like file in /proc/net/netfilter/xt_RLOG/.
>> You can read from it with and program you like (cat, syslog, etc...).
>> The default size is 1MiB. With this size it can store approximately 5000 messages.
>>
>> - Why not LOG?
>> I like the LOG target a lot but I really hat it when it floods my kernel syslog.
>> dmesg becomes useless.
>> Writing all log messages to a file using syslogd also not always the best solution.
>> Most of the time my firewall logs just waste disk space.
>>
>> Compared with Steve's ring_buffer, the kernel syslog is rather slow.
>> Especially when the firewall logs very much syslog becomes a bottleneck.
>> As we all know printk() is not fast.
>>
>> - Why not ULOG/NFLOG?
>> Because it cannot replace LOG.
>> Details like PHYSIN and PHYSOUT are not available form the packet headers.
>> Also on many Linux systems ulogd is not available/supported.
>
> We only include physin and phyout if netfilter bridge is enabled. I
> may be missing anything but, why can these be useful if bridging is not
> enabled?

Of course they are only useful if bridging enabled.
In nearly all of my use-cases I'm using bridging (KVM, LXC, ...).

>> - Why RLOG?
>> Using RLOG you can have many ring buffers with all kind of logs.
>> If your firewall goes nuts you don't have to mess you rule-set with adding
>> new LOG rules to find out what's going on.
>> Just install a few RLOG rules with small buffer sized and read them if you don't
>> know what's going on.
>> If you make you firewall rule-set per default verbose using LOG or NFLOG it will
>> generate lot's of useless messages which you'll never ever read.
>> With RLOG you can bypass this problem.
>> On my firewall I record only useful data to the disk. Everything else goes into RLOG.
>> If your firewall is really busy and you want to log nearly everything, c
>> reate a big ring buffer and read from is using your favorite userspace tool.
>> In case the buffer fills faster than the userspace consumes it, RLOG will warn you.
>> I'd also possible to resize the buffer.
>
> I still think this can be useful.
>
> But, why don't you add this to the LOG target as an extension instead
> of yet another target?

Yeah, I could add --ring, --ring-size and --add-timestamp to LOG.
What about a rlog_common.o which can be used by ipt_LOG and ip6t_LOG?

Thanks,
//richard

Re: Netfilter: New target: RLOG

[Pablo Neira Ayuso]

On Thu, Jan 19, 2012 at 10:12:48AM +0100, Pablo Neira Ayuso wrote:
> I still think this can be useful.
> 
> But, why don't you add this to the LOG target as an extension instead
> of yet another target?

By "extension", I meant to say "revision".

If you're not familiar with the iptables revision mechanism, for
instance, have a look at net/netfilter/xt_NFQUEUE.c

Look for the "revision" pattern in that code. Basicaly, it allows you
to enhance existing matches/targets without breaking backward
compatibility.

Re: Netfilter: New target: RLOG

[Eric Dumazet]

Le jeudi 19 janvier 2012 à 10:21 +0100, Richard Weinberger a écrit :

> Yeah, I could add --ring, --ring-size and --add-timestamp to LOG.
> What about a rlog_common.o which can be used by ipt_LOG and ip6t_LOG?
> 

or merge ipt_LOG and ip6t_LOG into xt_LOG ?

Re: Netfilter: New target: RLOG

[Richard Weinberger]

On 19.01.2012 10:25, Pablo Neira Ayuso wrote:
> On Thu, Jan 19, 2012 at 10:12:48AM +0100, Pablo Neira Ayuso wrote:
>> I still think this can be useful.
>>
>> But, why don't you add this to the LOG target as an extension instead
>> of yet another target?
>
> By "extension", I meant to say "revision".
>
> If you're not familiar with the iptables revision mechanism, for
> instance, have a look at net/netfilter/xt_NFQUEUE.c
>
> Look for the "revision" pattern in that code. Basicaly, it allows you
> to enhance existing matches/targets without breaking backward
> compatibility.

Okay, will look and send new patches. :-)

Thanks,
//richard

Re: Netfilter: New target: RLOG

[Jan Engelhardt]

On Thursday 2012-01-19 10:26, Eric Dumazet wrote:

>Le jeudi 19 janvier 2012 à 10:21 +0100, Richard Weinberger a écrit :
>
>> Yeah, I could add --ring, --ring-size and --add-timestamp to LOG.
>> What about a rlog_common.o which can be used by ipt_LOG and ip6t_LOG?
>> 
>
>or merge ipt_LOG and ip6t_LOG into xt_LOG ?

yes please :)

Re: Netfilter: New target: RLOG

[Richard Weinberger]

On 19.01.2012 10:46, Jan Engelhardt wrote:
> On Thursday 2012-01-19 10:26, Eric Dumazet wrote:
>
>> Le jeudi 19 janvier 2012 à 10:21 +0100, Richard Weinberger a écrit :
>>
>>> Yeah, I could add --ring, --ring-size and --add-timestamp to LOG.
>>> What about a rlog_common.o which can be used by ipt_LOG and ip6t_LOG?
>>>
>>
>> or merge ipt_LOG and ip6t_LOG into xt_LOG ?
>
> yes please :)

Will do.

Thanks,
//richard