[PATCH] Add support to log original and NAT-ed IP addresses

[PATCH] Add support to log original and NAT-ed IP addresses

[Jozsef Kadlecsik]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1605 bytes --]

Hi,

Attached you can find patches for netfilter and iptabes to support the 
logging of the original and NAT-ed IP addresses together.

Currently there's no way to do it by netfilter/iptables. If we log in the 
filter table, there we can record the original src IP address only. 
However, we cannot log the src IP address after NAT at all: SNAT happens 
in the nat table at POSTROUTING, and there's no other table to which the 
logging rule could be added (and the NAT targets return ACCEPT, so we 
cannot add the loggin rule to the nat table either).

The only way to log src/dst IP before/after NAT presently is to run 
'conntrack' in event mode like this:

	conntrack -E -e NEW | logger -p kernel.info

which is a little bit cumbersome and requires the libnfnetlink, 
libnetfilter_conntrack libraries too.

The attached netfilter patch is questionable/incomplete in the sense that:

- it adds the POST_ROUTING hook to the raw table and in the IPv4 case 
  only: but there's no much point to add the hook to IPv6
- for the sake of completeness the LOCAL_IN hook should be added too, 
  in order to make possilbe the logging of local SNAT-ed addresses
- the original src/dst ports could be logged as well: in practice the
  original src IP address matters only when one wants to hunt down
  an infected machine on a NATed network

Comments are welcomed.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: log-orig.patch --]
[-- Type: TEXT/x-diff; name=log-orig.patch, Size: 6981 bytes --]

diff --git a/include/linux/netfilter_ipv4.h b/include/linux/netfilter_ipv4.h
index 29c7727..2e6dad5 100644
--- a/include/linux/netfilter_ipv4.h
+++ b/include/linux/netfilter_ipv4.h
@@ -63,6 +63,7 @@ enum nf_ip_hook_priorities {
 	NF_IP_PRI_SECURITY = 50,
 	NF_IP_PRI_NAT_SRC = 100,
 	NF_IP_PRI_SELINUX_LAST = 225,
+	NF_IP_PRI_RAW_LOG = 300,
 	NF_IP_PRI_CONNTRACK_CONFIRM = INT_MAX,
 	NF_IP_PRI_LAST = INT_MAX,
 };
diff --git a/include/linux/netfilter_ipv4/ipt_LOG.h b/include/linux/netfilter_ipv4/ipt_LOG.h
index 90fa652..a63c0cd 100644
--- a/include/linux/netfilter_ipv4/ipt_LOG.h
+++ b/include/linux/netfilter_ipv4/ipt_LOG.h
@@ -7,7 +7,8 @@
 #define IPT_LOG_IPOPT		0x04	/* Log IP options */
 #define IPT_LOG_UID		0x08	/* Log UID owning local socket */
 #define IPT_LOG_NFLOG		0x10	/* Unsupported, don't reuse */
-#define IPT_LOG_MASK		0x1f
+#define IPT_LOG_ORIG		0x20	/* Log original src/dst from ct */
+#define IPT_LOG_MASK		0x3f
 
 struct ipt_log_info {
 	unsigned char level;
diff --git a/include/net/netfilter/nf_log.h b/include/net/netfilter/nf_log.h
index 7182c06..60def34 100644
--- a/include/net/netfilter/nf_log.h
+++ b/include/net/netfilter/nf_log.h
@@ -8,7 +8,8 @@
 #define NF_LOG_TCPOPT		0x02	/* Log TCP options */
 #define NF_LOG_IPOPT		0x04	/* Log IP options */
 #define NF_LOG_UID		0x08	/* Log UID owning local socket */
-#define NF_LOG_MASK		0x0f
+#define NF_LOG_ORIG		0x20	/* Log original src/dst from ct */
+#define NF_LOG_MASK		0x2f
 
 #define NF_LOG_TYPE_LOG		0x01
 #define NF_LOG_TYPE_ULOG	0x02
diff --git a/net/ipv4/netfilter/Kconfig b/net/ipv4/netfilter/Kconfig
index 3816e1d..98c4ad3 100644
--- a/net/ipv4/netfilter/Kconfig
+++ b/net/ipv4/netfilter/Kconfig
@@ -132,6 +132,16 @@ config IP_NF_TARGET_LOG
 
 	  To compile it as a module, choose M here.  If unsure, say N.
 
+config IP_NF_TARGET_LOG_ORIG
+	bool "LOG target support for logging of original src/dst IP"
+	depends on NF_NAT
+	default y
+	help
+	  This option enables logging of the original src/dst IP besides
+	  the ones which can be found in the packet. The feature makes
+	  possible to log the source/destination IP addresses before and
+	  after NAT.
+
 config IP_NF_TARGET_ULOG
 	tristate "ULOG target support"
 	default m if NETFILTER_ADVANCED=n
diff --git a/net/ipv4/netfilter/ipt_LOG.c b/net/ipv4/netfilter/ipt_LOG.c
index 27a78fb..076db8e 100644
--- a/net/ipv4/netfilter/ipt_LOG.c
+++ b/net/ipv4/netfilter/ipt_LOG.c
@@ -23,6 +23,9 @@
 #include <linux/netfilter/x_tables.h>
 #include <linux/netfilter_ipv4/ipt_LOG.h>
 #include <net/netfilter/nf_log.h>
+#ifdef CONFIG_IP_NF_TARGET_LOG_ORIG
+#include <net/netfilter/nf_conntrack.h>
+#endif
 
 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Netfilter Core Team <coreteam@netfilter.org>");
@@ -348,8 +351,28 @@ static void dump_packet(const struct nf_loginfo *info,
 	if (!iphoff && skb->mark)
 		printk("MARK=0x%x ", skb->mark);
 
+#ifdef CONFIG_IP_NF_TARGET_LOG_ORIG
+	/* Original src/dst from ct */
+	if ((logflags & IPT_LOG_ORIG) && !iphoff) {
+		enum ip_conntrack_info ctinfo;
+		struct nf_conn *ct;
+		
+		ct = nf_ct_get(skb, &ctinfo);
+		/* Max length: 48 "ORIGSRC=255.255.255.255 ORIGDST=255.255.255.255 " */
+		if (!ct || ct == &nf_conntrack_untracked)
+			printk("ORIGSRC=%u.%u.%u.%u ORIGDST=%u.%u.%u.%u ",
+			       NIPQUAD(ih->saddr), NIPQUAD(ih->daddr));
+		else {
+			enum ip_conntrack_dir dir = CTINFO2DIR(ctinfo);
+			printk("ORIGSRC=%u.%u.%u.%u ORIGDST=%u.%u.%u.%u ",
+			       NIPQUAD(ct->tuplehash[dir].tuple.src.u3.ip),
+			       NIPQUAD(ct->tuplehash[dir].tuple.dst.u3.ip));
+		}
+	}
+#endif
+
 	/* Proto    Max log string length */
-	/* IP:      40+46+6+11+127 = 230 */
+	/* IP:      40+46+6+11+127+48 = 278 */
 	/* TCP:     10+max(25,20+30+13+9+32+11+127) = 252 */
 	/* UDP:     10+max(25,20) = 35 */
 	/* UDPLITE: 14+max(25,20) = 39 */
@@ -360,7 +383,7 @@ static void dump_packet(const struct nf_loginfo *info,
 
 	/* (ICMP allows recursion one level deep) */
 	/* maxlen =  IP + ICMP +  IP + max(TCP,UDP,ICMP,unknown) */
-	/* maxlen = 230+   91  + 230 + 252 = 803 */
+	/* maxlen = 278+   91  + 230 + 252 = 851 */
 }
 
 static struct nf_loginfo default_loginfo = {
diff --git a/net/ipv4/netfilter/iptable_raw.c b/net/ipv4/netfilter/iptable_raw.c
index 7f65d18..466c8bc 100644
--- a/net/ipv4/netfilter/iptable_raw.c
+++ b/net/ipv4/netfilter/iptable_raw.c
@@ -1,5 +1,6 @@
 /*
  * 'raw' table, which is the very first hooked in at PRE_ROUTING and LOCAL_OUT .
+ * - POST_ROUTING is added to support logging after NAT (2009).
  *
  * Copyright (C) 2003 Jozsef Kadlecsik <kadlec@blackhole.kfki.hu>
  */
@@ -7,31 +8,36 @@
 #include <linux/netfilter_ipv4/ip_tables.h>
 #include <net/ip.h>
 
-#define RAW_VALID_HOOKS ((1 << NF_INET_PRE_ROUTING) | (1 << NF_INET_LOCAL_OUT))
+#define RAW_VALID_HOOKS ((1 << NF_INET_PRE_ROUTING) | \
+			 (1 << NF_INET_LOCAL_OUT) |\
+			 (1 << NF_INET_POST_ROUTING))
 
 static struct
 {
 	struct ipt_replace repl;
-	struct ipt_standard entries[2];
+	struct ipt_standard entries[3];
 	struct ipt_error term;
 } initial_table __net_initdata = {
 	.repl = {
 		.name = "raw",
 		.valid_hooks = RAW_VALID_HOOKS,
-		.num_entries = 3,
-		.size = sizeof(struct ipt_standard) * 2 + sizeof(struct ipt_error),
+		.num_entries = 4,
+		.size = sizeof(struct ipt_standard) * 3 + sizeof(struct ipt_error),
 		.hook_entry = {
 			[NF_INET_PRE_ROUTING] = 0,
-			[NF_INET_LOCAL_OUT] = sizeof(struct ipt_standard)
+			[NF_INET_LOCAL_OUT] = sizeof(struct ipt_standard),
+			[NF_INET_POST_ROUTING] = sizeof(struct ipt_standard) * 2,
 		},
 		.underflow = {
 			[NF_INET_PRE_ROUTING] = 0,
-			[NF_INET_LOCAL_OUT]  = sizeof(struct ipt_standard)
+			[NF_INET_LOCAL_OUT]  = sizeof(struct ipt_standard),
+			[NF_INET_POST_ROUTING] = sizeof(struct ipt_standard) * 2,
 		},
 	},
 	.entries = {
 		IPT_STANDARD_INIT(NF_ACCEPT),	/* PRE_ROUTING */
 		IPT_STANDARD_INIT(NF_ACCEPT),	/* LOCAL_OUT */
+		IPT_STANDARD_INIT(NF_ACCEPT),   /* POST_ROUTING */ 
 	},
 	.term = IPT_ERROR_INIT,			/* ERROR */
 };
@@ -71,6 +77,17 @@ ipt_local_hook(unsigned int hook,
 			    dev_net(out)->ipv4.iptable_raw);
 }
 
+static unsigned int
+ipt_log_hook(unsigned int hook,
+	     struct sk_buff *skb,
+	     const struct net_device *in,
+	     const struct net_device *out,
+	     int (*okfn)(struct sk_buff *))
+{
+	return ipt_do_table(skb, hook, in, out,
+			    dev_net(out)->ipv4.iptable_raw);
+}
+
 /* 'raw' is the very first table. */
 static struct nf_hook_ops ipt_ops[] __read_mostly = {
 	{
@@ -87,6 +104,13 @@ static struct nf_hook_ops ipt_ops[] __read_mostly = {
 		.priority = NF_IP_PRI_RAW,
 		.owner = THIS_MODULE,
 	},
+	{
+		.hook = ipt_log_hook,
+		.pf = PF_INET,
+		.hooknum = NF_INET_POST_ROUTING,
+		.priority = NF_IP_PRI_RAW_LOG,
+		.owner = THIS_MODULE,
+	},
 };
 
 static int __net_init iptable_raw_net_init(struct net *net)

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: log-orig.iptables.patch --]
[-- Type: TEXT/x-diff; name=log-orig.iptables.patch, Size: 3302 bytes --]

diff --git a/extensions/libipt_LOG.c b/extensions/libipt_LOG.c
index 50b1130..6f0b38b 100644
--- a/extensions/libipt_LOG.c
+++ b/extensions/libipt_LOG.c
@@ -14,7 +14,7 @@
 #ifndef IPT_LOG_UID /* Old kernel */
 #define IPT_LOG_UID	0x08	/* Log UID owning local socket */
 #undef  IPT_LOG_MASK
-#define IPT_LOG_MASK	0x0f
+#define IPT_LOG_MASK	0x2f
 #endif
 
 /* Function which prints out usage message. */
@@ -27,7 +27,8 @@ static void LOG_help(void)
 " --log-tcp-sequence		Log TCP sequence numbers.\n\n"
 " --log-tcp-options		Log TCP options.\n\n"
 " --log-ip-options		Log IP options.\n\n"
-" --log-uid			Log UID owning the local socket.\n\n");
+" --log-uid			Log UID owning the local socket.\n\n"
+" --log-orig			Log original src/dst IP.\n\n");
 }
 
 static const struct option LOG_opts[] = {
@@ -37,6 +38,7 @@ static const struct option LOG_opts[] = {
 	{ .name = "log-tcp-options",  .has_arg = 0, .val = '2' },
 	{ .name = "log-ip-options",   .has_arg = 0, .val = '3' },
 	{ .name = "log-uid",          .has_arg = 0, .val = '4' },
+	{ .name = "log-orig",         .has_arg = 0, .val = '5' },
 	{ .name = NULL }
 };
 
@@ -102,6 +104,7 @@ parse_level(const char *level)
 #define IPT_LOG_OPT_TCPOPT 0x08
 #define IPT_LOG_OPT_IPOPT 0x10
 #define IPT_LOG_OPT_UID 0x20
+#define IPT_LOG_OPT_ORIG 0x40
 
 /* Function which parses command options; returns true if it
    ate an option */
@@ -168,7 +171,6 @@ static int LOG_parse(int c, char **argv, int invert, unsigned int *flags,
 		loginfo->logflags |= IPT_LOG_TCPOPT;
 		*flags |= IPT_LOG_OPT_TCPOPT;
 		break;
-
 	case '3':
 		if (*flags & IPT_LOG_OPT_IPOPT)
 			exit_error(PARAMETER_PROBLEM,
@@ -187,6 +189,15 @@ static int LOG_parse(int c, char **argv, int invert, unsigned int *flags,
 		*flags |= IPT_LOG_OPT_UID;
 		break;
 
+	case '5':
+		if (*flags & IPT_LOG_OPT_ORIG)
+			exit_error(PARAMETER_PROBLEM,
+				   "Can't specify --log-orig twice");
+
+		loginfo->logflags |= IPT_LOG_ORIG;
+		*flags |= IPT_LOG_OPT_ORIG;
+		break;
+
 	default:
 		return 0;
 	}
@@ -225,6 +236,8 @@ static void LOG_print(const void *ip, const struct xt_entry_target *target,
 			printf("ip-options ");
 		if (loginfo->logflags & IPT_LOG_UID)
 			printf("uid ");
+		if (loginfo->logflags & IPT_LOG_ORIG)
+			printf("orig ");
 		if (loginfo->logflags & ~(IPT_LOG_MASK))
 			printf("unknown-flags ");
 	}
@@ -255,6 +268,8 @@ static void LOG_save(const void *ip, const struct xt_entry_target *target)
 		printf("--log-ip-options ");
 	if (loginfo->logflags & IPT_LOG_UID)
 		printf("--log-uid ");
+	if (loginfo->logflags & IPT_LOG_ORIG)
+		printf("--log-orig ");
 }
 
 static struct xtables_target log_tg_reg = {
diff --git a/include/linux/netfilter_ipv4/ipt_LOG.h b/include/linux/netfilter_ipv4/ipt_LOG.h
index 90fa652..a63c0cd 100644
--- a/include/linux/netfilter_ipv4/ipt_LOG.h
+++ b/include/linux/netfilter_ipv4/ipt_LOG.h
@@ -7,7 +7,8 @@
 #define IPT_LOG_IPOPT		0x04	/* Log IP options */
 #define IPT_LOG_UID		0x08	/* Log UID owning local socket */
 #define IPT_LOG_NFLOG		0x10	/* Unsupported, don't reuse */
-#define IPT_LOG_MASK		0x1f
+#define IPT_LOG_ORIG		0x20	/* Log original src/dst from ct */
+#define IPT_LOG_MASK		0x3f
 
 struct ipt_log_info {
 	unsigned char level;

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Pablo Neira Ayuso]

Hi Jozsef,

Jozsef Kadlecsik wrote:
> Hi,
> 
> Attached you can find patches for netfilter and iptabes to support the 
> logging of the original and NAT-ed IP addresses together.
> 
> Currently there's no way to do it by netfilter/iptables. If we log in the 
> filter table, there we can record the original src IP address only. 
> However, we cannot log the src IP address after NAT at all: SNAT happens 
> in the nat table at POSTROUTING, and there's no other table to which the 
> logging rule could be added (and the NAT targets return ACCEPT, so we 
> cannot add the loggin rule to the nat table either).
> 
> The only way to log src/dst IP before/after NAT presently is to run 
> 'conntrack' in event mode like this:
> 
> 	conntrack -E -e NEW | logger -p kernel.info

We can also do this by means of ulogd2 or, alternatively, conntrackd in
its very basic statistics mode.

> which is a little bit cumbersome and requires the libnfnetlink, 
> libnetfilter_conntrack libraries too.

Yes, this needs some extra user-space software. With regards to logging,
I think that the current policy is to move most of the logics to
user-space, I don't think that overloading iptables with yet another
logging facility is the way to go.

> The attached netfilter patch is questionable/incomplete in the sense that:
> 
> - it adds the POST_ROUTING hook to the raw table and in the IPv4 case 
>   only: but there's no much point to add the hook to IPv6
> - for the sake of completeness the LOCAL_IN hook should be added too, 
>   in order to make possilbe the logging of local SNAT-ed addresses
> - the original src/dst ports could be logged as well: in practice the
>   original src IP address matters only when one wants to hunt down
>   an infected machine on a NATed network

I think that Patrick is not going to like the idea of adding more hooks,
what do you think Patrick?

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Jozsef Kadlecsik]

Hi Pablo,

On Mon, 20 Apr 2009, Pablo Neira Ayuso wrote:

> > Attached you can find patches for netfilter and iptabes to support the 
> > logging of the original and NAT-ed IP addresses together.
> > 
> > Currently there's no way to do it by netfilter/iptables. If we log in the 
> > filter table, there we can record the original src IP address only. 
> > However, we cannot log the src IP address after NAT at all: SNAT happens 
> > in the nat table at POSTROUTING, and there's no other table to which the 
> > logging rule could be added (and the NAT targets return ACCEPT, so we 
> > cannot add the loggin rule to the nat table either).
> > 
> > The only way to log src/dst IP before/after NAT presently is to run 
> > 'conntrack' in event mode like this:
> > 
> > 	conntrack -E -e NEW | logger -p kernel.info
> 
> We can also do this by means of ulogd2 or, alternatively, conntrackd in
> its very basic statistics mode.

But ulogd2 requires the ULOG target and ULOG cannot log the SNAT-ed 
address either, similarly to the vanilla LOG target: there's presently no 
hook at which the information is available for the LOG/ULOG targets.
 
> > which is a little bit cumbersome and requires the libnfnetlink, 
> > libnetfilter_conntrack libraries too.
> 
> Yes, this needs some extra user-space software. With regards to logging,
> I think that the current policy is to move most of the logics to
> user-space, I don't think that overloading iptables with yet another
> logging facility is the way to go.

The extra software might be critical on embedded devices, home 
wifi/ethernet routers.

> I think that Patrick is not going to like the idea of adding more hooks,
> what do you think Patrick?

Yes, the additional hook is suboptimal. But I couldn't find any other 
way to get the data.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Jan Engelhardt]

On Monday 2009-04-20 12:12, Jozsef Kadlecsik wrote:
>
>But ulogd2 requires the ULOG target

What purpose would NFLOG serve then?

>and ULOG cannot log the SNAT-ed address either, similarly to the
>vanilla LOG target: there's presently no hook at which the
>information is available for the LOG/ULOG targets.

I favor the patch because the raw table gets a POSTROUTING hook ;-)

>> I think that Patrick is not going to like the idea of adding more hooks,
>> what do you think Patrick?
>
>Yes, the additional hook is suboptimal. But I couldn't find any other 
>way to get the data.

The rawpost_table, as in the RAWNAT branch of Xtables-addons. While
the additional table is probably just as suboptimal, at least you
can unload it.

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Jozsef Kadlecsik]

On Mon, 20 Apr 2009, Jan Engelhardt wrote:

> On Monday 2009-04-20 12:12, Jozsef Kadlecsik wrote:
> >
> >But ulogd2 requires the ULOG target
> 
> What purpose would NFLOG serve then?

Thanks for the correction. Still, whatever log target we consider 
(LOG/ULOG/NFLOG), currently there's no hook point at which a rule could be 
added to log the original and (src) NAT-ed IP addresses.
 
> >> I think that Patrick is not going to like the idea of adding more hooks,
> >> what do you think Patrick?
> >
> >Yes, the additional hook is suboptimal. But I couldn't find any other 
> >way to get the data.
> 
> The rawpost_table, as in the RAWNAT branch of Xtables-addons. While
> the additional table is probably just as suboptimal, at least you
> can unload it.

Increasing the number of the tables? I'd better add a module parameter to 
define the really used hook entry points for the tables (raw, mangle and 
maybe nat).
 
Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Pablo Neira Ayuso]

Jozsef Kadlecsik wrote:
> Hi Pablo,
> 
> On Mon, 20 Apr 2009, Pablo Neira Ayuso wrote:
> 
>>> Attached you can find patches for netfilter and iptabes to support the 
>>> logging of the original and NAT-ed IP addresses together.
>>>
>>> Currently there's no way to do it by netfilter/iptables. If we log in the 
>>> filter table, there we can record the original src IP address only. 
>>> However, we cannot log the src IP address after NAT at all: SNAT happens 
>>> in the nat table at POSTROUTING, and there's no other table to which the 
>>> logging rule could be added (and the NAT targets return ACCEPT, so we 
>>> cannot add the loggin rule to the nat table either).
>>>
>>> The only way to log src/dst IP before/after NAT presently is to run 
>>> 'conntrack' in event mode like this:
>>>
>>> 	conntrack -E -e NEW | logger -p kernel.info
>> We can also do this by means of ulogd2 or, alternatively, conntrackd in
>> its very basic statistics mode.
> 
> But ulogd2 requires the ULOG target and ULOG cannot log the SNAT-ed 
> address either, similarly to the vanilla LOG target: there's presently no 
> hook at which the information is available for the LOG/ULOG targets.

I wasn't refering to any iptables target. New ulogd2 includes support
for ctnetlink, which can do this. I know, that means the extra libraries
dependencies.

>>> which is a little bit cumbersome and requires the libnfnetlink, 
>>> libnetfilter_conntrack libraries too.
>> Yes, this needs some extra user-space software. With regards to logging,
>> I think that the current policy is to move most of the logics to
>> user-space, I don't think that overloading iptables with yet another
>> logging facility is the way to go.
> 
> The extra software might be critical on embedded devices, home 
> wifi/ethernet routers.

Yes, that's true.

>> I think that Patrick is not going to like the idea of adding more hooks,
>> what do you think Patrick?
> 
> Yes, the additional hook is suboptimal. But I couldn't find any other 
> way to get the data.

Keeping this in user-space allows to switch on/off the feature without
any suboptimal impact for users that don't want to use this.

-- 
"Los honestos son inadaptados sociales" -- Les Luthiers

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Jozsef Kadlecsik]

Hi Pablo,

On Mon, 20 Apr 2009, Pablo Neira Ayuso wrote:

> Jozsef Kadlecsik wrote:
> > 
> > On Mon, 20 Apr 2009, Pablo Neira Ayuso wrote:
> > 
> >>> Attached you can find patches for netfilter and iptabes to support the 
> >>> logging of the original and NAT-ed IP addresses together.
> >>>
> >>> Currently there's no way to do it by netfilter/iptables. If we log in the 
> >>> filter table, there we can record the original src IP address only. 
> >>> However, we cannot log the src IP address after NAT at all: SNAT happens 
> >>> in the nat table at POSTROUTING, and there's no other table to which the 
> >>> logging rule could be added (and the NAT targets return ACCEPT, so we 
> >>> cannot add the loggin rule to the nat table either).
> >>>
> >>> The only way to log src/dst IP before/after NAT presently is to run 
> >>> 'conntrack' in event mode like this:
> >>>
> >>> 	conntrack -E -e NEW | logger -p kernel.info
> >> We can also do this by means of ulogd2 or, alternatively, conntrackd in
> >> its very basic statistics mode.
> > 
> > But ulogd2 requires the ULOG target and ULOG cannot log the SNAT-ed 
> > address either, similarly to the vanilla LOG target: there's presently no 
> > hook at which the information is available for the LOG/ULOG targets.
> 
> I wasn't refering to any iptables target. New ulogd2 includes support
> for ctnetlink, which can do this. I know, that means the extra libraries
> dependencies.

I see. Thanks the info, good to know that ulogd2 is capable of this.
(Calling 'conntrack' for logging looked really ugly. :-)

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Patrick McHardy]

Jozsef Kadlecsik wrote:
> On Mon, 20 Apr 2009, Pablo Neira Ayuso wrote:
>> I wasn't refering to any iptables target. New ulogd2 includes support
>> for ctnetlink, which can do this. I know, that means the extra libraries
>> dependencies.
> 
> I see. Thanks the info, good to know that ulogd2 is capable of this.
> (Calling 'conntrack' for logging looked really ugly. :-)

In the kernel, we could log the information from the conntrack
entry, if any. That would allow to log the manips after they
have been set up.

Would Pablo's suggestion or the conntrack method work for you?

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Jozsef Kadlecsik]

On Mon, 20 Apr 2009, Patrick McHardy wrote:

> Jozsef Kadlecsik wrote:
> > On Mon, 20 Apr 2009, Pablo Neira Ayuso wrote:
> > > I wasn't refering to any iptables target. New ulogd2 includes support
> > > for ctnetlink, which can do this. I know, that means the extra libraries
> > > dependencies.
> > 
> > I see. Thanks the info, good to know that ulogd2 is capable of this.
> > (Calling 'conntrack' for logging looked really ugly. :-)
> 
> In the kernel, we could log the information from the conntrack
> entry, if any. That would allow to log the manips after they
> have been set up.

Yes, but I'd not want an unconditional logging.
 
> Would Pablo's suggestion or the conntrack method work for you?

Oh, it's not for me at all: at a workshop I was asked how to log the info 
(hint: conflicker ;-) and embarrased enough I had to admit there was no 
easy way. That's why I put together the patch, with all it's questionable 
details.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Patrick McHardy]

Jozsef Kadlecsik wrote:
> On Mon, 20 Apr 2009, Patrick McHardy wrote:
> 
>> Jozsef Kadlecsik wrote:
>>> On Mon, 20 Apr 2009, Pablo Neira Ayuso wrote:
>>>> I wasn't refering to any iptables target. New ulogd2 includes support
>>>> for ctnetlink, which can do this. I know, that means the extra libraries
>>>> dependencies.
>>> I see. Thanks the info, good to know that ulogd2 is capable of this.
>>> (Calling 'conntrack' for logging looked really ugly. :-)
>> In the kernel, we could log the information from the conntrack
>> entry, if any. That would allow to log the manips after they
>> have been set up.
> 
> Yes, but I'd not want an unconditional logging.

I missed the point, you're already doing what I had in mind (use
ct->tuplehash->...) and the new hook is needed to even get a chance
to log the packet after SNAT.

>> Would Pablo's suggestion or the conntrack method work for you?
> 
> Oh, it's not for me at all: at a workshop I was asked how to log the info 
> (hint: conflicker ;-) and embarrased enough I had to admit there was no 
> easy way. That's why I put together the patch, with all it's questionable 
> details.

I can see that it has some informational value, but for things like
locating infected hosts, why not simply look at the traffic before
it is NATed? I currently can't come up with a real use case for
this ...

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Jozsef Kadlecsik]

Hi Patrick,

On Tue, 21 Apr 2009, Patrick McHardy wrote:

> Jozsef Kadlecsik wrote:
>
> > On Mon, 20 Apr 2009, Patrick McHardy wrote:
> >
> > > Jozsef Kadlecsik wrote:
> > Oh, it's not for me at all: at a workshop I was asked how to log the info
> > (hint: conflicker ;-) and embarrased enough I had to admit there was no easy
> > way. That's why I put together the patch, with all it's questionable
> > details.
> 
> I can see that it has some informational value, but for things like
> locating infected hosts, why not simply look at the traffic before
> it is NATed? I currently can't come up with a real use case for
> this ...

The well-maintained sites manage to locate on-site infected machines by 
using IDS, traffic-analysis, etc. However the typical user case for small 
sites is that the ISP or other offsite host report an abuse: "your machine 
x.y.z.w attacked machine a.b.c.d/added to the RBL because of spamming/etc" 
- and the machine x.y.z.w is of course their firewall. And they have got 
nothing which could help them to pick the real machine from the NATed 
network behind the firewall. And the small sites is the main reason why 
I'd favor an "out of the box" solution, which does not rely on anything 
besides netfilter/iptables.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Patrick McHardy]

Jozsef Kadlecsik wrote:
> On Tue, 21 Apr 2009, Patrick McHardy wrote:
> 
>> I can see that it has some informational value, but for things like
>> locating infected hosts, why not simply look at the traffic before
>> it is NATed? I currently can't come up with a real use case for
>> this ...
> 
> The well-maintained sites manage to locate on-site infected machines by 
> using IDS, traffic-analysis, etc. However the typical user case for small 
> sites is that the ISP or other offsite host report an abuse: "your machine 
> x.y.z.w attacked machine a.b.c.d/added to the RBL because of spamming/etc" 
> - and the machine x.y.z.w is of course their firewall. And they have got 
> nothing which could help them to pick the real machine from the NATed 
> network behind the firewall. And the small sites is the main reason why 
> I'd favor an "out of the box" solution, which does not rely on anything 
> besides netfilter/iptables.

Yes, but at that point, you can look at your logfiles and see who
connected to a.b.c.d and you'll see the internal address. Logging
the NATed address is not really helping since you need the internal
address anyways, so it seems natural to look at the unNATed information.
What am I missing here?

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Jozsef Kadlecsik]

On Wed, 22 Apr 2009, Patrick McHardy wrote:

> Jozsef Kadlecsik wrote:
> > On Tue, 21 Apr 2009, Patrick McHardy wrote:
> > 
> > > I can see that it has some informational value, but for things like
> > > locating infected hosts, why not simply look at the traffic before
> > > it is NATed? I currently can't come up with a real use case for
> > > this ...
> > 
> > The well-maintained sites manage to locate on-site infected machines by
> > using IDS, traffic-analysis, etc. However the typical user case for small
> > sites is that the ISP or other offsite host report an abuse: "your machine
> > x.y.z.w attacked machine a.b.c.d/added to the RBL because of spamming/etc" -
> > and the machine x.y.z.w is of course their firewall. And they have got
> > nothing which could help them to pick the real machine from the NATed
> > network behind the firewall. And the small sites is the main reason why I'd
> > favor an "out of the box" solution, which does not rely on anything besides
> > netfilter/iptables.
> 
> Yes, but at that point, you can look at your logfiles and see who
> connected to a.b.c.d and you'll see the internal address. Logging
> the NATed address is not really helping since you need the internal
> address anyways, so it seems natural to look at the unNATed information.
> What am I missing here?

Hmm, if there are multiple clients connecting to the same external node 
(or even just a few ones but the exact time is not definite due to the 
unsynced clocks) then the hunted client does not stand out. But it might 
indicate multiple infected machines as well...

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Patrick McHardy]

Jozsef Kadlecsik wrote:
> On Wed, 22 Apr 2009, Patrick McHardy wrote:
> 
>>> The well-maintained sites manage to locate on-site infected machines by
>>> using IDS, traffic-analysis, etc. However the typical user case for small
>>> sites is that the ISP or other offsite host report an abuse: "your machine
>>> x.y.z.w attacked machine a.b.c.d/added to the RBL because of spamming/etc" -
>>> and the machine x.y.z.w is of course their firewall. And they have got
>>> nothing which could help them to pick the real machine from the NATed
>>> network behind the firewall. And the small sites is the main reason why I'd
>>> favor an "out of the box" solution, which does not rely on anything besides
>>> netfilter/iptables.
>> Yes, but at that point, you can look at your logfiles and see who
>> connected to a.b.c.d and you'll see the internal address. Logging
>> the NATed address is not really helping since you need the internal
>> address anyways, so it seems natural to look at the unNATed information.
>> What am I missing here?
> 
> Hmm, if there are multiple clients connecting to the same external node 
> (or even just a few ones but the exact time is not definite due to the 
> unsynced clocks) then the hunted client does not stand out. But it might 
> indicate multiple infected machines as well...

Yes, you'd need at least destination port numbers to reduce false
positives. But the same is also true if you do log the NATed address.
For the small-site case you describe, the external address is usually
know anyways since there is only a single one :)

Just to be clear about this, I don't generally object to this,
but before making this (quite undesirable IMO) change, I'd like
to be sure that it actually provides a tantamount benefit.
Please feel free to tell me that I'm missing the point :)

Re: [PATCH] Add support to log original and NAT-ed IP addresses

[Jozsef Kadlecsik]

Hi Patrick,

On Thu, 23 Apr 2009, Patrick McHardy wrote:

> > Hmm, if there are multiple clients connecting to the same external node (or
> > even just a few ones but the exact time is not definite due to the unsynced
> > clocks) then the hunted client does not stand out. But it might indicate
> > multiple infected machines as well...
> 
> Yes, you'd need at least destination port numbers to reduce false
> positives. But the same is also true if you do log the NATed address.
> For the small-site case you describe, the external address is usually
> know anyways since there is only a single one :)
> 
> Just to be clear about this, I don't generally object to this,
> but before making this (quite undesirable IMO) change, I'd like
> to be sure that it actually provides a tantamount benefit.
> Please feel free to tell me that I'm missing the point :)

Because there's other solutions to log the data (ulogd2, conntrack)
and because the patch is suboptimal, let's forget about it. Maybe we'll be 
able to figure out a better way in nftables.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary