[RFC] Patches to allow updates of timeouts in expectations

[RFC] Patches to allow updates of timeouts in expectations

[Kelvie Wong]

Hello all,

These are a couple of patches that I will be using in order to solve a problem
I am having with (permanent) expectation timeouts not being able to be updated.

Initial testing tells me that these work; when I call NFCT_Q_CREATE_UPDATE with
an existing permanent expectation, it updates it correctly with the new timeout
value. I use nfexp_query directly (rather than through conntrack-tools).

Anyway, this is something that was useful to me, so I'm not sure if you guys
would like these patches upstream but also, am I doing this correctly 
(especially with respect to locking), or is there any other thing I may have
overlooked?

Thanks,
Kelvie

[PATCH] expect: support NFCT_Q_CREATE_UPDATE in nfexp_query

[Kelvie Wong]

This will work as it does in conntrack; it won't pass NLM_F_ACK into
ctnetlink_new_expect in the kernel, and will thus invoke
ctnetlink_change_expect if the expectation already exists.

Signed-off-by: Kelvie Wong <kelvie@ieee.org>
---
 src/expect/api.c |    3 +++
 1 file changed, 3 insertions(+)

diff --git a/src/expect/api.c b/src/expect/api.c
index 4da44a0..a101042 100644
--- a/src/expect/api.c
+++ b/src/expect/api.c
@@ -533,6 +533,9 @@ __build_query_exp(struct nfnl_subsys_handle *ssh,
 	case NFCT_Q_CREATE:
 		__build_expect(ssh, req, size, IPCTNL_MSG_EXP_NEW, NLM_F_REQUEST|NLM_F_CREATE|NLM_F_ACK|NLM_F_EXCL, data);
 		break;
+	case NFCT_Q_CREATE_UPDATE:
+		__build_expect(ssh, req, size, IPCTNL_MSG_EXP_NEW, NLM_F_REQUEST|NLM_F_CREATE|NLM_F_ACK, data);
+		break;
 	case NFCT_Q_GET:
 		__build_expect(ssh, req, size, IPCTNL_MSG_EXP_GET, NLM_F_REQUEST|NLM_F_ACK, data);
 		break;
-- 
1.7.9.5

[PATCH] netfilter: nf_ct_expect: partially implement ctnetlink_change_expect

[Kelvie Wong]

This refreshes the "timeout" attribute in existing expectations if one is
given.

The use case for this would be for userspace helpers to extend the lifetime
of the expectation when requested, as this is not possible right now
without deleting/recreating the expectation.

I use this specifically for forwarding DCERPC traffic through:

DCERPC has a port mapper daemon that chooses a (seemingly) random port for
future traffic to go to. We expect this traffic (with a reasonable
timeout), but sometimes the port mapper will tell the client to continue
using the same port. This allows us to extend the expectation accordingly.

Signed-off-by: Kelvie Wong <kelvie@ieee.org>
---
 net/netfilter/nf_conntrack_netlink.c |   10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
index ca7e835..87a9682 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -2065,6 +2065,16 @@ static int
 ctnetlink_change_expect(struct nf_conntrack_expect *x,
 			const struct nlattr * const cda[])
 {
+	/* Refresh the timeout */
+	if (cda[CTA_EXPECT_TIMEOUT]) {
+		if (!del_timer(&x->timeout))
+			return -ETIME;
+
+		x->timeout.expires = jiffies +
+			ntohl(nla_get_be32(cda[CTA_EXPECT_TIMEOUT])) * HZ;
+		add_timer(&x->timeout);
+		return 0;
+	}
 	return -EOPNOTSUPP;
 }
 
-- 
1.7.9.5

Re: [PATCH] netfilter: nf_ct_expect: partially implement ctnetlink_change_expect

[Pablo Neira Ayuso]

Hi Kelvie,

On Wed, May 02, 2012 at 05:39:24PM -0700, Kelvie Wong wrote:
> This refreshes the "timeout" attribute in existing expectations if one is
> given.
> 
> The use case for this would be for userspace helpers to extend the lifetime
> of the expectation when requested, as this is not possible right now
> without deleting/recreating the expectation.
> 
> I use this specifically for forwarding DCERPC traffic through:
> 
> DCERPC has a port mapper daemon that chooses a (seemingly) random port for
> future traffic to go to. We expect this traffic (with a reasonable
> timeout), but sometimes the port mapper will tell the client to continue
> using the same port. This allows us to extend the expectation accordingly.
> 
> Signed-off-by: Kelvie Wong <kelvie@ieee.org>
> ---
>  net/netfilter/nf_conntrack_netlink.c |   10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
> index ca7e835..87a9682 100644
> --- a/net/netfilter/nf_conntrack_netlink.c
> +++ b/net/netfilter/nf_conntrack_netlink.c
> @@ -2065,6 +2065,16 @@ static int
>  ctnetlink_change_expect(struct nf_conntrack_expect *x,
>  			const struct nlattr * const cda[])
>  {
> +	/* Refresh the timeout */
> +	if (cda[CTA_EXPECT_TIMEOUT]) {
> +		if (!del_timer(&x->timeout))
> +			return -ETIME;
> +
> +		x->timeout.expires = jiffies +
> +			ntohl(nla_get_be32(cda[CTA_EXPECT_TIMEOUT])) * HZ;
> +		add_timer(&x->timeout);
> +		return 0;
> +	}

You have to protect this with nf_conntrack_lock spinlock. See
net/netfilter/nf_conntrack_expect.c for expectation handling.

>  	return -EOPNOTSUPP;

Now that we support expectation changing, this should be return 0.

We have two choices for this:

a) rework the patch with the suggestion that I made.
b) add some NF_CT_EXPECT_FIXED_TIMEOUT flag like we have in the
   connection tracking. Thus, the expectation will not ever expire.

I'd need to know more about how you're using this. Depending on that,
we can select a) or b).

BTW, I'm working on finishing some user-space framework for developing
helper in user-space. My question is: would you be interested in
integrating your DCERPC helper into it?

I expect to post some code soon, still working on it.

Re: [PATCH] netfilter: nf_ct_expect: partially implement ctnetlink_change_expect

[Kelvie Wong]

Hey Pablo,

On Sun, May 6, 2012 at 4:09 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> You have to protect this with nf_conntrack_lock spinlock. See
> net/netfilter/nf_conntrack_expect.c for expectation handling.

ctnetlink_change_expect is not exported, and it is only called in
ctnetlink_new_expect, which is protected by the spinlock.

>
>>       return -EOPNOTSUPP;
>
> Now that we support expectation changing, this should be return 0.

I can make this change.

> We have two choices for this:
>
> a) rework the patch with the suggestion that I made.
> b) add some NF_CT_EXPECT_FIXED_TIMEOUT flag like we have in the
>   connection tracking. Thus, the expectation will not ever expire.
>
> I'd need to know more about how you're using this. Depending on that,
> we can select a) or b).

I think we need to do a). A fixed timeout won't work, as in some cases we
need to extend the expectation (the server has asked to use the same port
again, so we need to give it another 10 minutes, possibly indefinitely),
whereas in other cases we can just safely let the expectation expire.

I want to avoid leaving the expectation forever, but I can't know until I see
the DCERPC traffic.

>
> BTW, I'm working on finishing some user-space framework for developing
> helper in user-space. My question is: would you be interested in
> integrating your DCERPC helper into it?
>
> I expect to post some code soon, still working on it.

I just need something to work right now (I'm going to use my original patch
as-is, unless I made a grave error somewhere), but maybe in the future if
it will ease maintenance.

Thanks for your help!
-- 
Kelvie Wong
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] netfilter: nf_ct_expect: partially implement ctnetlink_change_expect

[Kelvie Wong]

On Sun, May 6, 2012 at 6:51 PM, Kelvie Wong <kelvie@ieee.org> wrote:
> ctnetlink_change_expect is not exported, and it is only called in
> ctnetlink_new_expect, which is protected by the spinlock.
>

Also, in a similar function, ctnetlink_change_conntrack, the locking is
done by the caller, in a similar fashion.

-- 
Kelvie Wong

Re: [PATCH] netfilter: nf_ct_expect: partially implement ctnetlink_change_expect

[Pablo Neira Ayuso]

On Sun, May 06, 2012 at 06:51:45PM -0700, Kelvie Wong wrote:
> Hey Pablo,
> 
> On Sun, May 6, 2012 at 4:09 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > You have to protect this with nf_conntrack_lock spinlock. See
> > net/netfilter/nf_conntrack_expect.c for expectation handling.
> 
> ctnetlink_change_expect is not exported, and it is only called in
> ctnetlink_new_expect, which is protected by the spinlock.

You're right, I've overlooked this.

> >
> >>       return -EOPNOTSUPP;
> >
> > Now that we support expectation changing, this should be return 0.
> 
> I can make this change.
> 
> > We have two choices for this:
> >
> > a) rework the patch with the suggestion that I made.
> > b) add some NF_CT_EXPECT_FIXED_TIMEOUT flag like we have in the
> >   connection tracking. Thus, the expectation will not ever expire.
> >
> > I'd need to know more about how you're using this. Depending on that,
> > we can select a) or b).
> 
> I think we need to do a). A fixed timeout won't work, as in some cases we
> need to extend the expectation (the server has asked to use the same port
> again, so we need to give it another 10 minutes, possibly indefinitely),
> whereas in other cases we can just safely let the expectation expire.
> 
> I want to avoid leaving the expectation forever, but I can't know until I see
> the DCERPC traffic.

OK, then I'll take your patch. I'll mangle it to return 0 instead.

> > BTW, I'm working on finishing some user-space framework for developing
> > helper in user-space. My question is: would you be interested in
> > integrating your DCERPC helper into it?
> >
> > I expect to post some code soon, still working on it.
> 
> I just need something to work right now (I'm going to use my original patch
> as-is, unless I made a grave error somewhere), but maybe in the future if
> it will ease maintenance.

I guess it will ease maintainance, really.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] netfilter: nf_ct_expect: partially implement ctnetlink_change_expect

[Kelvie Wong]

On Mon, May 7, 2012 at 1:42 AM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> OK, then I'll take your patch. I'll mangle it to return 0 instead.

That will be fine.

Thank you,
-- 
Kelvie Wong

Re: [PATCH] netfilter: nf_ct_expect: partially implement ctnetlink_change_expect

[Kelvie Wong]

On Mon, May 07, 2012 at 09:28:58AM -0700, Kelvie Wong wrote:
> That will be fine.
> 
> Thank you,
> -- 
> Kelvie Wong

Here's the patch if you don't want to do the work :)

>From 2fa0ea4c54ad8b16e7978f8d41892f4d33b0db65 Mon Sep 17 00:00:00 2001
From: Kelvie Wong <kelvie@ieee.org>
Date: Mon, 7 May 2012 09:30:55 -0700
Subject: [PATCH] netfilter: nf_ct_expect: partially implement
 ctnetlink_change_expect

This refreshes the "timeout" attribute in existing expectations if one is
given.

The use case for this would be for userspace helpers to extend the lifetime
of the expectation when requested, as this is not possible right now
without deleting/recreating the expectation.

I use this specifically for forwarding DCERPC traffic through:

DCERPC has a port mapper daemon that chooses a (seemingly) random port for
future traffic to go to. We expect this traffic (with a reasonable
timeout), but sometimes the port mapper will tell the client to continue
using the same port. This allows us to extend the expectation accordingly.

Signed-off-by: Kelvie Wong <kelvie@ieee.org>
---
 net/netfilter/nf_conntrack_netlink.c |   11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/net/netfilter/nf_conntrack_netlink.c b/net/netfilter/nf_conntrack_netlink.c
index ca7e835..57518fc 100644
--- a/net/netfilter/nf_conntrack_netlink.c
+++ b/net/netfilter/nf_conntrack_netlink.c
@@ -2065,7 +2065,16 @@ static int
 ctnetlink_change_expect(struct nf_conntrack_expect *x,
 			const struct nlattr * const cda[])
 {
-	return -EOPNOTSUPP;
+	/* Refresh the timeout */
+	if (cda[CTA_EXPECT_TIMEOUT]) {
+		if (!del_timer(&x->timeout))
+			return -ETIME;
+
+		x->timeout.expires = jiffies +
+			ntohl(nla_get_be32(cda[CTA_EXPECT_TIMEOUT])) * HZ;
+		add_timer(&x->timeout);
+	}
+	return 0;
 }
 
 static const struct nla_policy exp_nat_nla_policy[CTA_EXPECT_NAT_MAX+1] = {
-- 
1.7.9.5

-- 
Kelvie Wong

Re: [PATCH] netfilter: nf_ct_expect: partially implement ctnetlink_change_expect

[Pablo Neira Ayuso]

On Mon, May 07, 2012 at 09:43:24AM -0700, Kelvie Wong wrote:
> On Mon, May 07, 2012 at 09:28:58AM -0700, Kelvie Wong wrote:
> > That will be fine.
> > 
> > Thank you,
> > -- 
> > Kelvie Wong
> 
> Here's the patch if you don't want to do the work :)
> 
> From 2fa0ea4c54ad8b16e7978f8d41892f4d33b0db65 Mon Sep 17 00:00:00 2001
> From: Kelvie Wong <kelvie@ieee.org>
> Date: Mon, 7 May 2012 09:30:55 -0700
> Subject: [PATCH] netfilter: nf_ct_expect: partially implement
>  ctnetlink_change_expect
> 
> This refreshes the "timeout" attribute in existing expectations if one is
> given.
> 
> The use case for this would be for userspace helpers to extend the lifetime
> of the expectation when requested, as this is not possible right now
> without deleting/recreating the expectation.
> 
> I use this specifically for forwarding DCERPC traffic through:
> 
> DCERPC has a port mapper daemon that chooses a (seemingly) random port for
> future traffic to go to. We expect this traffic (with a reasonable
> timeout), but sometimes the port mapper will tell the client to continue
> using the same port. This allows us to extend the expectation accordingly.
> 
> Signed-off-by: Kelvie Wong <kelvie@ieee.org>

Applied, thanks Kelvie.