[RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Stephen Smalley]

Since 4.14-rc1, the selinux-testsuite has been encountering sporadic
failures during testing of labeled IPSEC. git bisect pointed to
commit ec30d78c14a813db39a647b6a348b4286 ("xfrm: add xdst pcpu cache").
The xdst pcpu cache is only checking that the policies are the same,
but does not validate that the policy, state, and flow match with respect
to security context labeling.  As a result, the wrong SA could be used
and the receiver could end up performing permission checking and
providing SO_PEERSEC or SCM_SECURITY values for the wrong security context.
security_xfrm_state_pol_flow_match() exists for this purpose and is
already called from xfrm_state_look_at() for matching purposes.
Further, xfrm_state_look_at() also performs a xfrm_selector_match() test,
which is also missing from the xdst pcpu cache logic.  Add calls to both
of these functions when validating the cache entry.  With these changes,
the selinux-testsuite passes all tests again.

Fixes: ec30d78c14a813db39a647b6a348b4286ba4abf5 ("xfrm: add xdst pcpu cache")
Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
---
This is an RFC because I am not entirely confident in the fix, e.g. is it
sufficient to perform this matching only on the first xfrm or do they all
need to be walked as in xfrm_bundle_ok()?  Also, should we perform this
matching before (as in this patch) or after calling xfrm_bundle_ok()? Also,
do we need to test xfrm->sel.family before calling xfrm_selector_match
(as in this patch) or not - xfrm_state_look_at() does so when the
state is XFRM_STATE_VALID but not when it is _ERROR or _EXPIRED?

 net/xfrm/xfrm_policy.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
index 2746b62..171818b 100644
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -1820,6 +1820,11 @@ xfrm_resolve_and_create_bundle(struct xfrm_policy **pols, int num_pols,
 	    !xfrm_pol_dead(xdst) &&
 	    memcmp(xdst->pols, pols,
 		   sizeof(struct xfrm_policy *) * num_pols) == 0 &&
+	    (!xdst->u.dst.xfrm->sel.family ||
+	     xfrm_selector_match(&xdst->u.dst.xfrm->sel, fl,
+				 xdst->u.dst.xfrm->sel.family)) &&
+	    security_xfrm_state_pol_flow_match(xdst->u.dst.xfrm,
+					       xdst->pols[0], fl) &&
 	    xfrm_bundle_ok(xdst)) {
 		dst_hold(&xdst->u.dst);
 		return xdst;
-- 
2.9.5

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Florian Westphal]

Stephen Smalley <sds@tycho.nsa.gov> wrote:
> Since 4.14-rc1, the selinux-testsuite has been encountering sporadic
> failures during testing of labeled IPSEC. git bisect pointed to
> commit ec30d78c14a813db39a647b6a348b4286 ("xfrm: add xdst pcpu cache").
> The xdst pcpu cache is only checking that the policies are the same,
> but does not validate that the policy, state, and flow match with respect
> to security context labeling.  As a result, the wrong SA could be used
> and the receiver could end up performing permission checking and
> providing SO_PEERSEC or SCM_SECURITY values for the wrong security context.
> security_xfrm_state_pol_flow_match() exists for this purpose and is
> already called from xfrm_state_look_at() for matching purposes.
> Further, xfrm_state_look_at() also performs a xfrm_selector_match() test,
> which is also missing from the xdst pcpu cache logic.  Add calls to both
> of these functions when validating the cache entry.  With these changes,
> the selinux-testsuite passes all tests again.
> 
> Fixes: ec30d78c14a813db39a647b6a348b4286ba4abf5 ("xfrm: add xdst pcpu cache")
> Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
> ---
> This is an RFC because I am not entirely confident in the fix, e.g. is it
> sufficient to perform this matching only on the first xfrm or do they all
> need to be walked as in xfrm_bundle_ok()?  Also, should we perform this
> matching before (as in this patch) or after calling xfrm_bundle_ok()? Also,
> do we need to test xfrm->sel.family before calling xfrm_selector_match
> (as in this patch) or not - xfrm_state_look_at() does so when the
> state is XFRM_STATE_VALID but not when it is _ERROR or _EXPIRED?

No idea.

I looked at the old flow cache but i don't see any of these extra
checks there either.

However, old flow cache stored flowi struct as key, and that contains a
flowi_secid,  populated by the decode_session hooks.

Was it enough to check for identical flowi_secid in the flowi structs to
avoid this problem or am i missing something?

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Stephen Smalley]

On Tue, 2017-10-31 at 12:11 +0100, Florian Westphal wrote:
> Stephen Smalley <sds-+05T5uksL2qpZYMLLGbcSA@public.gmane.org> wrote:
> > Since 4.14-rc1, the selinux-testsuite has been encountering
> > sporadic
> > failures during testing of labeled IPSEC. git bisect pointed to
> > commit ec30d78c14a813db39a647b6a348b4286 ("xfrm: add xdst pcpu
> > cache").
> > The xdst pcpu cache is only checking that the policies are the
> > same,
> > but does not validate that the policy, state, and flow match with
> > respect
> > to security context labeling.  As a result, the wrong SA could be
> > used
> > and the receiver could end up performing permission checking and
> > providing SO_PEERSEC or SCM_SECURITY values for the wrong security
> > context.
> > security_xfrm_state_pol_flow_match() exists for this purpose and is
> > already called from xfrm_state_look_at() for matching purposes.
> > Further, xfrm_state_look_at() also performs a xfrm_selector_match()
> > test,
> > which is also missing from the xdst pcpu cache logic.  Add calls to
> > both
> > of these functions when validating the cache entry.  With these
> > changes,
> > the selinux-testsuite passes all tests again.
> > 
> > Fixes: ec30d78c14a813db39a647b6a348b4286ba4abf5 ("xfrm: add xdst
> > pcpu cache")
> > Signed-off-by: Stephen Smalley <sds-+05T5uksL2qpZYMLLGbcSA@public.gmane.org>
> > ---
> > This is an RFC because I am not entirely confident in the fix, e.g.
> > is it
> > sufficient to perform this matching only on the first xfrm or do
> > they all
> > need to be walked as in xfrm_bundle_ok()?  Also, should we perform
> > this
> > matching before (as in this patch) or after calling
> > xfrm_bundle_ok()? Also,
> > do we need to test xfrm->sel.family before calling
> > xfrm_selector_match
> > (as in this patch) or not - xfrm_state_look_at() does so when the
> > state is XFRM_STATE_VALID but not when it is _ERROR or _EXPIRED?
> 
> No idea.
> 
> I looked at the old flow cache but i don't see any of these extra
> checks there either.
> 
> However, old flow cache stored flowi struct as key, and that contains
> a
> flowi_secid,  populated by the decode_session hooks.
> 
> Was it enough to check for identical flowi_secid in the flowi structs
> to
> avoid this problem or am i missing something?

I'm not sure, but security_xfrm_state_pol_flow_match() ->
selinux_xfrm_state_pol_flow_match() does more than just compare flow
secids.

Also, there is the separate issue of the missing xfrm_selector_match()
call, which can also cause the wrong SA to be used independent of
anything LSM/SELinux-related.

It is a regression; the correct SA was being used prior to the xdst
pcpu cache commit.  Reproducible using the selinux-testsuite, most
easily run on a Fedora VM,
git clone https://github.com/SELinuxProject/selinux-testsuite/
sudo dnf install perl-Test perl-Test-Harness perl-Test-Simple selinux-policy-devel gcc libselinux-devel net-tools netlabel_tools iptables
sudo make -C policy load
cd tests/inet_socket
while sudo ./test; do : ; done

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Stephen Smalley]

On Tue, 2017-10-31 at 09:43 -0400, Stephen Smalley wrote:
> On Tue, 2017-10-31 at 12:11 +0100, Florian Westphal wrote:
> > Stephen Smalley <sds@tycho.nsa.gov> wrote:
> > > Since 4.14-rc1, the selinux-testsuite has been encountering
> > > sporadic
> > > failures during testing of labeled IPSEC. git bisect pointed to
> > > commit ec30d78c14a813db39a647b6a348b4286 ("xfrm: add xdst pcpu
> > > cache").
> > > The xdst pcpu cache is only checking that the policies are the
> > > same,
> > > but does not validate that the policy, state, and flow match with
> > > respect
> > > to security context labeling.  As a result, the wrong SA could be
> > > used
> > > and the receiver could end up performing permission checking and
> > > providing SO_PEERSEC or SCM_SECURITY values for the wrong
> > > security
> > > context.
> > > security_xfrm_state_pol_flow_match() exists for this purpose and
> > > is
> > > already called from xfrm_state_look_at() for matching purposes.
> > > Further, xfrm_state_look_at() also performs a
> > > xfrm_selector_match()
> > > test,
> > > which is also missing from the xdst pcpu cache logic.  Add calls
> > > to
> > > both
> > > of these functions when validating the cache entry.  With these
> > > changes,
> > > the selinux-testsuite passes all tests again.
> > > 
> > > Fixes: ec30d78c14a813db39a647b6a348b4286ba4abf5 ("xfrm: add xdst
> > > pcpu cache")
> > > Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>
> > > ---
> > > This is an RFC because I am not entirely confident in the fix,
> > > e.g.
> > > is it
> > > sufficient to perform this matching only on the first xfrm or do
> > > they all
> > > need to be walked as in xfrm_bundle_ok()?  Also, should we
> > > perform
> > > this
> > > matching before (as in this patch) or after calling
> > > xfrm_bundle_ok()? Also,
> > > do we need to test xfrm->sel.family before calling
> > > xfrm_selector_match
> > > (as in this patch) or not - xfrm_state_look_at() does so when the
> > > state is XFRM_STATE_VALID but not when it is _ERROR or _EXPIRED?
> > 
> > No idea.
> > 
> > I looked at the old flow cache but i don't see any of these extra
> > checks there either.
> > 
> > However, old flow cache stored flowi struct as key, and that
> > contains
> > a
> > flowi_secid,  populated by the decode_session hooks.
> > 
> > Was it enough to check for identical flowi_secid in the flowi
> > structs
> > to
> > avoid this problem or am i missing something?
> 
> I'm not sure, but security_xfrm_state_pol_flow_match() ->
> selinux_xfrm_state_pol_flow_match() does more than just compare flow
> secids.
> 
> Also, there is the separate issue of the missing
> xfrm_selector_match()
> call, which can also cause the wrong SA to be used independent of
> anything LSM/SELinux-related.
> 
> It is a regression; the correct SA was being used prior to the xdst
> pcpu cache commit.  Reproducible using the selinux-testsuite, most
> easily run on a Fedora VM,
> git clone https://github.com/SELinuxProject/selinux-testsuite/
> sudo dnf install perl-Test perl-Test-Harness perl-Test-Simple
> selinux-policy-devel gcc libselinux-devel net-tools netlabel_tools
> iptables

Actually, you should just run 'sudo make test' instead of the
individual commands below.  I was breaking out the individual commands
to avoid running the rest of the testsuite unrelated to networking, but
 that won't pick up all of the dependencies the first time.  Sorry.

> sudo make -C policy load
> cd tests/inet_socket
> while sudo ./test; do : ; done

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Florian Westphal]

Stephen Smalley <sds@tycho.nsa.gov> wrote:
> It is a regression; the correct SA was being used prior to the xdst
> pcpu cache commit.

I don't doubt that at all.  I would like to understand why the flow
cache did not have this problem.

> easily run on a Fedora VM,
> git clone https://github.com/SELinuxProject/selinux-testsuite/
> sudo dnf install perl-Test perl-Test-Harness perl-Test-Simple selinux-policy-devel gcc libselinux-devel net-tools netlabel_tools iptables
> sudo make -C policy load
> cd tests/inet_socket
> while sudo ./test; do : ; done

Thanks, I'll have a look.

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Paul Moore]

On Mon, Oct 30, 2017 at 10:58 AM, Stephen Smalley <sds@tycho.nsa.gov> wrote:
> Since 4.14-rc1, the selinux-testsuite has been encountering sporadic
> failures during testing of labeled IPSEC. git bisect pointed to
> commit ec30d78c14a813db39a647b6a348b4286 ("xfrm: add xdst pcpu cache").
> The xdst pcpu cache is only checking that the policies are the same,
> but does not validate that the policy, state, and flow match with respect
> to security context labeling.  As a result, the wrong SA could be used
> and the receiver could end up performing permission checking and
> providing SO_PEERSEC or SCM_SECURITY values for the wrong security context.
> security_xfrm_state_pol_flow_match() exists for this purpose and is
> already called from xfrm_state_look_at() for matching purposes.
> Further, xfrm_state_look_at() also performs a xfrm_selector_match() test,
> which is also missing from the xdst pcpu cache logic.  Add calls to both
> of these functions when validating the cache entry.  With these changes,
> the selinux-testsuite passes all tests again.
>
> Fixes: ec30d78c14a813db39a647b6a348b4286ba4abf5 ("xfrm: add xdst pcpu cache")
> Signed-off-by: Stephen Smalley <sds@tycho.nsa.gov>

Thanks for chasing this down while I was on vacation :)

> This is an RFC because I am not entirely confident in the fix, e.g. is it
> sufficient to perform this matching only on the first xfrm or do they all
> need to be walked as in xfrm_bundle_ok()?

If you look at how we handle outgoing labeled IPsec traffic, e.g.
selinux_xfrm_skb_sid_egress(), you'll see that we only check the first
xfrm because I don't believe it is ever possible for us to create a
xfrm bundle with mis-matching SELinux labels.

Inbound traffic is another story, we need to check the entire bundle.

> ... Also, should we perform this
> matching before (as in this patch) or after calling xfrm_bundle_ok()?

I would probably make the LSM call the last check, as you've done; but
I have to say that is just so it is consistent with the "LSM last"
philosophy and not because of any performance related argument.

> ... Also,
> do we need to test xfrm->sel.family before calling xfrm_selector_match
> (as in this patch) or not - xfrm_state_look_at() does so when the
> state is XFRM_STATE_VALID but not when it is _ERROR or _EXPIRED?

Speaking purely from a SELinux perspective, I'm not sure it matters:
as long as the labels match we are happy.  However, from a general
IPsec perspective it does seem like a reasonable thing.

Granted I'm probably missing something, but it seems a little odd that
the code isn't already checking that the selectors match (... what am
I missing?).  It does check the policies, maybe that is enough in the
normal IPsec case?

> diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> index 2746b62..171818b 100644
> --- a/net/xfrm/xfrm_policy.c
> +++ b/net/xfrm/xfrm_policy.c
> @@ -1820,6 +1820,11 @@ xfrm_resolve_and_create_bundle(struct xfrm_policy **pols, int num_pols,
>             !xfrm_pol_dead(xdst) &&
>             memcmp(xdst->pols, pols,
>                    sizeof(struct xfrm_policy *) * num_pols) == 0 &&
> +           (!xdst->u.dst.xfrm->sel.family ||
> +            xfrm_selector_match(&xdst->u.dst.xfrm->sel, fl,
> +                                xdst->u.dst.xfrm->sel.family)) &&
> +           security_xfrm_state_pol_flow_match(xdst->u.dst.xfrm,
> +                                              xdst->pols[0], fl) &&
>             xfrm_bundle_ok(xdst)) {
>                 dst_hold(&xdst->u.dst);
>                 return xdst;
> --
> 2.9.5
>

-- 
paul moore
www.paul-moore.com

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Florian Westphal]

Paul Moore <paul@paul-moore.com> wrote:
> On Mon, Oct 30, 2017 at 10:58 AM, Stephen Smalley <sds@tycho.nsa.gov> wrote:
> > matching before (as in this patch) or after calling xfrm_bundle_ok()?
> 
> I would probably make the LSM call the last check, as you've done; but
> I have to say that is just so it is consistent with the "LSM last"
> philosophy and not because of any performance related argument.
> 
> > ... Also,
> > do we need to test xfrm->sel.family before calling xfrm_selector_match
> > (as in this patch) or not - xfrm_state_look_at() does so when the
> > state is XFRM_STATE_VALID but not when it is _ERROR or _EXPIRED?
> 
> Speaking purely from a SELinux perspective, I'm not sure it matters:
> as long as the labels match we are happy.  However, from a general
> IPsec perspective it does seem like a reasonable thing.
> 
> Granted I'm probably missing something, but it seems a little odd that
> the code isn't already checking that the selectors match (... what am
> I missing?).  It does check the policies, maybe that is enough in the
> normal IPsec case?

The assumption was that identical policies would yield the same SAs,
but thats not correct.

> > diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> > index 2746b62..171818b 100644
> > --- a/net/xfrm/xfrm_policy.c
> > +++ b/net/xfrm/xfrm_policy.c
> > @@ -1820,6 +1820,11 @@ xfrm_resolve_and_create_bundle(struct xfrm_policy **pols, int num_pols,
> >             !xfrm_pol_dead(xdst) &&
> >             memcmp(xdst->pols, pols,
> >                    sizeof(struct xfrm_policy *) * num_pols) == 0 &&
> > +           (!xdst->u.dst.xfrm->sel.family ||
> > +            xfrm_selector_match(&xdst->u.dst.xfrm->sel, fl,
> > +                                xdst->u.dst.xfrm->sel.family)) &&
> > +           security_xfrm_state_pol_flow_match(xdst->u.dst.xfrm,
> > +                                              xdst->pols[0], fl) &&

... so this needs to walk the bundle and validate each selector.

Alternatively we could always do template resolution and then check
that all states found match those of the old pcpu xdst:

diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
--- a/net/xfrm/xfrm_policy.c
+++ b/net/xfrm/xfrm_policy.c
@@ -1786,19 +1786,23 @@ void xfrm_policy_cache_flush(void)
 	put_online_cpus();
 }
 
-static bool xfrm_pol_dead(struct xfrm_dst *xdst)
+static bool xfrm_xdst_can_reuse(struct xfrm_dst *xdst,
+				struct xfrm_state * const xfrm[],
+				int num)
 {
-	unsigned int num_pols = xdst->num_pols;
-	unsigned int pol_dead = 0, i;
+	const struct dst_entry *dst = &xdst->u.dst;
+	int i;
 
-	for (i = 0; i < num_pols; i++)
-		pol_dead |= xdst->pols[i]->walk.dead;
+	if (xdst->num_xfrms != num)
+		return false;
 
-	/* Mark DST_OBSOLETE_DEAD to fail the next xfrm_dst_check() */
-	if (pol_dead)
-		xdst->u.dst.obsolete = DST_OBSOLETE_DEAD;
+	for (i = 0; i < num; i++) {
+		if (!dst || dst->xfrm != xfrm[i])
+			return false;
+		dst = dst->child;
+	}
 
-	return pol_dead;
+	return xfrm_bundle_ok(xdst);
 }
 
 static struct xfrm_dst *
@@ -1812,26 +1816,28 @@ xfrm_resolve_and_create_bundle(struct xfrm_policy **pols, int num_pols,
 	struct dst_entry *dst;
 	int err;
 
+	/* Try to instantiate a bundle */
+	err = xfrm_tmpl_resolve(pols, num_pols, fl, xfrm, family);
+	if (err <= 0) {
+		if (err != 0 && err != -EAGAIN)
+			XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTPOLERROR);
+		return ERR_PTR(err);
+	}
+
 	xdst = this_cpu_read(xfrm_last_dst);
 	if (xdst &&
 	    xdst->u.dst.dev == dst_orig->dev &&
 	    xdst->num_pols == num_pols &&
-	    !xfrm_pol_dead(xdst) &&
 	    memcmp(xdst->pols, pols,
 		   sizeof(struct xfrm_policy *) * num_pols) == 0 &&
-	    xfrm_bundle_ok(xdst)) {
+	    xfrm_xdst_can_reuse(xdst, xfrm, err)) {
 		dst_hold(&xdst->u.dst);
+		while (err > 0)
+			xfrm_state_put(xfrm[--err]);
 		return xdst;
 	}
 
 	old = xdst;
-	/* Try to instantiate a bundle */
-	err = xfrm_tmpl_resolve(pols, num_pols, fl, xfrm, family);
-	if (err <= 0) {
-		if (err != 0 && err != -EAGAIN)
-			XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTPOLERROR);
-		return ERR_PTR(err);
-	}
 
 	dst = xfrm_bundle_create(pols[0], xfrm, err, fl, dst_orig);
 	if (IS_ERR(dst)) {
-- 
2.13.6

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Stephen Smalley]

On Wed, 2017-11-01 at 00:08 +0100, Florian Westphal wrote:
> Paul Moore <paul@paul-moore.com> wrote:
> > On Mon, Oct 30, 2017 at 10:58 AM, Stephen Smalley <sds@tycho.nsa.go
> > v> wrote:
> > > matching before (as in this patch) or after calling
> > > xfrm_bundle_ok()?
> > 
> > I would probably make the LSM call the last check, as you've done;
> > but
> > I have to say that is just so it is consistent with the "LSM last"
> > philosophy and not because of any performance related argument.
> > 
> > > ... Also,
> > > do we need to test xfrm->sel.family before calling
> > > xfrm_selector_match
> > > (as in this patch) or not - xfrm_state_look_at() does so when the
> > > state is XFRM_STATE_VALID but not when it is _ERROR or _EXPIRED?
> > 
> > Speaking purely from a SELinux perspective, I'm not sure it
> > matters:
> > as long as the labels match we are happy.  However, from a general
> > IPsec perspective it does seem like a reasonable thing.
> > 
> > Granted I'm probably missing something, but it seems a little odd
> > that
> > the code isn't already checking that the selectors match (... what
> > am
> > I missing?).  It does check the policies, maybe that is enough in
> > the
> > normal IPsec case?
> 
> The assumption was that identical policies would yield the same SAs,
> but thats not correct.
> 
> > > diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> > > index 2746b62..171818b 100644
> > > --- a/net/xfrm/xfrm_policy.c
> > > +++ b/net/xfrm/xfrm_policy.c
> > > @@ -1820,6 +1820,11 @@ xfrm_resolve_and_create_bundle(struct
> > > xfrm_policy **pols, int num_pols,
> > >             !xfrm_pol_dead(xdst) &&
> > >             memcmp(xdst->pols, pols,
> > >                    sizeof(struct xfrm_policy *) * num_pols) == 0
> > > &&
> > > +           (!xdst->u.dst.xfrm->sel.family ||
> > > +            xfrm_selector_match(&xdst->u.dst.xfrm->sel, fl,
> > > +                                xdst->u.dst.xfrm->sel.family))
> > > &&
> > > +           security_xfrm_state_pol_flow_match(xdst->u.dst.xfrm,
> > > +                                              xdst->pols[0], fl)
> > > &&
> 
> ... so this needs to walk the bundle and validate each selector.
> 
> Alternatively we could always do template resolution and then check
> that all states found match those of the old pcpu xdst:

With your patch below, the selinux-testsuite passes, and I couldn't
trigger any failures even running the inet_socket tests repeatedly.

Tested-by: Stephen Smalley <sds@tycho.nsa.gov>

> 
> diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> --- a/net/xfrm/xfrm_policy.c
> +++ b/net/xfrm/xfrm_policy.c
> @@ -1786,19 +1786,23 @@ void xfrm_policy_cache_flush(void)
>  	put_online_cpus();
>  }
>  
> -static bool xfrm_pol_dead(struct xfrm_dst *xdst)
> +static bool xfrm_xdst_can_reuse(struct xfrm_dst *xdst,
> +				struct xfrm_state * const xfrm[],
> +				int num)
>  {
> -	unsigned int num_pols = xdst->num_pols;
> -	unsigned int pol_dead = 0, i;
> +	const struct dst_entry *dst = &xdst->u.dst;
> +	int i;
>  
> -	for (i = 0; i < num_pols; i++)
> -		pol_dead |= xdst->pols[i]->walk.dead;
> +	if (xdst->num_xfrms != num)
> +		return false;
>  
> -	/* Mark DST_OBSOLETE_DEAD to fail the next xfrm_dst_check()
> */
> -	if (pol_dead)
> -		xdst->u.dst.obsolete = DST_OBSOLETE_DEAD;
> +	for (i = 0; i < num; i++) {
> +		if (!dst || dst->xfrm != xfrm[i])
> +			return false;
> +		dst = dst->child;
> +	}
>  
> -	return pol_dead;
> +	return xfrm_bundle_ok(xdst);
>  }
>  
>  static struct xfrm_dst *
> @@ -1812,26 +1816,28 @@ xfrm_resolve_and_create_bundle(struct
> xfrm_policy **pols, int num_pols,
>  	struct dst_entry *dst;
>  	int err;
>  
> +	/* Try to instantiate a bundle */
> +	err = xfrm_tmpl_resolve(pols, num_pols, fl, xfrm, family);
> +	if (err <= 0) {
> +		if (err != 0 && err != -EAGAIN)
> +			XFRM_INC_STATS(net,
> LINUX_MIB_XFRMOUTPOLERROR);
> +		return ERR_PTR(err);
> +	}
> +
>  	xdst = this_cpu_read(xfrm_last_dst);
>  	if (xdst &&
>  	    xdst->u.dst.dev == dst_orig->dev &&
>  	    xdst->num_pols == num_pols &&
> -	    !xfrm_pol_dead(xdst) &&
>  	    memcmp(xdst->pols, pols,
>  		   sizeof(struct xfrm_policy *) * num_pols) == 0 &&
> -	    xfrm_bundle_ok(xdst)) {
> +	    xfrm_xdst_can_reuse(xdst, xfrm, err)) {
>  		dst_hold(&xdst->u.dst);
> +		while (err > 0)
> +			xfrm_state_put(xfrm[--err]);
>  		return xdst;
>  	}
>  
>  	old = xdst;
> -	/* Try to instantiate a bundle */
> -	err = xfrm_tmpl_resolve(pols, num_pols, fl, xfrm, family);
> -	if (err <= 0) {
> -		if (err != 0 && err != -EAGAIN)
> -			XFRM_INC_STATS(net,
> LINUX_MIB_XFRMOUTPOLERROR);
> -		return ERR_PTR(err);
> -	}
>  
>  	dst = xfrm_bundle_create(pols[0], xfrm, err, fl, dst_orig);
>  	if (IS_ERR(dst)) {

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Paul Moore]

On Tue, Oct 31, 2017 at 7:08 PM, Florian Westphal <fw@strlen.de> wrote:
> Paul Moore <paul@paul-moore.com> wrote:
>> On Mon, Oct 30, 2017 at 10:58 AM, Stephen Smalley <sds@tycho.nsa.gov> wrote:
>> > matching before (as in this patch) or after calling xfrm_bundle_ok()?
>>
>> I would probably make the LSM call the last check, as you've done; but
>> I have to say that is just so it is consistent with the "LSM last"
>> philosophy and not because of any performance related argument.
>>
>> > ... Also,
>> > do we need to test xfrm->sel.family before calling xfrm_selector_match
>> > (as in this patch) or not - xfrm_state_look_at() does so when the
>> > state is XFRM_STATE_VALID but not when it is _ERROR or _EXPIRED?
>>
>> Speaking purely from a SELinux perspective, I'm not sure it matters:
>> as long as the labels match we are happy.  However, from a general
>> IPsec perspective it does seem like a reasonable thing.
>>
>> Granted I'm probably missing something, but it seems a little odd that
>> the code isn't already checking that the selectors match (... what am
>> I missing?).  It does check the policies, maybe that is enough in the
>> normal IPsec case?
>
> The assumption was that identical policies would yield the same SAs,
> but thats not correct.

Well, to be fair, I think the assumption is valid for normal,
unlabeled IPsec.  The problem comes when SELinux starts labeling SAs
and now you have multiple SAs for a given policy, each differing only
in the SELinux/LSM label.

Considering that adding the SELinux/LSM label effectively adds an
additional selector, I'm wondering if we should simply add the
SELinux/LSM label matching to xfrm_selector_match()?  Looking quickly
at the code it seems as though we always follow xfrm_selector_match()
with a LSM check anyway, the one exception being in
__xfrm_policy_check() ... which *might* be a valid exception, as we
don't do our access checks for inbound traffic at that point in the
stack.

>> > diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
>> > index 2746b62..171818b 100644
>> > --- a/net/xfrm/xfrm_policy.c
>> > +++ b/net/xfrm/xfrm_policy.c
>> > @@ -1820,6 +1820,11 @@ xfrm_resolve_and_create_bundle(struct xfrm_policy **pols, int num_pols,
>> >             !xfrm_pol_dead(xdst) &&
>> >             memcmp(xdst->pols, pols,
>> >                    sizeof(struct xfrm_policy *) * num_pols) == 0 &&
>> > +           (!xdst->u.dst.xfrm->sel.family ||
>> > +            xfrm_selector_match(&xdst->u.dst.xfrm->sel, fl,
>> > +                                xdst->u.dst.xfrm->sel.family)) &&
>> > +           security_xfrm_state_pol_flow_match(xdst->u.dst.xfrm,
>> > +                                              xdst->pols[0], fl) &&
>
> ... so this needs to walk the bundle and validate each selector.
>
> Alternatively we could always do template resolution and then check
> that all states found match those of the old pcpu xdst:
>
> diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> --- a/net/xfrm/xfrm_policy.c
> +++ b/net/xfrm/xfrm_policy.c
> @@ -1786,19 +1786,23 @@ void xfrm_policy_cache_flush(void)
>         put_online_cpus();
>  }
>
> -static bool xfrm_pol_dead(struct xfrm_dst *xdst)
> +static bool xfrm_xdst_can_reuse(struct xfrm_dst *xdst,
> +                               struct xfrm_state * const xfrm[],
> +                               int num)
>  {
> -       unsigned int num_pols = xdst->num_pols;
> -       unsigned int pol_dead = 0, i;
> +       const struct dst_entry *dst = &xdst->u.dst;
> +       int i;
>
> -       for (i = 0; i < num_pols; i++)
> -               pol_dead |= xdst->pols[i]->walk.dead;
> +       if (xdst->num_xfrms != num)
> +               return false;
>
> -       /* Mark DST_OBSOLETE_DEAD to fail the next xfrm_dst_check() */
> -       if (pol_dead)
> -               xdst->u.dst.obsolete = DST_OBSOLETE_DEAD;
> +       for (i = 0; i < num; i++) {
> +               if (!dst || dst->xfrm != xfrm[i])
> +                       return false;
> +               dst = dst->child;
> +       }
>
> -       return pol_dead;
> +       return xfrm_bundle_ok(xdst);
>  }
>
>  static struct xfrm_dst *
> @@ -1812,26 +1816,28 @@ xfrm_resolve_and_create_bundle(struct xfrm_policy **pols, int num_pols,
>         struct dst_entry *dst;
>         int err;
>
> +       /* Try to instantiate a bundle */
> +       err = xfrm_tmpl_resolve(pols, num_pols, fl, xfrm, family);
> +       if (err <= 0) {
> +               if (err != 0 && err != -EAGAIN)
> +                       XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTPOLERROR);
> +               return ERR_PTR(err);
> +       }
> +
>         xdst = this_cpu_read(xfrm_last_dst);
>         if (xdst &&
>             xdst->u.dst.dev == dst_orig->dev &&
>             xdst->num_pols == num_pols &&
> -           !xfrm_pol_dead(xdst) &&
>             memcmp(xdst->pols, pols,
>                    sizeof(struct xfrm_policy *) * num_pols) == 0 &&
> -           xfrm_bundle_ok(xdst)) {
> +           xfrm_xdst_can_reuse(xdst, xfrm, err)) {
>                 dst_hold(&xdst->u.dst);
> +               while (err > 0)
> +                       xfrm_state_put(xfrm[--err]);
>                 return xdst;
>         }
>
>         old = xdst;
> -       /* Try to instantiate a bundle */
> -       err = xfrm_tmpl_resolve(pols, num_pols, fl, xfrm, family);
> -       if (err <= 0) {
> -               if (err != 0 && err != -EAGAIN)
> -                       XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTPOLERROR);
> -               return ERR_PTR(err);
> -       }
>
>         dst = xfrm_bundle_create(pols[0], xfrm, err, fl, dst_orig);
>         if (IS_ERR(dst)) {
> --
> 2.13.6

-- 
paul moore
www.paul-moore.com

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Stephen Smalley]

On Wed, 2017-11-01 at 17:39 -0400, Paul Moore wrote:
> On Tue, Oct 31, 2017 at 7:08 PM, Florian Westphal <fw@strlen.de>
> wrote:
> > Paul Moore <paul@paul-moore.com> wrote:
> > > On Mon, Oct 30, 2017 at 10:58 AM, Stephen Smalley <sds@tycho.nsa.
> > > gov> wrote:
> > > > matching before (as in this patch) or after calling
> > > > xfrm_bundle_ok()?
> > > 
> > > I would probably make the LSM call the last check, as you've
> > > done; but
> > > I have to say that is just so it is consistent with the "LSM
> > > last"
> > > philosophy and not because of any performance related argument.
> > > 
> > > > ... Also,
> > > > do we need to test xfrm->sel.family before calling
> > > > xfrm_selector_match
> > > > (as in this patch) or not - xfrm_state_look_at() does so when
> > > > the
> > > > state is XFRM_STATE_VALID but not when it is _ERROR or
> > > > _EXPIRED?
> > > 
> > > Speaking purely from a SELinux perspective, I'm not sure it
> > > matters:
> > > as long as the labels match we are happy.  However, from a
> > > general
> > > IPsec perspective it does seem like a reasonable thing.
> > > 
> > > Granted I'm probably missing something, but it seems a little odd
> > > that
> > > the code isn't already checking that the selectors match (...
> > > what am
> > > I missing?).  It does check the policies, maybe that is enough in
> > > the
> > > normal IPsec case?
> > 
> > The assumption was that identical policies would yield the same
> > SAs,
> > but thats not correct.
> 
> Well, to be fair, I think the assumption is valid for normal,
> unlabeled IPsec.  The problem comes when SELinux starts labeling SAs
> and now you have multiple SAs for a given policy, each differing only
> in the SELinux/LSM label.

No, it is invalid for normal, unlabeled IPSEC too, in the case where
one has defined xfrm state selectors.  That's what my other testsuite
patch (which is presently only on the xfrmselectortest branch) is
exercising - matching of xfrm state selectors.  But in any event,
Florian's patch fixes both, so I'm fine with it.  I don't know though
how it compares performance-wise with walking the bundle and just
calling security_xfrm_state_pol_flow_match() and xfrm_selector_match()
on each one.

> 
> Considering that adding the SELinux/LSM label effectively adds an
> additional selector, I'm wondering if we should simply add the
> SELinux/LSM label matching to xfrm_selector_match()?  Looking quickly
> at the code it seems as though we always follow xfrm_selector_match()
> with a LSM check anyway, the one exception being in
> __xfrm_policy_check() ... which *might* be a valid exception, as we
> don't do our access checks for inbound traffic at that point in the
> stack.

Possibly, but that should probably be a separate patch. We should just
fix this regression for 4.14, either via Florian's patch or by
augmenting my patch to perform the matching calls on all of the xfrms.

> 
> > > > diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> > > > index 2746b62..171818b 100644
> > > > --- a/net/xfrm/xfrm_policy.c
> > > > +++ b/net/xfrm/xfrm_policy.c
> > > > @@ -1820,6 +1820,11 @@ xfrm_resolve_and_create_bundle(struct
> > > > xfrm_policy **pols, int num_pols,
> > > >             !xfrm_pol_dead(xdst) &&
> > > >             memcmp(xdst->pols, pols,
> > > >                    sizeof(struct xfrm_policy *) * num_pols) ==
> > > > 0 &&
> > > > +           (!xdst->u.dst.xfrm->sel.family ||
> > > > +            xfrm_selector_match(&xdst->u.dst.xfrm->sel, fl,
> > > > +                                xdst->u.dst.xfrm->sel.family)) 
> > > > &&
> > > > +           security_xfrm_state_pol_flow_match(xdst-
> > > > >u.dst.xfrm,
> > > > +                                              xdst->pols[0],
> > > > fl) &&
> > 
> > ... so this needs to walk the bundle and validate each selector.
> > 
> > Alternatively we could always do template resolution and then check
> > that all states found match those of the old pcpu xdst:
> > 
> > diff --git a/net/xfrm/xfrm_policy.c b/net/xfrm/xfrm_policy.c
> > --- a/net/xfrm/xfrm_policy.c
> > +++ b/net/xfrm/xfrm_policy.c
> > @@ -1786,19 +1786,23 @@ void xfrm_policy_cache_flush(void)
> >         put_online_cpus();
> >  }
> > 
> > -static bool xfrm_pol_dead(struct xfrm_dst *xdst)
> > +static bool xfrm_xdst_can_reuse(struct xfrm_dst *xdst,
> > +                               struct xfrm_state * const xfrm[],
> > +                               int num)
> >  {
> > -       unsigned int num_pols = xdst->num_pols;
> > -       unsigned int pol_dead = 0, i;
> > +       const struct dst_entry *dst = &xdst->u.dst;
> > +       int i;
> > 
> > -       for (i = 0; i < num_pols; i++)
> > -               pol_dead |= xdst->pols[i]->walk.dead;
> > +       if (xdst->num_xfrms != num)
> > +               return false;
> > 
> > -       /* Mark DST_OBSOLETE_DEAD to fail the next xfrm_dst_check()
> > */
> > -       if (pol_dead)
> > -               xdst->u.dst.obsolete = DST_OBSOLETE_DEAD;
> > +       for (i = 0; i < num; i++) {
> > +               if (!dst || dst->xfrm != xfrm[i])
> > +                       return false;
> > +               dst = dst->child;
> > +       }
> > 
> > -       return pol_dead;
> > +       return xfrm_bundle_ok(xdst);
> >  }
> > 
> >  static struct xfrm_dst *
> > @@ -1812,26 +1816,28 @@ xfrm_resolve_and_create_bundle(struct
> > xfrm_policy **pols, int num_pols,
> >         struct dst_entry *dst;
> >         int err;
> > 
> > +       /* Try to instantiate a bundle */
> > +       err = xfrm_tmpl_resolve(pols, num_pols, fl, xfrm, family);
> > +       if (err <= 0) {
> > +               if (err != 0 && err != -EAGAIN)
> > +                       XFRM_INC_STATS(net,
> > LINUX_MIB_XFRMOUTPOLERROR);
> > +               return ERR_PTR(err);
> > +       }
> > +
> >         xdst = this_cpu_read(xfrm_last_dst);
> >         if (xdst &&
> >             xdst->u.dst.dev == dst_orig->dev &&
> >             xdst->num_pols == num_pols &&
> > -           !xfrm_pol_dead(xdst) &&
> >             memcmp(xdst->pols, pols,
> >                    sizeof(struct xfrm_policy *) * num_pols) == 0 &&
> > -           xfrm_bundle_ok(xdst)) {
> > +           xfrm_xdst_can_reuse(xdst, xfrm, err)) {
> >                 dst_hold(&xdst->u.dst);
> > +               while (err > 0)
> > +                       xfrm_state_put(xfrm[--err]);
> >                 return xdst;
> >         }
> > 
> >         old = xdst;
> > -       /* Try to instantiate a bundle */
> > -       err = xfrm_tmpl_resolve(pols, num_pols, fl, xfrm, family);
> > -       if (err <= 0) {
> > -               if (err != 0 && err != -EAGAIN)
> > -                       XFRM_INC_STATS(net,
> > LINUX_MIB_XFRMOUTPOLERROR);
> > -               return ERR_PTR(err);
> > -       }
> > 
> >         dst = xfrm_bundle_create(pols[0], xfrm, err, fl, dst_orig);
> >         if (IS_ERR(dst)) {
> > --
> > 2.13.6
> 
> 

Re: [RFC PATCH] xfrm: fix regression introduced by xdst pcpu cache

[Paul Moore]

On Thu, Nov 2, 2017 at 8:58 AM, Stephen Smalley <sds@tycho.nsa.gov> wrote:
> On Wed, 2017-11-01 at 17:39 -0400, Paul Moore wrote:
>> On Tue, Oct 31, 2017 at 7:08 PM, Florian Westphal <fw@strlen.de>
>> wrote:
>> > Paul Moore <paul@paul-moore.com> wrote:
>> > > On Mon, Oct 30, 2017 at 10:58 AM, Stephen Smalley <sds@tycho.nsa.
>> > > gov> wrote:
>> > > > matching before (as in this patch) or after calling
>> > > > xfrm_bundle_ok()?
>> > >
>> > > I would probably make the LSM call the last check, as you've
>> > > done; but
>> > > I have to say that is just so it is consistent with the "LSM
>> > > last"
>> > > philosophy and not because of any performance related argument.
>> > >
>> > > > ... Also,
>> > > > do we need to test xfrm->sel.family before calling
>> > > > xfrm_selector_match
>> > > > (as in this patch) or not - xfrm_state_look_at() does so when
>> > > > the
>> > > > state is XFRM_STATE_VALID but not when it is _ERROR or
>> > > > _EXPIRED?
>> > >
>> > > Speaking purely from a SELinux perspective, I'm not sure it
>> > > matters:
>> > > as long as the labels match we are happy.  However, from a
>> > > general
>> > > IPsec perspective it does seem like a reasonable thing.
>> > >
>> > > Granted I'm probably missing something, but it seems a little odd
>> > > that
>> > > the code isn't already checking that the selectors match (...
>> > > what am
>> > > I missing?).  It does check the policies, maybe that is enough in
>> > > the
>> > > normal IPsec case?
>> >
>> > The assumption was that identical policies would yield the same
>> > SAs,
>> > but thats not correct.
>>
>> Well, to be fair, I think the assumption is valid for normal,
>> unlabeled IPsec.  The problem comes when SELinux starts labeling SAs
>> and now you have multiple SAs for a given policy, each differing only
>> in the SELinux/LSM label.
>
> No, it is invalid for normal, unlabeled IPSEC too, in the case where
> one has defined xfrm state selectors.  That's what my other testsuite
> patch (which is presently only on the xfrmselectortest branch) is
> exercising - matching of xfrm state selectors.  But in any event,
> Florian's patch fixes both, so I'm fine with it.  I don't know though
> how it compares performance-wise with walking the bundle and just
> calling security_xfrm_state_pol_flow_match() and xfrm_selector_match()
> on each one.
>
>> Considering that adding the SELinux/LSM label effectively adds an
>> additional selector, I'm wondering if we should simply add the
>> SELinux/LSM label matching to xfrm_selector_match()?  Looking quickly
>> at the code it seems as though we always follow xfrm_selector_match()
>> with a LSM check anyway, the one exception being in
>> __xfrm_policy_check() ... which *might* be a valid exception, as we
>> don't do our access checks for inbound traffic at that point in the
>> stack.
>
> Possibly, but that should probably be a separate patch. We should just
> fix this regression for 4.14, either via Florian's patch or by
> augmenting my patch to perform the matching calls on all of the xfrms.

I agree that v4.14 should get the smallest patch possible that fixes
the problem.  I was just looking at the patches presented so far and
thinking out loud.

-- 
paul moore
www.paul-moore.com