nft equivalent of iptables command

nft equivalent of iptables command

[Serguei Bezverkhi (sbezverk)]

Hello,

I am trying to find an equivalent nft command for the following iptables command.  Specifically "physdev" and "addrtype", I could not find so far, some help would be very appreciated.

-A KUBE-SERVICES 
-m comment --comment "default/portal:portal external IP" 
-m tcp 
-p tcp 
-d 192.168.80.104/32 
--dport 8989 
-m physdev ! --physdev-is-in             
-m addrtype ! --src-type LOCAL 
-j KUBE-SVC-MUPXPVK4XAZHSWAR

Thank you
Serguei

Re: nft equivalent of iptables command

[Florian Westphal]

Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com> wrote:
> Hello,
> 
> I am trying to find an equivalent nft command for the following iptables command.  Specifically "physdev" and "addrtype", I could not find so far, some help would be very appreciated.

> -m physdev ! --physdev-is-in            

This has no equivalent.  The rule above matches when 'call-iptables' sysctl
is enabled and the packet did not enter via a bridge interface.
So, its only false when it did enter via a bridge interface.

In case the sysctl is off, the rule always matches and can be omitted.

nftables currently assumes that call-iptables is off, and that
bridges have their own filter rules in the netdev and/or
bridge families.

inet/ip/ip6 are assumed to only see packets that are routed by the ip
stack.

> -m addrtype ! --src-type LOCAL 

fib saddr type != local

Re: nft equivalent of iptables command

[Dan Williams]

On Thu, 2019-08-22 at 16:16 +0200, Florian Westphal wrote:
> Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com> wrote:
> > Hello,
> > 
> > I am trying to find an equivalent nft command for the following
> > iptables command.  Specifically "physdev" and "addrtype", I could
> > not find so far, some help would be very appreciated.
> > -m physdev ! --physdev-is-in            
> 
> This has no equivalent.  The rule above matches when 'call-iptables'
> sysctl
> is enabled and the packet did not enter via a bridge interface.
> So, its only false when it did enter via a bridge interface.

Also note that the rule in kube-proxy that adds physdev/physdev-is-in
has the comment:

// This is imperfect in the face of network plugins that might not use
a bridge, but we can revisit that later.

and it clearly doesn't work when the network plugin doesn't use a
bridge interface for containers, which is a lot of them. In fact, that
rule should instead be rewritten upstream to use "-s !<ClusterCIDR>" or
something rather than rules about a network interface that may/may not
exist.

IMHO this is really an issue in kube-proxy (code was added in 2015)
that hasn't been cleaned up since Kubernetes started supporting more
diverse network plugins.

Dan

> In case the sysctl is off, the rule always matches and can be
> omitted.
> 
> nftables currently assumes that call-iptables is off, and that
> bridges have their own filter rules in the netdev and/or
> bridge families.
> 
> inet/ip/ip6 are assumed to only see packets that are routed by the ip
> stack.
> 
> > -m addrtype ! --src-type LOCAL 
> 
> fib saddr type != local

Re: nft equivalent of iptables command

[Serguei Bezverkhi (sbezverk)]

Thank you very much Florian. I will use fib type local as a replacement.
Serguei 

On 2019-08-22, 10:20 AM, "Florian Westphal" <fw@strlen.de> wrote:

    Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com> wrote:
    > Hello,
    > 
    > I am trying to find an equivalent nft command for the following iptables command.  Specifically "physdev" and "addrtype", I could not find so far, some help would be very appreciated.
    
    > -m physdev ! --physdev-is-in            
    
    This has no equivalent.  The rule above matches when 'call-iptables' sysctl
    is enabled and the packet did not enter via a bridge interface.
    So, its only false when it did enter via a bridge interface.
    
    In case the sysctl is off, the rule always matches and can be omitted.
    
    nftables currently assumes that call-iptables is off, and that
    bridges have their own filter rules in the netdev and/or
    bridge families.
    
    inet/ip/ip6 are assumed to only see packets that are routed by the ip
    stack.
    
    > -m addrtype ! --src-type LOCAL 
    
    fib saddr type != local
    

Re: nft equivalent of iptables command

[Serguei Bezverkhi (sbezverk)]

On 2019-08-22, 10:49 AM, "Dan Williams" <dcbw@redhat.com> wrote:

    On Thu, 2019-08-22 at 16:16 +0200, Florian Westphal wrote:
    > Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com> wrote:
    > > Hello,
    > > 
    > > I am trying to find an equivalent nft command for the following
    > > iptables command.  Specifically "physdev" and "addrtype", I could
    > > not find so far, some help would be very appreciated.
    > > -m physdev ! --physdev-is-in            
    > 
    > This has no equivalent.  The rule above matches when 'call-iptables'
    > sysctl
    > is enabled and the packet did not enter via a bridge interface.
    > So, its only false when it did enter via a bridge interface.
    
    Also note that the rule in kube-proxy that adds physdev/physdev-is-in
    has the comment:
    
    // This is imperfect in the face of network plugins that might not use
    a bridge, but we can revisit that later.
    
    and it clearly doesn't work when the network plugin doesn't use a
    bridge interface for containers, which is a lot of them. In fact, that
    rule should instead be rewritten upstream to use "-s !<ClusterCIDR>" or
    something rather than rules about a network interface that may/may not
    exist.


Thank you Dan for your input.   
That was exactly what I thought about "-s !<ClusterCIDR>" when I saw Florian reply.  I will use it for now in nft rules which nft kube-proxy builds for this specific case.

Serguei

    IMHO this is really an issue in kube-proxy (code was added in 2015)
    that hasn't been cleaned up since Kubernetes started supporting more
    diverse network plugins.
    
    Dan
    
    > In case the sysctl is off, the rule always matches and can be
    > omitted.
    > 
    > nftables currently assumes that call-iptables is off, and that
    > bridges have their own filter rules in the netdev and/or
    > bridge families.
    > 
    > inet/ip/ip6 are assumed to only see packets that are routed by the ip
    > stack.
    > 
    > > -m addrtype ! --src-type LOCAL 
    > 
    > fib saddr type != local
    
    

Re: nft equivalent of iptables command

[Florian Westphal]

Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com> wrote:
> That was exactly what I thought about "-s !<ClusterCIDR>" when I saw Florian reply.  I will use it for now in nft rules which nft kube-proxy builds for this specific case.

I think that in ideal case, no rules would be generated on the fly,
and that instead it should add/remove elements from nftables maps and sets.

Re: nft equivalent of iptables command

[Serguei Bezverkhi (sbezverk)]

On 2019-08-22, 11:16 AM, "Florian Westphal" <fw@strlen.de> wrote:

    Serguei Bezverkhi (sbezverk) <sbezverk@cisco.com> wrote:
    > That was exactly what I thought about "-s !<ClusterCIDR>" when I saw Florian reply.  I will use it for now in nft rules which nft kube-proxy builds for this specific case.
    
    I think that in ideal case, no rules would be generated on the fly,
    and that instead it should add/remove elements from nftables maps and sets.

Great idea, once we have API implemented for maps I will give it a try to see how it would fit into proxy logic.