delete matching rule like it can be done in case of iptables

delete matching rule like it can be done in case of iptables

[Amish]

Hello,

nftables wiki [1] mentions this:

 > Note: There are plans to support rule deletion by passing:
 > % nft delete rule filter output ip saddr 192.168.1.1 counter

Any idea when will this happen? Because I thought it was very important 
feature. (unless I missed an alternate way to do it)

I want to migrate from iptables to nftables (from many years) but 
deleting a rule via script is not as easy as in case of iptables.

Obtaining the handle first and then deleting it is difficult 
programmatically.

Why is it difficult for nftables to find and delete matching rule?

Is there any ETA for this because its a roadblock for migrating my scripts.

Thank you,

Amish.

[1] 
https://wiki.nftables.org/wiki-nftables/index.php/Simple_rule_management#Removing_rules

Re: delete matching rule like it can be done in case of iptables

[Pablo Neira Ayuso]

On Sun, Dec 05, 2021 at 05:25:29PM +0530, Amish wrote:
> Hello,
> 
> nftables wiki [1] mentions this:
> 
> > Note: There are plans to support rule deletion by passing:
> > % nft delete rule filter output ip saddr 192.168.1.1 counter
> 
> Any idea when will this happen? Because I thought it was very important
> feature. (unless I missed an alternate way to do it)
> 
> I want to migrate from iptables to nftables (from many years) but deleting a
> rule via script is not as easy as in case of iptables.
> 
> Obtaining the handle first and then deleting it is difficult
> programmatically.

You can use --echo and --handle options to fetch the rule handle.

 # nft -e -a  add rule x y counter
 add rule ip x y counter packets 0 bytes 0 # handle 3
 # new generation 5 by process 91190 (nft)

Re: delete matching rule like it can be done in case of iptables

[Amish]

On 06/12/21 18:41, Pablo Neira Ayuso wrote:
> On Sun, Dec 05, 2021 at 05:25:29PM +0530, Amish wrote:
>> Hello,
>>
>> nftables wiki [1] mentions this:
>>
>>> Note: There are plans to support rule deletion by passing:
>>> % nft delete rule filter output ip saddr 192.168.1.1 counter
>> Any idea when will this happen? Because I thought it was very important
>> feature. (unless I missed an alternate way to do it)
>>
>> I want to migrate from iptables to nftables (from many years) but deleting a
>> rule via script is not as easy as in case of iptables.
>>
>> Obtaining the handle first and then deleting it is difficult
>> programmatically.
> You can use --echo and --handle options to fetch the rule handle.
>
>   # nft -e -a  add rule x y counter
>   add rule ip x y counter packets 0 bytes 0 # handle 3
>   # new generation 5 by process 91190 (nft)

Well then I need to keep recording each rule addition somewhere so that 
I can delete by handle in future.

Some rules are added manually, some added by scripts. Scripts may want 
to remove manually added rule.

So its not as easy like in iptables.

I guess nftables will never get that feature then :(

Regards,

Amish

Re: delete matching rule like it can be done in case of iptables

[Daniel]

Hi Amish

Le 06/12/2021 à 14:30, Amish a écrit :
> On 06/12/21 18:41, Pablo Neira Ayuso wrote:
>> On Sun, Dec 05, 2021 at 05:25:29PM +0530, Amish wrote:
>>> Hello,
>>>
>>> nftables wiki [1] mentions this:
>>>
>>>> Note: There are plans to support rule deletion by passing:
>>>> % nft delete rule filter output ip saddr 192.168.1.1 counter
>>> Any idea when will this happen? Because I thought it was very important
>>> feature. (unless I missed an alternate way to do it)
>>>
>>> I want to migrate from iptables to nftables (from many years) but 
>>> deleting a
>>> rule via script is not as easy as in case of iptables.
>>>
>>> Obtaining the handle first and then deleting it is difficult
>>> programmatically.
>> You can use --echo and --handle options to fetch the rule handle.
>>
>>   # nft -e -a  add rule x y counter
>>   add rule ip x y counter packets 0 bytes 0 # handle 3
>>   # new generation 5 by process 91190 (nft)
>
> Well then I need to keep recording each rule addition somewhere so 
> that I can delete by handle in future.
>
> Some rules are added manually, some added by scripts. Scripts may want 
> to remove manually added rule.
>
> So its not as easy like in iptables.
In a script you can use eg

myhandle=$(echo `$nft -sa list chain ip mangle prerouting |grep -F "ct 
state new counter jump MAIN"|grep -oP '(# handle ).*'`|cut -d " " -f 3)
$fwtables delete rule ip mangle prerouting handle $myhandle

and you're done. ip, mangle prerouting and rule to delete could be 
sended as parameters in a bash function for instance.

[...]

-- 
Daniel

Re: delete matching rule like it can be done in case of iptables

[Eric Garver]

On Mon, Dec 06, 2021 at 07:00:47PM +0530, Amish wrote:
> On 06/12/21 18:41, Pablo Neira Ayuso wrote:
> > On Sun, Dec 05, 2021 at 05:25:29PM +0530, Amish wrote:
> > > Hello,
> > > 
> > > nftables wiki [1] mentions this:
> > > 
> > > > Note: There are plans to support rule deletion by passing:
> > > > % nft delete rule filter output ip saddr 192.168.1.1 counter
> > > Any idea when will this happen? Because I thought it was very important
> > > feature. (unless I missed an alternate way to do it)
> > > 
> > > I want to migrate from iptables to nftables (from many years) but deleting a
> > > rule via script is not as easy as in case of iptables.
> > > 
> > > Obtaining the handle first and then deleting it is difficult
> > > programmatically.
> > You can use --echo and --handle options to fetch the rule handle.
> > 
> >   # nft -e -a  add rule x y counter
> >   add rule ip x y counter packets 0 bytes 0 # handle 3
> >   # new generation 5 by process 91190 (nft)
> 
> Well then I need to keep recording each rule addition somewhere so that I
> can delete by handle in future.
> 
> Some rules are added manually, some added by scripts. Scripts may want to
> remove manually added rule.
> 
> So its not as easy like in iptables.
> 
> I guess nftables will never get that feature then :(

Maybe you can use a comment to set your own identifier. At delete time
you would have to grep the ruleset (--handle) listing for your comment
then parse the handle.

In the past we've talked about a user cookie/identifier, but it never
got implemented.

firewalld solves it by maintaining a cache of created rules and the
handle they were created with (--echo --handle).

Re: delete matching rule like it can be done in case of iptables

[Amish]

On 06/12/21 19:17, Daniel wrote:
> Hi Amish
>
> Le 06/12/2021 à 14:30, Amish a écrit :
>> On 06/12/21 18:41, Pablo Neira Ayuso wrote:
>>> On Sun, Dec 05, 2021 at 05:25:29PM +0530, Amish wrote:
>>>> Hello,
>>>>
>>>> nftables wiki [1] mentions this:
>>>>
>>>>> Note: There are plans to support rule deletion by passing:
>>>>> % nft delete rule filter output ip saddr 192.168.1.1 counter
>>>> Any idea when will this happen? Because I thought it was very 
>>>> important
>>>> feature. (unless I missed an alternate way to do it)
>>>>
>>>> I want to migrate from iptables to nftables (from many years) but 
>>>> deleting a
>>>> rule via script is not as easy as in case of iptables.
>>>>
>>>> Obtaining the handle first and then deleting it is difficult
>>>> programmatically.
>>> You can use --echo and --handle options to fetch the rule handle.
>>>
>>>   # nft -e -a  add rule x y counter
>>>   add rule ip x y counter packets 0 bytes 0 # handle 3
>>>   # new generation 5 by process 91190 (nft)
>>
>> Well then I need to keep recording each rule addition somewhere so 
>> that I can delete by handle in future.
>>
>> Some rules are added manually, some added by scripts. Scripts may 
>> want to remove manually added rule.
>>
>> So its not as easy like in iptables.

> In a script you can use eg
>
> myhandle=$(echo `$nft -sa list chain ip mangle prerouting |grep -F "ct 
> state new counter jump MAIN"|grep -oP '(# handle ).*'`|cut -d " " -f 3)
> $fwtables delete rule ip mangle prerouting handle $myhandle
>
> and you're done. ip, mangle prerouting and rule to delete could be 
> sended as parameters in a bash function for instance.
>
> [...]
>
Hi Daniel

Thank you for your reply.

This actually is my basic problem. Rules are complex. They are not as 
simple as above example.

Hence the parsing (grepping) is not straight forward like in above example.

state may not always be "new". (can be established or related or both)

There may not always be counter in rule.

Some rules may have anonymous sets.

Some rules may have TCP port redirection.

So on ...

I can not write a script for each and every combination to detect the 
handle.

In case of iptables I can delete the rule by giving exact same 
expression except instead of giving -A, I just have to give -D.

Regards,

Amish.

Re: delete matching rule like it can be done in case of iptables

[Daniel]

Amish

Le 07/12/2021 à 06:29, Amish a écrit :
>
> On 06/12/21 19:17, Daniel wrote:
>> Hi Amish
>>
>> Le 06/12/2021 à 14:30, Amish a écrit :
>>> On 06/12/21 18:41, Pablo Neira Ayuso wrote:
>>>> On Sun, Dec 05, 2021 at 05:25:29PM +0530, Amish wrote:
>>>>> Hello,
>>>>>
>>>>> nftables wiki [1] mentions this:
>>>>>
>>>>>> Note: There are plans to support rule deletion by passing:
>>>>>> % nft delete rule filter output ip saddr 192.168.1.1 counter
>>>>> Any idea when will this happen? Because I thought it was very 
>>>>> important
>>>>> feature. (unless I missed an alternate way to do it)
>>>>>
>>>>> I want to migrate from iptables to nftables (from many years) but 
>>>>> deleting a
>>>>> rule via script is not as easy as in case of iptables.
>>>>>
>>>>> Obtaining the handle first and then deleting it is difficult
>>>>> programmatically.
>>>> You can use --echo and --handle options to fetch the rule handle.
>>>>
>>>>   # nft -e -a  add rule x y counter
>>>>   add rule ip x y counter packets 0 bytes 0 # handle 3
>>>>   # new generation 5 by process 91190 (nft)
>>>
>>> Well then I need to keep recording each rule addition somewhere so 
>>> that I can delete by handle in future.
>>>
>>> Some rules are added manually, some added by scripts. Scripts may 
>>> want to remove manually added rule.
>>>
>>> So its not as easy like in iptables.
>
>> In a script you can use eg
>>
>> myhandle=$(echo `$nft -sa list chain ip mangle prerouting |grep -F 
>> "ct state new counter jump MAIN"|grep -oP '(# handle ).*'`|cut -d " " 
>> -f 3)
>> $fwtables delete rule ip mangle prerouting handle $myhandle
>>
>> and you're done. ip, mangle prerouting and rule to delete could be 
>> sended as parameters in a bash function for instance.
>>
>> [...]
>>
> Hi Daniel
>
> Thank you for your reply.
>
> This actually is my basic problem. Rules are complex. They are not as 
> simple as above example.
>
> Hence the parsing (grepping) is not straight forward like in above 
> example.
>
> state may not always be "new". (can be established or related or both)
>
> There may not always be counter in rule.
>
> Some rules may have anonymous sets.
>
> Some rules may have TCP port redirection.
>
> So on ...
>
> I can not write a script for each and every combination to detect the 
> handle.
>
> In case of iptables I can delete the rule by giving exact same 
> expression except instead of giving -A, I just have to give -D.

You don't understand the purpose of the abvoe exemple. More clear:

#!/bin/bash

MyFct {

myhandle=$(echo `$nft -sa list chain $1 $2 $3 |grep -F $4|grep -oP '(# 
handle ).*'`|cut -d " " -f 3)
$fwtables delete rule $1 $2 $3 handle $myhandle

}

MyFct ip mangle prerouting "ct state new counter jump MAIN"

or

MyFct ip6 filter input "iif \"lan\" ct state invalid drop"

or whatever rule you want to delete

-- 
Daniel

Re: delete matching rule like it can be done in case of iptables

[Amish]

On 07/12/21 14:02, Daniel wrote:
> Amish
>
> Le 07/12/2021 à 06:29, Amish a écrit :
>>
>> On 06/12/21 19:17, Daniel wrote:
>>> Hi Amish
>>>
>>> Le 06/12/2021 à 14:30, Amish a écrit :
>>>> On 06/12/21 18:41, Pablo Neira Ayuso wrote:
>>>>> On Sun, Dec 05, 2021 at 05:25:29PM +0530, Amish wrote:
>>>>>> Hello,
>>>>>>
>>>>>> nftables wiki [1] mentions this:
>>>>>>
>>>>>>> Note: There are plans to support rule deletion by passing:
>>>>>>> % nft delete rule filter output ip saddr 192.168.1.1 counter
>>>>>> Any idea when will this happen? Because I thought it was very 
>>>>>> important
>>>>>> feature. (unless I missed an alternate way to do it)
>>>>>>
>>>>>> I want to migrate from iptables to nftables (from many years) but 
>>>>>> deleting a
>>>>>> rule via script is not as easy as in case of iptables.
>>>>>>
>>>>>> Obtaining the handle first and then deleting it is difficult
>>>>>> programmatically.
>>>>> You can use --echo and --handle options to fetch the rule handle.
>>>>>
>>>>>   # nft -e -a  add rule x y counter
>>>>>   add rule ip x y counter packets 0 bytes 0 # handle 3
>>>>>   # new generation 5 by process 91190 (nft)
>>>>
>>>> Well then I need to keep recording each rule addition somewhere so 
>>>> that I can delete by handle in future.
>>>>
>>>> Some rules are added manually, some added by scripts. Scripts may 
>>>> want to remove manually added rule.
>>>>
>>>> So its not as easy like in iptables.
>>
>>> In a script you can use eg
>>>
>>> myhandle=$(echo `$nft -sa list chain ip mangle prerouting |grep -F 
>>> "ct state new counter jump MAIN"|grep -oP '(# handle ).*'`|cut -d " 
>>> " -f 3)
>>> $fwtables delete rule ip mangle prerouting handle $myhandle
>>>
>>> and you're done. ip, mangle prerouting and rule to delete could be 
>>> sended as parameters in a bash function for instance.
>>>
>>> [...]
>>>
>> Hi Daniel
>>
>> Thank you for your reply.
>>
>> This actually is my basic problem. Rules are complex. They are not as 
>> simple as above example.
>>
>> Hence the parsing (grepping) is not straight forward like in above 
>> example.
>>
>> state may not always be "new". (can be established or related or both)
>>
>> There may not always be counter in rule.
>>
>> Some rules may have anonymous sets.
>>
>> Some rules may have TCP port redirection.
>>
>> So on ...
>>
>> I can not write a script for each and every combination to detect the 
>> handle.
>>
>> In case of iptables I can delete the rule by giving exact same 
>> expression except instead of giving -A, I just have to give -D.
>
> You don't understand the purpose of the abvoe exemple. More clear:
>
> #!/bin/bash
>
> MyFct {
>
> myhandle=$(echo `$nft -sa list chain $1 $2 $3 |grep -F $4|grep -oP '(# 
> handle ).*'`|cut -d " " -f 3)
> $fwtables delete rule $1 $2 $3 handle $myhandle
>
> }
>
> MyFct ip mangle prerouting "ct state new counter jump MAIN"
>
> or
>
> MyFct ip6 filter input "iif \"lan\" ct state invalid drop"
>
> or whatever rule you want to delete


Thank you but problem with this is that you need to know order of 
strings in nft output.

i.e. if ct state will be first or iif "lan" will be first. What will be 
quoted and what will not be quoted?

In above examples it looks easy to grep but when your rule gets complex 
(src, dst, ports etc. multiple checks), you dont really know the exact 
output order is expected when grepping the full string.

What if nft changes output format (order) slightly in future, all my 
scripts will start breaking.

Hence this approach is not an elegant one.

Regards,

Amish.

Re: delete matching rule like it can be done in case of iptables

[G.W. Haywood]

Hi there,

On Tue, 7 Dec 2021, Daniel wrote:

> myhandle=$(echo `$nft -sa list chain $1 $2 $3 |grep -F $4|grep -oP '(# handle ).*'`|cut -d " " -f 3)

To me, quite apart from the reliance on a bunch of system utilities
which I'd really prefer to avoid in an operation of this kind, that
looks unnecessarily complex and rather fragile.

This whole discussion suggests that something is missing from nft.

-- 

73,
Ged.

Re: delete matching rule like it can be done in case of iptables

[Jozsef Kadlecsik]

Hi,

On Wed, 8 Dec 2021, G.W. Haywood wrote:

> On Tue, 7 Dec 2021, Daniel wrote:
> 
> > myhandle=$(echo `$nft -sa list chain $1 $2 $3 |grep -F $4|grep -oP '(#
> > handle ).*'`|cut -d " " -f 3)
> 
> To me, quite apart from the reliance on a bunch of system utilities 
> which I'd really prefer to avoid in an operation of this kind, that 
> looks unnecessarily complex and rather fragile.
> 
> This whole discussion suggests that something is missing from nft.

I know not all kind of rulesets can be managed thus, but I suggest to rely 
heavily on sets, maps in nft. In lots of cases one can achieve technically 
static rules while the ruleset is fully dynamic, because all the 
modifications happen in the sets/maps.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlecsik.jozsef@wigner.hu
PGP key : https://wigner.hu/~kadlec/pgp_public_key.txt
Address : Wigner Research Centre for Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: delete matching rule like it can be done in case of iptables

[G.W. Haywood]

Hi there,

On Wed, 8 Dec 2021, Jozsef Kadlecsik wrote:

> On Wed, 8 Dec 2021, G.W. Haywood wrote:
>> On Tue, 7 Dec 2021, Daniel wrote:
>>
>>> myhandle=$(echo `$nft -sa list chain $1 $2 $3 |grep -F $4|grep -oP '(#
>>> handle ).*'`|cut -d " " -f 3)
>>
>> To me, quite apart from the reliance on a bunch of system utilities
>> which I'd really prefer to avoid in an operation of this kind, that
>> looks unnecessarily complex and rather fragile.
>>
>> This whole discussion suggests that something is missing from nft.
>
> I know not all kind of rulesets can be managed thus, but I suggest to rely
> heavily on sets, maps in nft. In lots of cases one can achieve technically
> static rules while the ruleset is fully dynamic, because all the
> modifications happen in the sets/maps.

With your help, as you know, I already do that using ipsets - but the few
'technically static' rules are all iptables rules.  I have tried very hard
to find a reason to switch from iptables to nft, but so far I have failed
(a) fully to grasp the nft syntax and (b) to find that (e|i)llusive reason. :)

-- 

73,
Ged.

Re: delete matching rule like it can be done in case of iptables

[Amish]

On 08/12/21 15:25, G.W. Haywood wrote:
> Hi there,
>
> On Tue, 7 Dec 2021, Daniel wrote:
>
>> myhandle=$(echo `$nft -sa list chain $1 $2 $3 |grep -F $4|grep -oP 
>> '(# handle ).*'`|cut -d " " -f 3)
>
> To me, quite apart from the reliance on a bunch of system utilities
> which I'd really prefer to avoid in an operation of this kind, that
> looks unnecessarily complex and rather fragile.
>
> This whole discussion suggests that something is missing from nft.

Yes, its a blocker for me.

As a workaround I switched from iptables-legacy to iptables-nft. (So 
atleast I am using nft backend)

Irony is that iptables (nft backend) can actually delete the matching 
rule (via -D option) but native nft itself can not delete a matching rule.

Regards

Amish.