Re: Reload IPtables

Re: Reload IPtables

[Neal P. Murphy]

On Mon, 28 Jun 2021 10:43:10 +0100
Kerin Millar <kfm@plushkava.net> wrote:

> Now you benefit from atomicity (the rules will either be committed at once, in full, or not at all) and proper error handling (the exit status value of iptables-restore is meaningful and acted upon). Further, should you prefer to indent the body of the heredoc, you may write <<-EOF, though only leading tab characters will be stripped out.
> 

[minor digression]

Is iptables-restore truly atomic in *all* cases? Some years ago, I found through experimentation that some rules were 'lost' when restoring more than 25 000 rules. If I placed a COMMIT every 20 000 rules or so, then all rules would be properly loaded. I think COMMITs break atomicity. I tested with 100k to 1M rules. I was comparing the efficiency of iptables-restore with another tool that read from STDIN; the other tool was about 5% more efficient.

Re: Reload IPtables

[Pablo Neira Ayuso]

On Mon, Jun 28, 2021 at 10:02:41PM -0400, Neal P. Murphy wrote:
> On Mon, 28 Jun 2021 10:43:10 +0100
> Kerin Millar <kfm@plushkava.net> wrote:
> 
> > Now you benefit from atomicity (the rules will either be committed at once, in full, or not at all) and proper error handling (the exit status value of iptables-restore is meaningful and acted upon). Further, should you prefer to indent the body of the heredoc, you may write <<-EOF, though only leading tab characters will be stripped out.
> > 
> 
> [minor digression]
> 
> Is iptables-restore truly atomic in *all* cases?

Packets either see the old table or the new table, no intermediate
ruleset state is exposed to packet path.

> Some years ago, I found through experimentation that some rules were
> 'lost' when restoring more than 25 000 rules.

Could you specify kernel and userspace versions? Rules are not 'lost'
when restoring large rulesets.

> If I placed a COMMIT every 20 000 rules or so, then all rules would
> be properly loaded. I think COMMITs break atomicity.

Why are you placing COMMIT in every few rules 20 000 rules?

> I tested with 100k to 1M rules.

iptables is handling very large rulesets already.

> I was comparing the efficiency of iptables-restore with another tool
> that read from STDIN; the other tool was about 5% more efficient.

Could you please specify what other tool are you refering to?

iptables-restore is the best practise to restore your ruleset.

You should also iptables-restore to perform incremental updates via
--noflush.

Re: Reload IPtables

[slow_speed]

On 6/28/21 10:02 PM, Neal P. Murphy wrote:
> On Mon, 28 Jun 2021 10:43:10 +0100
> Kerin Millar <kfm@plushkava.net> wrote:
> 
>> Now you benefit from atomicity (the rules will either be committed at once, in full, or not at all) and proper error handling (the exit status value of iptables-restore is meaningful and acted upon). Further, should you prefer to indent the body of the heredoc, you may write <<-EOF, though only leading tab characters will be stripped out.
>>
> 
> [minor digression]
> 
> Is iptables-restore truly atomic in *all* cases? Some years ago, I found through experimentation that some rules were 'lost' when restoring more than 25 000 rules. If I placed a COMMIT every 20 000 rules or so, then all rules would be properly loaded. I think COMMITs break atomicity. I tested with 100k to 1M rules. I was comparing the efficiency of iptables-restore with another tool that read from STDIN; the other tool was about 5% more efficient.
> 

Please explain why you might have so many rules.  My server is pushing 
it at a dozen.

Re: Reload IPtables

[Reindl Harald]

Am 29.06.21 um 16:52 schrieb slow_speed@att.net:
> 
> 
> On 6/28/21 10:02 PM, Neal P. Murphy wrote:
>> On Mon, 28 Jun 2021 10:43:10 +0100
>> Kerin Millar <kfm@plushkava.net> wrote:
>>
>>> Now you benefit from atomicity (the rules will either be committed at 
>>> once, in full, or not at all) and proper error handling (the exit 
>>> status value of iptables-restore is meaningful and acted upon). 
>>> Further, should you prefer to indent the body of the heredoc, you may 
>>> write <<-EOF, though only leading tab characters will be stripped out.
>>>
>>
>> [minor digression]
>>
>> Is iptables-restore truly atomic in *all* cases? Some years ago, I 
>> found through experimentation that some rules were 'lost' when 
>> restoring more than 25 000 rules. If I placed a COMMIT every 20 000 
>> rules or so, then all rules would be properly loaded. I think COMMITs 
>> break atomicity. I tested with 100k to 1M rules. I was comparing the 
>> efficiency of iptables-restore with another tool that read from STDIN; 
>> the other tool was about 5% more efficient.
>>
> 
> Please explain why you might have so many rules.  My server is pushing 
> it at a dozen

likely because people don't use "ipset" and "chains" instead repeat the 
same stuff again and again so that every single package has to travel 
over hundrets and thousands of rules :-)

Re: Reload IPtables

[slow_speed]

On 6/29/21 11:18 AM, Reindl Harald wrote:
> 
> 
> Am 29.06.21 um 16:52 schrieb slow_speed@att.net:
>>
>>
>> On 6/28/21 10:02 PM, Neal P. Murphy wrote:
>>> On Mon, 28 Jun 2021 10:43:10 +0100
>>> Kerin Millar <kfm@plushkava.net> wrote:
>>>
>>>> Now you benefit from atomicity (the rules will either be committed 
>>>> at once, in full, or not at all) and proper error handling (the exit 
>>>> status value of iptables-restore is meaningful and acted upon). 
>>>> Further, should you prefer to indent the body of the heredoc, you 
>>>> may write <<-EOF, though only leading tab characters will be 
>>>> stripped out.
>>>>
>>>
>>> [minor digression]
>>>
>>> Is iptables-restore truly atomic in *all* cases? Some years ago, I 
>>> found through experimentation that some rules were 'lost' when 
>>> restoring more than 25 000 rules. If I placed a COMMIT every 20 000 
>>> rules or so, then all rules would be properly loaded. I think COMMITs 
>>> break atomicity. I tested with 100k to 1M rules. I was comparing the 
>>> efficiency of iptables-restore with another tool that read from 
>>> STDIN; the other tool was about 5% more efficient.
>>>
>>
>> Please explain why you might have so many rules.  My server is pushing 
>> it at a dozen
> 
> likely because people don't use "ipset" and "chains" instead repeat the 
> same stuff again and again so that every single package has to travel 
> over hundrets and thousands of rules :-)

Exactly my thoughts.  Of course I understand that there may be long 
lists in some odd situations, but I wonder what kind of server is being 
referenced.

Re: Reload IPtables

[Neal P. Murphy]

On Tue, 29 Jun 2021 10:37:18 +0200
Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> On Mon, Jun 28, 2021 at 10:02:41PM -0400, Neal P. Murphy wrote:
> > On Mon, 28 Jun 2021 10:43:10 +0100
> > Kerin Millar <kfm@plushkava.net> wrote:
> >   
> > > Now you benefit from atomicity (the rules will either be committed at once, in full, or not at all) and proper error handling (the exit status value of iptables-restore is meaningful and acted upon). Further, should you prefer to indent the body of the heredoc, you may write <<-EOF, though only leading tab characters will be stripped out.
> > >   
> > 
> > [minor digression]
> > 
> > Is iptables-restore truly atomic in *all* cases?  
> 
> Packets either see the old table or the new table, no intermediate
> ruleset state is exposed to packet path.
> 
> > Some years ago, I found through experimentation that some rules were
> > 'lost' when restoring more than 25 000 rules.  
> 
> Could you specify kernel and userspace versions? Rules are not 'lost'
> when restoring large rulesets.

This goes back to late '10, early '11: linux 2.6.35, iptables 1.4.10; possibly even earlier releases. I don't recall trying the experiment since.

> 
> > If I placed a COMMIT every 20 000 rules or so, then all rules would
> > be properly loaded. I think COMMITs break atomicity.  
> 
> Why are you placing COMMIT in every few rules 20 000 rules?

If I recall correctly, when I attempted to restore/load more than 'N' rules or so back then, a few rules at that 'N' 'boundary' would not be in the final iptables ruleset. IIRC, it didn't matter how quickly or slowly I added the rules. By including a COMMIT every 20k rules or so, all of the rules I loaded would be added, be it 10k or 100k (or more).

There was also a vmalloc failure up around 130k-150k rules.

A couple references:
  - https://marc.info/?l=netfilter&m=133814281919741&w=2
  - https://community.smoothwall.org/forum/viewtopic.php?p=292390#p292390

> 
> > I tested with 100k to 1M rules.  
> 
> iptables is handling very large rulesets already.
> 
> > I was comparing the efficiency of iptables-restore with another tool
> > that read from STDIN; the other tool was about 5% more efficient.  
> 
> Could you please specify what other tool are you refering to?

The tool is ipbatch (part of Smoothwall Express). It was written because, at least back then, iptables was not able to read from STDIN. Smoothwall Express reads its config/settings files and generates the firewall rules on the fly and pipes them to ipbatch. Also, the rules are generated by function/application, not by table; iptables-restore would not be usable without major changes to the way Smoothwall Express generates its firewall rules.

Neal

Re: Reload IPtables

[Neal P. Murphy]

On Tue, 29 Jun 2021 10:52:38 -0400
slow_speed@att.net wrote:

> On 6/28/21 10:02 PM, Neal P. Murphy wrote:
> > On Mon, 28 Jun 2021 10:43:10 +0100
> > Kerin Millar <kfm@plushkava.net> wrote:
> >   
> >> Now you benefit from atomicity (the rules will either be committed at once, in full, or not at all) and proper error handling (the exit status value of iptables-restore is meaningful and acted upon). Further, should you prefer to indent the body of the heredoc, you may write <<-EOF, though only leading tab characters will be stripped out.
> >>  
> > 
> > [minor digression]
> > 
> > Is iptables-restore truly atomic in *all* cases? Some years ago, I found through experimentation that some rules were 'lost' when restoring more than 25 000 rules. If I placed a COMMIT every 20 000 rules or so, then all rules would be properly loaded. I think COMMITs break atomicity. I tested with 100k to 1M rules. I was comparing the efficiency of iptables-restore with another tool that read from STDIN; the other tool was about 5% more efficient.
> >   
> 
> Please explain why you might have so many rules.  My server is pushing 
> it at a dozen.

In this particular case, I needed a lot of rules to obtain a reasonable estimate of the relative efficiencies of the two programs, enough rules to mask program initialization time.

I agree that tens of thousands of rules is probably outrageous for most purposes. A good firewall--that restricts outgoing conns, handles TESTNET addresses, isolates traffic between several internal LANs while allowing certain traffic to pass between them, employs NAT, and uses CONNMARKs to tag traffic and to provide traffic stats--can easily use over 600 rules, if not a good deal more. Yet a 1.6GHz Atom N270 can still process four saturated gigE NICs with plenty of cycles to spare with those 600 rules.

N