netfilter-devel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: "Neal P. Murphy" <neal.p.murphy@alum.wpi.edu>
To: unlisted-recipients:; (no To-header on input)
Cc: netfilter-devel@vger.kernel.org, netfilter@vger.kernel.org
Subject: Re: Reload IPtables
Date: Wed, 30 Jun 2021 21:49:38 -0400	[thread overview]
Message-ID: <20210630214938.1396b4a0@playground> (raw)
In-Reply-To: <20210629083718.GA10943@salvia>

On Tue, 29 Jun 2021 10:37:18 +0200
Pablo Neira Ayuso <pablo@netfilter.org> wrote:

> On Mon, Jun 28, 2021 at 10:02:41PM -0400, Neal P. Murphy wrote:
> > On Mon, 28 Jun 2021 10:43:10 +0100
> > Kerin Millar <kfm@plushkava.net> wrote:
> >   
> > > Now you benefit from atomicity (the rules will either be committed at once, in full, or not at all) and proper error handling (the exit status value of iptables-restore is meaningful and acted upon). Further, should you prefer to indent the body of the heredoc, you may write <<-EOF, though only leading tab characters will be stripped out.
> > >   
> > 
> > [minor digression]
> > 
> > Is iptables-restore truly atomic in *all* cases?  
> 
> Packets either see the old table or the new table, no intermediate
> ruleset state is exposed to packet path.
> 
> > Some years ago, I found through experimentation that some rules were
> > 'lost' when restoring more than 25 000 rules.  
> 
> Could you specify kernel and userspace versions? Rules are not 'lost'
> when restoring large rulesets.

This goes back to late '10, early '11: linux 2.6.35, iptables 1.4.10; possibly even earlier releases. I don't recall trying the experiment since.

> 
> > If I placed a COMMIT every 20 000 rules or so, then all rules would
> > be properly loaded. I think COMMITs break atomicity.  
> 
> Why are you placing COMMIT in every few rules 20 000 rules?

If I recall correctly, when I attempted to restore/load more than 'N' rules or so back then, a few rules at that 'N' 'boundary' would not be in the final iptables ruleset. IIRC, it didn't matter how quickly or slowly I added the rules. By including a COMMIT every 20k rules or so, all of the rules I loaded would be added, be it 10k or 100k (or more).

There was also a vmalloc failure up around 130k-150k rules.

A couple references:
  - https://marc.info/?l=netfilter&m=133814281919741&w=2
  - https://community.smoothwall.org/forum/viewtopic.php?p=292390#p292390

> 
> > I tested with 100k to 1M rules.  
> 
> iptables is handling very large rulesets already.
> 
> > I was comparing the efficiency of iptables-restore with another tool
> > that read from STDIN; the other tool was about 5% more efficient.  
> 
> Could you please specify what other tool are you refering to?

The tool is ipbatch (part of Smoothwall Express). It was written because, at least back then, iptables was not able to read from STDIN. Smoothwall Express reads its config/settings files and generates the firewall rules on the fly and pipes them to ipbatch. Also, the rules are generated by function/application, not by table; iptables-restore would not be usable without major changes to the way Smoothwall Express generates its firewall rules.

Neal

  reply	other threads:[~2021-07-01  1:49 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <08f069e3-914f-204a-dfd6-a56271ec1e55.ref@att.net>
     [not found] ` <08f069e3-914f-204a-dfd6-a56271ec1e55@att.net>
     [not found]   ` <4ac5ff0d-4c6f-c963-f2c5-29154e0df24b@hajes.org>
     [not found]     ` <6430a511-9cb0-183d-ed25-553b5835fa6a@att.net>
     [not found]       ` <877683bf-6ea4-ca61-ba41-5347877d3216@thelounge.net>
     [not found]         ` <d2156e5b-2be9-c0cf-7f5b-aaf8b81769f8@att.net>
     [not found]           ` <f5314629-8a08-3b5f-cfad-53bf13483ec3@hajes.org>
     [not found]             ` <adc28927-724f-2cdb-ca6a-ff39be8de3ba@thelounge.net>
     [not found]               ` <96559e16-e3a6-cefd-6183-1b47f31b9345@hajes.org>
     [not found]                 ` <16b55f10-5171-590f-f9d2-209cfaa7555d@thelounge.net>
     [not found]                   ` <54e70d0a-0398-16e4-a79e-ec96a8203b22@tana.it>
     [not found]                     ` <f0daea91-4d12-1605-e6df-e7f95ba18cac@thelounge.net>
     [not found]                       ` <8395d083-022b-f6f7-b2d3-e2a83b48c48a@tana.it>
     [not found]                         ` <20210628104310.61bd287ff147a59b12e23533@plushkava.net>
2021-06-29  2:02                           ` Reload IPtables Neal P. Murphy
     [not found]                             ` <20210629083652.GA10896@salvia>
2021-06-29  8:37                               ` Pablo Neira Ayuso
2021-07-01  1:49                                 ` Neal P. Murphy [this message]
2021-06-29 14:52                             ` slow_speed
2021-06-29 15:18                               ` Reindl Harald
2021-06-29 16:50                                 ` slow_speed
2021-07-01  2:31                               ` Neal P. Murphy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210630214938.1396b4a0@playground \
    --to=neal.p.murphy@alum.wpi.edu \
    --cc=netfilter-devel@vger.kernel.org \
    --cc=netfilter@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).