From: Phil Sutter <phil@nwl.cc>
To: Pablo Neira Ayuso <pablo@netfilter.org>
Cc: netfilter-devel@vger.kernel.org
Subject: Re: [iptables PATCH v3 0/7] Improve xtables-restore performance
Date: Thu, 31 Oct 2019 18:19:47 +0100 [thread overview]
Message-ID: <20191031171947.GF8531@orbyte.nwl.cc> (raw)
In-Reply-To: <20191031150234.osfnsa2emuvhocrc@salvia>
On Thu, Oct 31, 2019 at 04:02:34PM +0100, Pablo Neira Ayuso wrote:
> On Thu, Oct 24, 2019 at 06:37:05PM +0200, Phil Sutter wrote:
> > This series speeds up xtables-restore calls with --noflush (typically
> > used to batch a few commands for faster execution) by preliminary input
> > inspection.
> >
> > Before, setting --noflush flag would inevitably lead to full cache
> > population. With this series in place, if input can be fully buffered
> > and no commands requiring full cache is contained, no initial cache
> > population happens and each rule parsed will cause fetching of cache
> > bits as required.
> >
> > The input buffer size is arbitrarily chosen to be 64KB.
> >
> > Patches one and two prepare code for patch three which moves the loop
> > content parsing each line of input into a separate function. The
> > reduction of code indenting is used by patch four which deals with
> > needless line breaks.
>
> For patches from 1 to 4 in this batch:
>
> Acked-by: Pablo Neira Ayuso <pablo@netfilter.org>
>
> > Patch five deals with another requirement of input buffering, namely
> > stripping newline characters from each line. This is not a problem by
> > itself, but add_param_to_argv() replaces them by nul-chars and so
> > strings stop being consistently terminated (some by a single, some by
> > two nul-chars).
> >
> > Patch six then finally adds the buffering and caching decision code.
> >
> > Patch seven is pretty unrelated but tests a specific behaviour of
> > *tables-restore I wasn't sure of at first.
>
> Do you have any number?
Yes, I wrote a small benchmark based on some Kubernetes use-case. It
measures loading of dumps like:
| *nat
| :KUBE-SVC-23 - [0:0]
| :KUBE-SEP-23 - [0:0]
| -A KUBE-HOOK ! -s 10.128.0.0/14 -d 172.30.108.136/32 -p tcp -m comment --comment \"openshift-controller-manager/controller-manager:https cluster IP\" -m tcp --dport 443 -j KUBE-MARK-MASQ
| -A KUBE-HOOK -d 172.30.108.136/32 -p tcp -m comment --comment \"openshift-controller-manager/controller-manager:https cluster IP\" -m tcp --dport 443 -j KUBE-SVC-23
| -A KUBE-SVC-23 -j KUBE-SEP-23
| -A KUBE-SEP-23 -s 10.128.0.38/32 -j KUBE-MARK-MASQ
| -A KUBE-SEP-23 -p tcp -m tcp -j DNAT --to-destination 10.128.0.38:8443
| COMMIT
Into a ruleset with increasing size (created by repeating the snippet above):
size (*100) | legacy | nft-pre | nft-post
---------------------------------------------------------
1 .0040366426 .0079313714 .0025598650
10 .0146918664 .0459193868 .0025134858
25 .0361553334 .1195503778 .0024202904
50 .0699177362 .2547542626 .0024351612
75 .1062593206 .4078182120 .0024362044
100 .1614045514 .5636617378 .0024195190
The graph says it all: http://nwl.cc/~n0-1/kube.png
Cheers, Phil
next prev parent reply other threads:[~2019-10-31 17:19 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-10-24 16:37 [iptables PATCH v3 0/7] Improve xtables-restore performance Phil Sutter
2019-10-24 16:37 ` [iptables PATCH v3 1/7] xtables-restore: Integrate restore callbacks into struct nft_xt_restore_parse Phil Sutter
2019-10-24 16:37 ` [iptables PATCH v3 2/7] xtables-restore: Introduce struct nft_xt_restore_state Phil Sutter
2019-10-24 16:37 ` [iptables PATCH v3 3/7] xtables-restore: Introduce line parsing function Phil Sutter
2019-10-24 16:37 ` [iptables PATCH v3 4/7] xtables-restore: Remove some pointless linebreaks Phil Sutter
2019-10-24 16:37 ` [iptables PATCH v3 5/7] xtables-restore: Allow lines without trailing newline character Phil Sutter
2019-10-24 16:37 ` [iptables PATCH v3 6/7] xtables-restore: Improve performance of --noflush operation Phil Sutter
2019-10-24 16:37 ` [iptables PATCH v3 7/7] tests: shell: Add ipt-restore/0007-flush-noflush_0 Phil Sutter
2019-10-31 15:02 ` [iptables PATCH v3 0/7] Improve xtables-restore performance Pablo Neira Ayuso
2019-10-31 17:19 ` Phil Sutter [this message]
2019-11-06 9:24 ` Pablo Neira Ayuso
2019-11-06 12:31 ` Phil Sutter
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20191031171947.GF8531@orbyte.nwl.cc \
--to=phil@nwl.cc \
--cc=netfilter-devel@vger.kernel.org \
--cc=pablo@netfilter.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).