Atomic changes to IP sets

Atomic changes to IP sets

[Anna Fischer]

Hi,

I'm using ip sets in my iptables firwall rules. I don't just use those for firewalling, but also for packet mangling (marking). Now I'm quite frequently changing these sets and also the firewall rules. I know that I can atomically switch firewall rules by using iptables-restore. But how can I make changes to ip sets atomic? It seems to be as if packets are always passing the firewall whilst I do ipset commands. Currently I flush all ip sets, and then rebuild them. I understand this will cause problems because at times my ip sets are empty and so the firewall does not behave how I want it to behave. But what is the correct way to atomically update ip sets? I have seen that there is a command to swap an ip set. So I would build up a new set, then swap it with the old one, and then delete the
  old one. Is that the right way of changing ip sets? The other option would be to create a whole new set of ipsets and the a new set of iptables rules with these sets, and then I use iptables-restore to atomically switch the full firewall table. This seems like quite a bit of overkill though, doesn't it? Does anyone have an idea about how to best work with ip sets without building glitches into my firewall whilst reconfiguring ip sets?

Thanks for any pointers.

Anna

Re: Atomic changes to IP sets

[Nikolay S.]

В Пн, 13/04/2015 в 08:23 +0000, Anna Fischer пишет:
> Hi,
> 
> I'm using ip sets in my iptables firwall rules. I don't just use those for firewalling, but also for packet mangling (marking). Now I'm quite frequently changing these sets and also the firewall rules. I know that I can atomically switch firewall rules by using iptables-restore. But how can I make changes to ip sets atomic? It seems to be as if packets are always passing the firewall whilst I do ipset commands. Currently I flush all ip sets, and then rebuild them. I understand this will cause problems because at times my ip sets are empty and so the firewall does not behave how I want it to behave. But what is the correct way to atomically update ip sets? I have seen that there is a command to swap an ip set. So I would build up a new set, then swap it with the old one, and then delete the old one. Is that the right way of changing ip sets? The other option would be to create a whole new set of ipsets and the a new set of iptables rules with these sets, and then I use iptables-restore to atomically switch the full firewall table. This seems like quite a bit of overkill though, doesn't it? Does anyone have an idea about how to best work with ip sets without building glitches into my firewall whilst reconfiguring ip sets?
> 
> Thanks for any pointers.

Did you try ipset {save,restore}? This will pass all the commands in one
transaction through netlink.

> Anna
> 
> 
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Atomic changes to IP sets

[Koen Zandberg]

On 13/04/15 10:23, Anna Fischer wrote:
> Hi,
>
> I'm using ip sets in my iptables firwall rules. I don't just use those for firewalling, but also for packet mangling (marking). Now I'm quite frequently changing these sets and also the firewall rules. I know that I can atomically switch firewall rules by using iptables-restore. But how can I make changes to ip sets atomic? It seems to be as if packets are always passing the firewall whilst I do ipset commands. Currently I flush all ip sets, and then rebuild them. I understand this will cause problems because at times my ip sets are empty and so the firewall does not behave how I want it to behave. But what is the correct way to atomically update ip sets? I have seen that there is a command to swap an ip set. So I would build up a new set, then swap it with the old one, and then delete t
 he old one. Is that the right way of changing ip sets? The other option would be to create a whole new set of ipsets and the a new set of iptables rules with these sets, and then I use iptables-restor
>   e to atomically switch the full firewall table. This seems like quite a bit of overkill though, doesn't it? Does anyone have an idea about how to best work with ip sets without building glitches into my firewall whilst reconfiguring ip sets?
>
> Thanks for any pointers.
>
> Anna
The way I learned to do this atomically was by creating a new set and 
using "ipset swap $OLDLIST $NEWLIST" to swap your old set with your new set.

Re: Atomic changes to IP sets

[Neal Murphy]

On Monday, April 13, 2015 06:31:47 AM Koen Zandberg wrote:
> On 13/04/15 10:23, Anna Fischer wrote:
> > Hi,
> > 
> > I'm using ip sets in my iptables firwall rules. I don't just use those
> > for firewalling, but also for packet mangling (marking). Now I'm quite
> > frequently changing these sets and also the firewall rules. I know that
> > I can atomically switch firewall rules by using iptables-restore. But
> > how can I make changes to ip sets atomic? It seems to be as if packets
> > are always passing the firewall whilst I do ipset commands. Currently I
> > flush all ip sets, and then rebuild them. I understand this will cause
> > problems because at times my ip sets are empty and so the firewall does
> > not behave how I want it to behave. But what is the correct way to
> > atomically update ip sets? I have seen that there is a command to swap
> > an ip set. So I would build up a new set, then swap it with the old one,
> > and then delete the old one. Is that the right way of changing ip sets?
> > The other option would be to create a whole new set of ipsets and the a
> > new set of iptables rules with these sets, and then I use
> > iptables-restor
> > 
> >   e to atomically switch the full firewall table. This seems like quite a
> >   bit of overkill though, doesn't it? Does anyone have an idea about how
> >   to best work with ip sets without building glitches into my firewall
> >   whilst reconfiguring ip sets?
> > 
> > Thanks for any pointers.
> > 
> > Anna
> 
> The way I learned to do this atomically was by creating a new set and
> using "ipset swap $OLDLIST $NEWLIST" to swap your old set with your new
> set.

I believe it waits for relative quiesence, then atomically swaps the names of 
the two sets. So don't forget to delete set $NEWLIST after the swap (which is 
now the old set).

Re: Atomic changes to IP sets

[Paul Robert Marino]

take a look at my tool ipset-manager
https://github.com/prmarino1/HadrianWall/tree/master/Tools/IPSet-Manager

This uses a swap method which can update changes sets without deleting them.
documentation is embedded in the script in POD format. It can be
extracted with pod2man, pod2text, pod2html, etc.

It requires two Perl 5 modules  XML::Twig and Getopt::Long





On Mon, Apr 13, 2015 at 12:20 PM, Neal Murphy
<neal.p.murphy@alum.wpi.edu> wrote:
> On Monday, April 13, 2015 06:31:47 AM Koen Zandberg wrote:
>> On 13/04/15 10:23, Anna Fischer wrote:
>> > Hi,
>> >
>> > I'm using ip sets in my iptables firwall rules. I don't just use those
>> > for firewalling, but also for packet mangling (marking). Now I'm quite
>> > frequently changing these sets and also the firewall rules. I know that
>> > I can atomically switch firewall rules by using iptables-restore. But
>> > how can I make changes to ip sets atomic? It seems to be as if packets
>> > are always passing the firewall whilst I do ipset commands. Currently I
>> > flush all ip sets, and then rebuild them. I understand this will cause
>> > problems because at times my ip sets are empty and so the firewall does
>> > not behave how I want it to behave. But what is the correct way to
>> > atomically update ip sets? I have seen that there is a command to swap
>> > an ip set. So I would build up a new set, then swap it with the old one,
>> > and then delete the old one. Is that the right way of changing ip sets?
>> > The other option would be to create a whole new set of ipsets and the a
>> > new set of iptables rules with these sets, and then I use
>> > iptables-restor
>> >
>> >   e to atomically switch the full firewall table. This seems like quite a
>> >   bit of overkill though, doesn't it? Does anyone have an idea about how
>> >   to best work with ip sets without building glitches into my firewall
>> >   whilst reconfiguring ip sets?
>> >
>> > Thanks for any pointers.
>> >
>> > Anna
>>
>> The way I learned to do this atomically was by creating a new set and
>> using "ipset swap $OLDLIST $NEWLIST" to swap your old set with your new
>> set.
>
> I believe it waits for relative quiesence, then atomically swaps the names of
> the two sets. So don't forget to delete set $NEWLIST after the swap (which is
> now the old set).
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: AW: Atomic changes to IP sets

[Nikolay S.]

В Вт, 14/04/2015 в 05:36 +0000, Anna Fischer пишет:
> > Hi,
> >
> > I'm using ip sets in my iptables firwall rules. I don't just use those for firewalling, but also for packet mangling (marking). Now I'm quite frequently changing these sets and also the firewall rules. I know that I can atomically switch firewall rules by using iptables-restore. But how can I make changes to ip sets atomic? It seems to be as if packets are always passing the firewall whilst I do ipset commands. Currently I flush all ip sets, and then rebuild them. I understand this will cause problems because at times my ip sets are empty and so the firewall does not behave how I want it to behave. But what is the correct way to atomically update ip sets? I have seen that there is a command to swap an ip set. So I would build up a new set, then swap it with the old one, and then delete the old one. Is that the right way of changing ip sets? The other option would be to create a whole new set of ipsets and the a new set of iptables rules with these sets, and then I use iptables-restore to atomically switch the full firewall table. This seems like quite a bit of overkill though, doesn't it? Does anyone have an idea about how to best work with ip sets without building glitches into my firewall whilst reconfiguring ip sets?
> >
> > Thanks for any pointers.
> 
> > Did you try ipset {save,restore}? This will pass all the commands in one
> > transaction through netlink.
> 
> I have tried ipset {save, restore} but it does not seem to work if I have a large set of commands to restore. Is there a limit on the number of commands I can save/restore?
> 
> Thanks, Anna

I don't know of such a limit. Quick test with adding 100000 addresess
through restore gives correct result here