ipset -R

ipset -R

[Mr Dash Four]

When I execute 'ipset -R < some_file' ipset ignores lines with the '-D' 
option specified. Is that intentional?

I am trying to execute a script file creating 'pinholes' in (previously 
built) ipsets, but, as it stands, I have to execute a shell script 
containing 'ipset -D' for every pinhole/range I am interested in, which 
isn't very convenient. Ideally I'd like for ipset to honour the '-D' 
option in a restore file.

The reason I need this is because I am defining ipset ranges coming from 
the geoip database and later, with the '-D' option in a script, I am 
trying to create the pinholes - more convenient since ipset has a very 
nice feature to re-adjust the ipset ranges automatically after delete, 
instead of me using endless '-A' statements adding the resulting ipset 
sub-ranges.

Re: ipset -R

[Jozsef Kadlecsik]

On Wed, 23 Feb 2011, Mr Dash Four wrote:

> When I execute 'ipset -R < some_file' ipset ignores lines with the '-D' option
> specified. Is that intentional?

Deletion is not ignored in restore mode. The commands which are not 
allowed in restore mode are restore, help and version.

> I am trying to execute a script file creating 'pinholes' in (previously built)
> ipsets, but, as it stands, I have to execute a shell script containing 'ipset
> -D' for every pinhole/range I am interested in, which isn't very convenient.
> Ideally I'd like for ipset to honour the '-D' option in a restore file.
> 
> The reason I need this is because I am defining ipset ranges coming from the
> geoip database and later, with the '-D' option in a script, I am trying to
> create the pinholes - more convenient since ipset has a very nice feature to
> re-adjust the ipset ranges automatically after delete, instead of me using
> endless '-A' statements adding the resulting ipset sub-ranges.

The iptreemap type of ipset 4.x had the feature you are referring here.

The iptree and iptreemap types are not implemented in ipset 5.x. However 
you can achieve the same effect by using two sets, the first one for the 
pinholes and the second one for the networks.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipset -R

[Mr Dash Four]

> The iptreemap type of ipset 4.x had the feature you are referring here.
>
> The iptree and iptreemap types are not implemented in ipset 5.x. However 
> you can achieve the same effect by using two sets, the first one for the 
> pinholes and the second one for the networks.
>   
It would be a bit cumbersome to achieve that.

I use the geoip database to construct, among other things, a permanent 
block lists (mainly based on country of origin, but there are other 
factors which I won't go in here) and these sets are only present in my 
block chains (there is also a bit of logic involved, but that is another 
set of issues altogether). I also have a separate list with host ip 
addresses/ranges (my 'whitelist' so to speak) against which I 
subsequently execute the delete statements to create the pinholes 
(another useful ipset feature is that it silently ignores delete 
requests if the ip address/range is not present in the set).

As you know the geoip database regularly changes (so does, albeit 
occasionally, the country of origin of various ip address ranges), so I 
normally execute an automated set of scripts to update the database, 
pick the right address ranges for blocking, construct-and-replace my 
ipsets in the 3 or so blocking chains I have on the main firewall and 
then apply the deletion statements to the same sets to create the pinholes.

The problem I see with your approach above is that I would have to 
explicitly grant some sort of access to my 'whitelist' addresses (those 
with which I create the pinholes), which isn't possible as I do not know 
which ports (or port ranges) would need to be enabled - that depends on 
various factors which I cannot always control. Besides, I would like the 
'whitelist' addresses to be traversed further down the ip chain as there 
may be other rules which catch them. In other words, I cannot explicitly 
allow access to the whitelist addresses as you are suggesting above (so 
is my understanding).

The ideal scenario, as I pointed out above, would be to use the 
whitelist to create the pinholes (i.e. execute a deletion against 
already registered sets) without explicitly allowing access to those ip 
addresses/ranges.

Another pretty good alternative would be (if I am, somehow, able) to 
'subtract' my whitelist members from the blacklisted ones and then use 
the resulting set (or sets), but as far as I know I can't do that in 
ipset, can I?

Re: ipset -R

[Pandu Poluan]

What if:

-m set ! --match-set $WHITELIST src -m set --match-set $BLACKLIST src -j DROP

The first -m set (with the "!") punches 'holes' into the 2nd. You can
follow up with

-m set --match-set $WHITELIST src -j ACCEPT

Rgds,


On 2011-02-24, Mr Dash Four <mr.dash.four@googlemail.com> wrote:
>
>> The iptreemap type of ipset 4.x had the feature you are referring here.
>>
>> The iptree and iptreemap types are not implemented in ipset 5.x. However
>> you can achieve the same effect by using two sets, the first one for the
>> pinholes and the second one for the networks.
>>
> It would be a bit cumbersome to achieve that.
>
> I use the geoip database to construct, among other things, a permanent
> block lists (mainly based on country of origin, but there are other
> factors which I won't go in here) and these sets are only present in my
> block chains (there is also a bit of logic involved, but that is another
> set of issues altogether). I also have a separate list with host ip
> addresses/ranges (my 'whitelist' so to speak) against which I
> subsequently execute the delete statements to create the pinholes
> (another useful ipset feature is that it silently ignores delete
> requests if the ip address/range is not present in the set).
>
> As you know the geoip database regularly changes (so does, albeit
> occasionally, the country of origin of various ip address ranges), so I
> normally execute an automated set of scripts to update the database,
> pick the right address ranges for blocking, construct-and-replace my
> ipsets in the 3 or so blocking chains I have on the main firewall and
> then apply the deletion statements to the same sets to create the pinholes.
>
> The problem I see with your approach above is that I would have to
> explicitly grant some sort of access to my 'whitelist' addresses (those
> with which I create the pinholes), which isn't possible as I do not know
> which ports (or port ranges) would need to be enabled - that depends on
> various factors which I cannot always control. Besides, I would like the
> 'whitelist' addresses to be traversed further down the ip chain as there
> may be other rules which catch them. In other words, I cannot explicitly
> allow access to the whitelist addresses as you are suggesting above (so
> is my understanding).
>
> The ideal scenario, as I pointed out above, would be to use the
> whitelist to create the pinholes (i.e. execute a deletion against
> already registered sets) without explicitly allowing access to those ip
> addresses/ranges.
>
> Another pretty good alternative would be (if I am, somehow, able) to
> 'subtract' my whitelist members from the blacklisted ones and then use
> the resulting set (or sets), but as far as I know I can't do that in
> ipset, can I?
>
> --
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


-- 
--
Pandu E Poluan - IT Optimizer
My website: http://pandu.poluan.info/

Re: ipset -R

[Pandu Poluan]

Okay, the 2nd rule should be:

-m set --match-set $WHITELIST -j $WHITELIST_HANDLING_CHAIN

but I think you'd get my drift.

Rgds,


On 2011-02-24, Pandu Poluan <pandu@poluan.info> wrote:
> What if:
>
> -m set ! --match-set $WHITELIST src -m set --match-set $BLACKLIST src -j
> DROP
>
> The first -m set (with the "!") punches 'holes' into the 2nd. You can
> follow up with
>
> -m set --match-set $WHITELIST src -j ACCEPT
>
> Rgds,
>
>
> On 2011-02-24, Mr Dash Four <mr.dash.four@googlemail.com> wrote:
>>
>>> The iptreemap type of ipset 4.x had the feature you are referring here.
>>>
>>> The iptree and iptreemap types are not implemented in ipset 5.x. However
>>> you can achieve the same effect by using two sets, the first one for the
>>> pinholes and the second one for the networks.
>>>
>> It would be a bit cumbersome to achieve that.
>>
>> I use the geoip database to construct, among other things, a permanent
>> block lists (mainly based on country of origin, but there are other
>> factors which I won't go in here) and these sets are only present in my
>> block chains (there is also a bit of logic involved, but that is another
>> set of issues altogether). I also have a separate list with host ip
>> addresses/ranges (my 'whitelist' so to speak) against which I
>> subsequently execute the delete statements to create the pinholes
>> (another useful ipset feature is that it silently ignores delete
>> requests if the ip address/range is not present in the set).
>>
>> As you know the geoip database regularly changes (so does, albeit
>> occasionally, the country of origin of various ip address ranges), so I
>> normally execute an automated set of scripts to update the database,
>> pick the right address ranges for blocking, construct-and-replace my
>> ipsets in the 3 or so blocking chains I have on the main firewall and
>> then apply the deletion statements to the same sets to create the
>> pinholes.
>>
>> The problem I see with your approach above is that I would have to
>> explicitly grant some sort of access to my 'whitelist' addresses (those
>> with which I create the pinholes), which isn't possible as I do not know
>> which ports (or port ranges) would need to be enabled - that depends on
>> various factors which I cannot always control. Besides, I would like the
>> 'whitelist' addresses to be traversed further down the ip chain as there
>> may be other rules which catch them. In other words, I cannot explicitly
>> allow access to the whitelist addresses as you are suggesting above (so
>> is my understanding).
>>
>> The ideal scenario, as I pointed out above, would be to use the
>> whitelist to create the pinholes (i.e. execute a deletion against
>> already registered sets) without explicitly allowing access to those ip
>> addresses/ranges.
>>
>> Another pretty good alternative would be (if I am, somehow, able) to
>> 'subtract' my whitelist members from the blacklisted ones and then use
>> the resulting set (or sets), but as far as I know I can't do that in
>> ipset, can I?
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe netfilter" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>
> --
> --
> Pandu E Poluan - IT Optimizer
> My website: http://pandu.poluan.info/
>


-- 
--
Pandu E Poluan - IT Optimizer
My website: http://pandu.poluan.info/

Re: ipset -R

[Mr Dash Four]

> What if:
>
> -m set ! --match-set $WHITELIST src -m set --match-set $BLACKLIST src -j DROP
>   
Yeah, never thought of that before, thanks, though this would increase 
processing time as I have to include that extra 'whitelist' non-match in 
every iptables '--match-set' statement I have where my blacklist sets 
are involved (and I have quite a lot of them!). It would be better if I 
just delete the whitelist members permanently once at the beginning and 
just forget about it (though as Jozsef hinted yesterday that may not 
work in 5.x and 6 any more - see below).

The second statement (in both of your posts) won't be needed as I simply 
propagate the resulting non-match throughout the ip chain (the packets 
therefore could be dropped for a variety of other reasons regardless of 
whether they belong to the 'whitelist' or not, which is what I wanted to 
have in a first place).

Jozsef, you mentioned yesterday that ipset 5.x (and I presume 6.0+ also) 
does not implement ip range 'readjustment' any more. If so, what happens 
when I list the set below:

ipset -F test
ipset -A test 10.1.1.0/24
ipset -D test 10.1.1.12
ipset -L test

> -m set --match-set $WHITELIST src -j ACCEPT
>   
That won't be needed as I simply want the packet to be propagated down 
the ip chain as normal (the packet could be dropped for other reasons 
and the fact that it belongs to the whitelist won't matter - its sole 
purpose is for punching holes in my blacklist, then it should be treated 
as any other packet).

Re: ipset -R

[Jozsef Kadlecsik]

On Thu, 24 Feb 2011, Mr Dash Four wrote:

> Jozsef, you mentioned yesterday that ipset 5.x (and I presume 6.0+ also) does
> not implement ip range 'readjustment' any more. If so, what happens when I
> list the set below:
> 
> ipset -F test
> ipset -A test 10.1.1.0/24
> ipset -D test 10.1.1.12
> ipset -L test

It depends on the type of the set. With hash:ip type it's simple

# ipset -N test hash:ip
# ipset -A test 10.1.1.0/24 
# ipset -D test 10.1.1.12 
# ipset -L test | grep 10.1.1.12
10.1.1.125                                                                      
10.1.1.126                                                                      
10.1.1.124                                                                      
10.1.1.122                                                                      
10.1.1.129                                                                      
10.1.1.123                                                                      
10.1.1.121                                                                      
10.1.1.120                                                                      
10.1.1.127                                                                      
10.1.1.128 

However with hash:net type

# ipset -N test hash:net
# ipset -A test 10.1.1.0/24
# ipset -D test 10.1.1.12
ipset v6.0: Element cannot be deleted from the set: it's not added

and that's also right, because the hash types do not magically figure out 
overlapping ranges and collapse those or break up ranges into parts when 
deleting intersecting elements.
 
The hash:*net* types could be extended to store non-matching elements, 
something like this:

# ipset -N test hash:net
# ipset -A test 10.1.1.0/24
# ipset -A test 10.1.1.12 --nomatch

That way overlapping entries with different "access right" could be stored 
in a single set. But any coding needs time and testing.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipset -R

[Mr Dash Four]

> However with hash:net type
>
> # ipset -N test hash:net
> # ipset -A test 10.1.1.0/24
> # ipset -D test 10.1.1.12
> ipset v6.0: Element cannot be deleted from the set: it's not added
>   
Well, that's plain wrong, isn't it? The 'element' 10.1.1.12 does exist 
and it is added (albeit implicitly as part of 10.1.1.0/24). I also 
presume 'ipset -T test 10.1.1.12' will return a positive result, so 
there is something which isn't quite right.

> and that's also right, because the hash types do not magically figure out 
> overlapping ranges and collapse those or break up ranges into parts when 
> deleting intersecting elements.
>   
I do not know the reasons for removing this as it was a nice 'feature' 
though I now understand the 'deletion' part even if I do not agree with it.

> The hash:*net* types could be extended to store non-matching elements, 
> something like this:
>
> # ipset -N test hash:net
> # ipset -A test 10.1.1.0/24
> # ipset -A test 10.1.1.12 --nomatch
>
> That way overlapping entries with different "access right" could be stored 
> in a single set. But any coding needs time and testing.
>   
I am not sure I understand the above - is this already implemented (in 
6.0?) or is this on the 'drawing board' so to speak? What do you mean by 
'access right'?

On a separate query relating to my initial post of this thread - Pandu 
Poluan proposed a nice 'get-out-of-jail' solution to my problem, which I 
already found a way to optimise a little, but need to make sure I am 
doing the right thing: if I have my whitelist, blacklist-1, blacklist-2 
... blacklist-x sets registered and they have (quite a lot of) members 
can I then do this:

ipset -N list blacklist-all
ipset -A blacklist-all blacklist-1
...
ipset -A blacklist-all blacklist-x

and then add my iptables statement:

(1) -m set ! --match-set whitelist src -m set --match-set blacklist-all 
src -j DROP

Would that be the equivalent of (2):

-m set ! --match-set whitelist src -m set --match-set blacklist-1 src -j 
DROP
...
-m set ! --match-set whitelist src -m set --match-set blacklist-x src -j 
DROP

In other words, I combine all of my blacklisted sets into one of type 
'list' where I assume 1) that all sub-set elements of the 'list' set are 
queried for a match; and 2) executing one iptables statement with a set 
match/mismatch (i.e. 1 above) is better, performance-wise, than 
(potentially) traversing through multiple iptables statements with a set 
match/mismatch (as shown on 2 above)?

If so, how is the blacklist-all set stored - do you copy all the 
elements of all the sets into a separate memory space or do you just 
reference the set (which means that if I alter, say, blacklist-2, the 
changes are 'automatically' applied to blacklist-all as well)?

I can't combine all elements of my blacklist-x sets into one big one 
because 1) I use separate blacklist-x sets elsewhere in my ip chains; 
and 2) my blacklist-x sets are not of the same type.

Re: ipset -R

[Jozsef Kadlecsik]

On Fri, 25 Feb 2011, Mr Dash Four wrote:

> > However with hash:net type
> > 
> > # ipset -N test hash:net
> > # ipset -A test 10.1.1.0/24
> > # ipset -D test 10.1.1.12
> > ipset v6.0: Element cannot be deleted from the set: it's not added
> >   
> Well, that's plain wrong, isn't it? The 'element' 10.1.1.12 does exist and it
> is added (albeit implicitly as part of 10.1.1.0/24). I also presume 'ipset -T
> test 10.1.1.12' will return a positive result, so there is something which
> isn't quite right.

10.1.1.12/32 is not an explicit member of the set above, therefore you 
cannot delete it.

At testing elements, the host addresses are a special case and checked 
from the kernel point of view. So *testing* 10.1.1.12 returns a true 
value. The reason for the exception is that the kernel at matching, 
deleting, adding entries works on host addresses and that way one can 
check the kernel view of the set from userspace.
 
> > and that's also right, because the hash types do not magically figure out
> > overlapping ranges and collapse those or break up ranges into parts when
> > deleting intersecting elements.
> >   
> I do not know the reasons for removing this as it was a nice 'feature' though
> I now understand the 'deletion' part even if I do not agree with it.

The feature was the part of a single set type and not the ipset core.
 
> > The hash:*net* types could be extended to store non-matching elements,
> > something like this:
> > 
> > # ipset -N test hash:net
> > # ipset -A test 10.1.1.0/24
> > # ipset -A test 10.1.1.12 --nomatch
> > 
> > That way overlapping entries with different "access right" could be stored
> > in a single set. But any coding needs time and testing.
> >   
> I am not sure I understand the above - is this already implemented (in 6.0?)
> or is this on the 'drawing board' so to speak? What do you mean by 'access
> right'?

Not implemented, just thinking. If the feature were implemented then the 
testing in the set would return false for 10.1.1.12 and true for every 
other element from 10.1.1.0/24.

> On a separate query relating to my initial post of this thread - Pandu Poluan
> proposed a nice 'get-out-of-jail' solution to my problem, which I already
> found a way to optimise a little, but need to make sure I am doing the right
> thing: if I have my whitelist, blacklist-1, blacklist-2 ... blacklist-x sets
> registered and they have (quite a lot of) members can I then do this:
> 
> ipset -N list blacklist-all
> ipset -A blacklist-all blacklist-1
> ...
> ipset -A blacklist-all blacklist-x
> 
> and then add my iptables statement:
> 
> (1) -m set ! --match-set whitelist src -m set --match-set blacklist-all src -j
> DROP
> 
> Would that be the equivalent of (2):
> 
> -m set ! --match-set whitelist src -m set --match-set blacklist-1 src -j DROP
> ...
> -m set ! --match-set whitelist src -m set --match-set blacklist-x src -j DROP

Yes, it would be equivalent.
 
> In other words, I combine all of my blacklisted sets into one of type 'list'
> where I assume 1) that all sub-set elements of the 'list' set are queried for
> a match; and 2) executing one iptables statement with a set match/mismatch
> (i.e. 1 above) is better, performance-wise, than (potentially) traversing
> through multiple iptables statements with a set match/mismatch (as shown on 2
> above)?

With first case you spare the iptables rules and the matchings in 
"whitelist".
 
> If so, how is the blacklist-all set stored - do you copy all the elements of
> all the sets into a separate memory space or do you just reference the set
> (which means that if I alter, say, blacklist-2, the changes are
> 'automatically' applied to blacklist-all as well)?

No copying whatsoever: the member sets are referenced and pointed to. 
Please note, you cannot delete the member sets, however you can swap them 
anytime with another, same type of set.
 
> I can't combine all elements of my blacklist-x sets into one big one because
> 1) I use separate blacklist-x sets elsewhere in my ip chains; and 2) my
> blacklist-x sets are not of the same type.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipset -R

[Pandu Poluan]

On Fri, Feb 25, 2011 at 20:27, Mr Dash Four <mr.dash.four@googlemail.com> wrote:
>
>> However with hash:net type
>>
>> # ipset -N test hash:net
>> # ipset -A test 10.1.1.0/24
>> # ipset -D test 10.1.1.12
>> ipset v6.0: Element cannot be deleted from the set: it's not added
>>
>
> Well, that's plain wrong, isn't it? The 'element' 10.1.1.12 does exist and it is added (albeit implicitly as part of 10.1.1.0/24). I also presume 'ipset -T test 10.1.1.12' will return a positive result, so there is something which isn't quite right.
>
>> and that's also right, because the hash types do not magically figure out overlapping ranges and collapse those or break up ranges into parts when deleting intersecting elements.
>>
>
> I do not know the reasons for removing this as it was a nice 'feature' though I now understand the 'deletion' part even if I do not agree with it.
>
>> The hash:*net* types could be extended to store non-matching elements, something like this:
>>
>> # ipset -N test hash:net
>> # ipset -A test 10.1.1.0/24
>> # ipset -A test 10.1.1.12 --nomatch
>>
>> That way overlapping entries with different "access right" could be stored in a single set. But any coding needs time and testing.
>>
>
> I am not sure I understand the above - is this already implemented (in 6.0?) or is this on the 'drawing board' so to speak? What do you mean by 'access right'?
>
> On a separate query relating to my initial post of this thread - Pandu Poluan proposed a nice 'get-out-of-jail' solution to my problem, which I already found a way to optimise a little, but need to make sure I am doing the right thing: if I have my whitelist, blacklist-1, blacklist-2 ... blacklist-x sets registered and they have (quite a lot of) members can I then do this:
>
> ipset -N list blacklist-all
> ipset -A blacklist-all blacklist-1
> ...
> ipset -A blacklist-all blacklist-x
>
> and then add my iptables statement:
>
> (1) -m set ! --match-set whitelist src -m set --match-set blacklist-all src -j DROP
>
> Would that be the equivalent of (2):
>
> -m set ! --match-set whitelist src -m set --match-set blacklist-1 src -j DROP
> ...
> -m set ! --match-set whitelist src -m set --match-set blacklist-x src -j DROP
>
> In other words, I combine all of my blacklisted sets into one of type 'list' where I assume 1) that all sub-set elements of the 'list' set are queried for a match; and 2) executing one iptables statement with a set match/mismatch (i.e. 1 above) is better, performance-wise, than (potentially) traversing through multiple iptables statements with a set match/mismatch (as shown on 2 above)?
>
> If so, how is the blacklist-all set stored - do you copy all the elements of all the sets into a separate memory space or do you just reference the set (which means that if I alter, say, blacklist-2, the changes are 'automatically' applied to blacklist-all as well)?
>
> I can't combine all elements of my blacklist-x sets into one big one because 1) I use separate blacklist-x sets elsewhere in my ip chains; and 2) my blacklist-x sets are not of the same type.
>

I haven't perused netfilter code yet, so what I'll say is highly conjectural.

IMO, the single (1) rule will be a lot faster:
* Only 1 (one) check for whitelist
* x checks for blacklist-checks

Total checks (worst-case): 1+x (and if the negated result of whitelist
check == false, no need for x blacklist-checks)

Best case: 1 check ( IP in whitelist, so ! whitelist == false,
iptables' rule is short-circuited )

Rule (2):
* A total of x times ( whitelist check + blacklist-check )

Total checks (worst-case): x * 2

Best case: x * 1 ( only check against whitelist, but repeated for x rules )

IMO, iptables lookups are much more expensive than ipset lookups. (
IOW, n * iptables checks is much more expensive than 1 * iptables
check against a setlist with n members ). So, the speedup of (1)
against (2) will be even more significant.

CMIIW

Rgds,

--
Pandu E Poluan
~ IT Optimizer ~
Visit my Blog: http://pepoluan.posterous.com
Google Talk:    pepoluan
Y! messenger: pepoluan
MSN / Live:      pepoluan@hotmail.com (do not send email here)
Skype:            pepoluan

Re: ipset -R

[Mr Dash Four]

>>> However with hash:net type
>>>
>>> # ipset -N test hash:net
>>> # ipset -A test 10.1.1.0/24
>>> # ipset -D test 10.1.1.12
>>> ipset v6.0: Element cannot be deleted from the set: it's not added
>>>   
>>>       
>> Well, that's plain wrong, isn't it? The 'element' 10.1.1.12 does exist and it
>> is added (albeit implicitly as part of 10.1.1.0/24). I also presume 'ipset -T
>> test 10.1.1.12' will return a positive result, so there is something which
>> isn't quite right.
>>     
>
> 10.1.1.12/32 is not an explicit member of the set above, therefore you 
> cannot delete it.
>   
Right, so the error message should probably say "Element cannot be 
deleted from the set: it's not *explicitly* added" as this makes it more 
clear as the element in question is clearly added, though implicitly, 
via the 10.1.1.0/24 route.

I know this might be interpreted as 'just semantics', but it would avoid 
any type of confusion and would have spared me the typing trying to ask 
for clarify as to what the above error message means.

> At testing elements, the host addresses are a special case and checked 
> from the kernel point of view. So *testing* 10.1.1.12 returns a true 
> value. The reason for the exception is that the kernel at matching, 
> deleting, adding entries works on host addresses and that way one can 
> check the kernel view of the set from userspace.
>   
I take it that was done differently in the same kernel modules for ipset 
4.x, right?

>>> The hash:*net* types could be extended to store non-matching elements,
>>> something like this:
>>>
>>> # ipset -N test hash:net
>>> # ipset -A test 10.1.1.0/24
>>> # ipset -A test 10.1.1.12 --nomatch
>>>
>>> That way overlapping entries with different "access right" could be stored
>>> in a single set. But any coding needs time and testing.
>>>   
>>>       
>> I am not sure I understand the above - is this already implemented (in 6.0?)
>> or is this on the 'drawing board' so to speak? What do you mean by 'access
>> right'?
>>     
>
> Not implemented, just thinking. If the feature were implemented then the 
> testing in the set would return false for 10.1.1.12 and true for every 
> other element from 10.1.1.0/24.
>   
Call be dumb, but I still fail to see what is the sense in implementing 
the above, or are you suggesting that the above would create a pinhole 
with the "--nomatch" option instead of deleting the element itself and 
therefore remove the need for a 'whitelist'?

> With first case you spare the iptables rules and the matchings in 
> "whitelist".
>   
And, presumably, improve performance, right?

>  
>   
>> If so, how is the blacklist-all set stored - do you copy all the elements of
>> all the sets into a separate memory space or do you just reference the set
>> (which means that if I alter, say, blacklist-2, the changes are
>> 'automatically' applied to blacklist-all as well)?
>>     
>
> No copying whatsoever: the member sets are referenced and pointed to. 
> Please note, you cannot delete the member sets, however you can swap them 
> anytime with another, same type of set.
>   
Please clarify - can I remove elements of a set, i.e. execute "ipset -D 
blacklist-2 <blacklist-2 member(s)>", if blacklist-2 is part (i.e. a 
member) of a list set called blacklist-all, or do you mean that I cannot 
remove blacklist-2 from blacklist-all once added?

>> I can't combine all elements of my blacklist-x sets into one big one because
>> 1) I use separate blacklist-x sets elsewhere in my ip chains; and 2) my
>> blacklist-x sets are not of the same type.
>>     
You didn't clarify this point - can I have different type sub-sets as 
part of a list set or do they have to be of the same type?

Re: ipset -R

[Mr Dash Four]

> I haven't perused netfilter code yet, so what I'll say is highly conjectural.
>
> IMO, the single (1) rule will be a lot faster:
> * Only 1 (one) check for whitelist
> * x checks for blacklist-checks
>
> Total checks (worst-case): 1+x (and if the negated result of whitelist
> check == false, no need for x blacklist-checks)
>
> Best case: 1 check ( IP in whitelist, so ! whitelist == false,
> iptables' rule is short-circuited )
>
> Rule (2):
> * A total of x times ( whitelist check + blacklist-check )
>
> Total checks (worst-case): x * 2
>
> Best case: x * 1 ( only check against whitelist, but repeated for x rules )
>   
The above makes perfect sense and is more or less what I assumed would 
be the case, though I thought of even a better way, which would remove 
the need of using a list-type set:

-m set ! --match-set whitelist src -j $BLACKLIST_PROCESSING_CHAIN

where $BLACKLIST_PROCESSING_CHAIN contains all the blacklist elements 
and another jump to DROP - as is the case at present. There are at least 
two major advantages as far as I can see: 1) I could log/see the number 
of packets dropped for each particular blacklist-x set (something I 
would have lost with the use of list-type set as everything would have 
been lumbered in one place); and 2) performance-wise there won't be any 
difference from what is the case at present - with deleting set members 
and only using the blacklist-x sets, although I have to admit that if I 
have used a single list-type set with a single iptables statement I 
think the performance would have been a little bit better.

> IMO, iptables lookups are much more expensive than ipset lookups. (
> IOW, n * iptables checks is much more expensive than 1 * iptables
> check against a setlist with n members ). So, the speedup of (1)
> against (2) will be even more significant.
>   
Absolutely - the main reason I switched to ipset, as asking iptables to 
traverse through more than, say, 2500 statements in a single chain is 
asking for trouble!

Re: ipset -R

[Jozsef Kadlecsik]

On Fri, 25 Feb 2011, Mr Dash Four wrote:

> > > > However with hash:net type
> > > > 
> > > > # ipset -N test hash:net
> > > > # ipset -A test 10.1.1.0/24
> > > > # ipset -D test 10.1.1.12
> > > > ipset v6.0: Element cannot be deleted from the set: it's not added
> > > >         
> > > Well, that's plain wrong, isn't it? The 'element' 10.1.1.12 does exist and
> > > it
> > > is added (albeit implicitly as part of 10.1.1.0/24). I also presume 'ipset
> > > -T
> > > test 10.1.1.12' will return a positive result, so there is something which
> > > isn't quite right.
> > >     
> > 
> > 10.1.1.12/32 is not an explicit member of the set above, therefore you
> > cannot delete it.
> >   
> Right, so the error message should probably say "Element cannot be deleted
> from the set: it's not *explicitly* added" as this makes it more clear as the
> element in question is clearly added, though implicitly, via the 10.1.1.0/24
> route.

The error messages are general. So I'm worrying that such wording can be 
confusing for other set types, where there's no such distiction as 
explicit/implicit membership.
 
> I know this might be interpreted as 'just semantics', but it would avoid any
> type of confusion and would have spared me the typing trying to ask for
> clarify as to what the above error message means.

The manpage could clearly be improved...
 
> > At testing elements, the host addresses are a special case and checked from
> > the kernel point of view. So *testing* 10.1.1.12 returns a true value. The
> > reason for the exception is that the kernel at matching, deleting, adding
> > entries works on host addresses and that way one can check the kernel view
> > of the set from userspace.
> >   
> I take it that was done differently in the same kernel modules for ipset 4.x,
> right?

ipset 4.x did not support adding host addresses to the *net* types, so 
the issue was nonexistent there.
 
> > > > The hash:*net* types could be extended to store non-matching elements,
> > > > something like this:
> > > > 
> > > > # ipset -N test hash:net
> > > > # ipset -A test 10.1.1.0/24
> > > > # ipset -A test 10.1.1.12 --nomatch
> > > > 
> > > > That way overlapping entries with different "access right" could be
> > > > stored
> > > > in a single set. But any coding needs time and testing.
> > > >         
> > > I am not sure I understand the above - is this already implemented (in
> > > 6.0?)
> > > or is this on the 'drawing board' so to speak? What do you mean by 'access
> > > right'?
> > >     
> > 
> > Not implemented, just thinking. If the feature were implemented then the
> > testing in the set would return false for 10.1.1.12 and true for every other
> > element from 10.1.1.0/24.
> >   
> Call be dumb, but I still fail to see what is the sense in implementing the
> above, or are you suggesting that the above would create a pinhole with the
> "--nomatch" option instead of deleting the element itself and therefore remove
> the need for a 'whitelist'?

Yes, that is what I'm thinking. One could just punch holes into networks, 
even stacked, i.e.

ipset -A test 10.1.1.0/24
ipset -A test 10.1.1.64/26 --nomatch
ipset -A test 10.1.1.80/28

would mean a match for the elements in the inclusive ranges
10.1.1.0-10.1.1.63,10.1.1.80-10.1.1.95,10.1.1.128-10.1.1.255
 
> > With first case you spare the iptables rules and the matchings in
> > "whitelist".
> >   
> And, presumably, improve performance, right?

Yes, exactly.
    
> > > If so, how is the blacklist-all set stored - do you copy all the elements
> > > of
> > > all the sets into a separate memory space or do you just reference the set
> > > (which means that if I alter, say, blacklist-2, the changes are
> > > 'automatically' applied to blacklist-all as well)?
> > >     
> > 
> > No copying whatsoever: the member sets are referenced and pointed to. Please
> > note, you cannot delete the member sets, however you can swap them anytime
> > with another, same type of set.
> >   
> Please clarify - can I remove elements of a set, i.e. execute "ipset -D
> blacklist-2 <blacklist-2 member(s)>", if blacklist-2 is part (i.e. a member)
> of a list set called blacklist-all, or do you mean that I cannot remove
> blacklist-2 from blacklist-all once added?

You can add, delete, even flush the entries from a member set, that's not 
problem. But it's not possible to delete the set while it's the member of 
another set.
 
> > > I can't combine all elements of my blacklist-x sets into one big one
> > > because
> > > 1) I use separate blacklist-x sets elsewhere in my ip chains; and 2) my
> > > blacklist-x sets are not of the same type.
> > >     
> You didn't clarify this point - can I have different type sub-sets as part of
> a list set or do they have to be of the same type?

Yes, you can have different types as subsets, even different dimensions of 
set types. But the dimension of the matching plays an important role: lets 
assume you have got the following sets

ipset create a hash:ip,port,ip
ipset create b hash:ip,port
ipset create c hash:ip
ipset create d list:set
ispet add d a
ipset add d b
ipset add d c

then when called from the rule

-m set --match-set d dst,dst

then the first subset "a" is always skipped, because the matching 
dimension is smaller than the dimension of the set.

However if you have got the rule

-m set --match-set d dst,dst,src

then all subsets are tried in order until a match is found.

Best regards,
Jozsef
-
E-mail  : kadlec@blackhole.kfki.hu, kadlec@mail.kfki.hu
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
          H-1525 Budapest 114, POB. 49, Hungary

Re: ipset -R

[Mr Dash Four]

>> Call be dumb, but I still fail to see what is the sense in implementing the
>> above, or are you suggesting that the above would create a pinhole with the
>> "--nomatch" option instead of deleting the element itself and therefore remove
>> the need for a 'whitelist'?
>>     
>
> Yes, that is what I'm thinking. One could just punch holes into networks, 
> even stacked, i.e.
>
> ipset -A test 10.1.1.0/24
> ipset -A test 10.1.1.64/26 --nomatch
> ipset -A test 10.1.1.80/28
>
> would mean a match for the elements in the inclusive ranges
> 10.1.1.0-10.1.1.63,10.1.1.80-10.1.1.95,10.1.1.128-10.1.1.255
>   
I see! Yes, I think that is a very good idea (you could also introduce a 
separate ipset option to only list '--nomatch' elements, something like 
'ipset -L --nomatch test' for example), though I have to admit I am 
coming round to the idea of using my whitelist as a separate set - I 
think this is a much cleaner solution and it also gives me a greater 
deal of flexibility as I can easily see the whitelist members and employ 
them freely anywhere in the ip chains.

>> Please clarify - can I remove elements of a set, i.e. execute "ipset -D
>> blacklist-2 <blacklist-2 member(s)>", if blacklist-2 is part (i.e. a member)
>> of a list set called blacklist-all, or do you mean that I cannot remove
>> blacklist-2 from blacklist-all once added?
>>     
>
> You can add, delete, even flush the entries from a member set, that's not 
> problem. But it's not possible to delete the set while it's the member of 
> another set.
>   
Makes perfect sense. I also take it since the subsets are referenced as 
soon as a particular subset is changed that change is 'automatically' 
applied to every list set this subset belongs to, right?

> Yes, you can have different types as subsets, even different dimensions of 
> set types. But the dimension of the matching plays an important role: lets 
> assume you have got the following sets
>
> ipset create a hash:ip,port,ip
> ipset create b hash:ip,port
> ipset create c hash:ip
> ipset create d list:set
> ispet add d a
> ipset add d b
> ipset add d c
>
> then when called from the rule
>
> -m set --match-set d dst,dst
>
> then the first subset "a" is always skipped, because the matching 
> dimension is smaller than the dimension of the set.
>
> However if you have got the rule
>
> -m set --match-set d dst,dst,src
>
> then all subsets are tried in order until a match is found.
>   
That's brilliant - I avoided using list sets as, to be honest, I was 
unclear as to how the matching is performed (particularly with different 
subset types), but the above clarifies matters for me completely.

Have you considered using the subsets in a given list set as a 
mathematical entities and applying the usual (mathematical) set 
operations (like intersection, union, complement etc)? Currently, you 
are applying a union of all subsets when they are registered in a list 
set, but this could be expanded as I see the list set as a potentially 
very powerful tool.

For example I could define the whitelist as a set and then register it 
together with all of my blacklist-x in a new list set and then say 
'apply (blacklist-1 + blacklist-2 + ... blacklist-x) - whitelist' (in 
other words, build a union of all blacklist-x subsets and then subtract 
the whitelist subset from that resulting union). I don't know how easy 
this would be to implement though!