iptables ip tracking buffer size?

iptables ip tracking buffer size?

[Matthew Sims]

Quick backstory. I recently took over an unused project from a co-worker
who left over a year ago. The idea was to block IPs that reached a certain
hit_count limit within a minute.

When this was enabled, it worked...sort of. While iptables was keeping
track of IPs, we weren't able to block when tests were performed.

The reason had something to do with IPs in a "buffer pool" (??) were being
knocked out when new IPs were tracked? I don't have the actual issue as he
never documented it. I'm guessing that while iptables was keeping track of
hit_counts, the list grew so big that it had to bump out IPs at the end of
the list to track new ones.

This is the best I can do to describe this and wondered if anyone knows
about this. I don't have a name that allows me to research this so its been
difficult to find a solution.

Re: iptables ip tracking buffer size?

[Pascal Hambourg]

Le 03/03/2017 à 19:20, Matthew Sims a écrit :
> Quick backstory. I recently took over an unused project from a co-worker
> who left over a year ago. The idea was to block IPs that reached a certain
> hit_count limit within a minute.
>
> When this was enabled, it worked...sort of. While iptables was keeping
> track of IPs, we weren't able to block when tests were performed.
>
> The reason had something to do with IPs in a "buffer pool" (??) were being
> knocked out when new IPs were tracked? I don't have the actual issue as he
> never documented it. I'm guessing that while iptables was keeping track of
> hit_counts, the list grew so big that it had to bump out IPs at the end of
> the list to track new ones.


You may be talking about the "recent" match.
The iptables or iptables-extensions manpage describes how to tune its 
table sizes.

Re: iptables ip tracking buffer size?

[Robert White]

On 03/03/17 18:20, Matthew Sims wrote:
> Quick backstory. I recently took over an unused project from a co-worker
> who left over a year ago. The idea was to block IPs that reached a certain
> hit_count limit within a minute.

You don't need "a project" you just need the "recent" match and a branch 
chain.

Consider this set of rules for throttling SSH attempts to five per hour.


iptables --new-chain SSHTHROTTLE

iptables --append SSHTHROTTLE --match recent --name bad_actors --update 
--seconds 86400 --jump DROP

iptables --append SSHTHROTTLE --match hashlimit --hashlimit-name 
ssh_throttle --hashlimit-upto 5/hour --hashlimit-mode srcip 
--hashlimit-burst 2 --jump ACCEPT

iptables --append SSHTHROTTLE --match recent --name bad_actors --set 
--jump DROP

iptables --append INPUT --in-interface ext+ --proto tcp --match 
conntrack --ctstate NEW --dport 22 --syn --jump SSHTHROTTLE


So the core thing is the three rules in the SSHTHROTTLE chain. How does 
it do what you want?

In the _last_ rule of the chain, the source ip address is added to the 
"bad_actors" set and the packet is dropped.

In the first rule the "--match recent" with "--update" is only true if 
the packet's source ip address is already in the "bad_actors" set. So it 
only does the DROP if some previous pass through the chain got all the 
way to the end of the chain.

So the middle rule, the hashlimit rule will ACCEPT up to five attempts 
an hour. Since "ACCEPT" stops the chain evaluation the "hashlimit" 
allows up to five connections an hour. So as long as the client never 
does six or more an hour he's fine.

Now back to that first rule. The --seconds value (one day in this case) 
acts as a grace period. If the bad actor goes away for a day then the 
match will miss and they can proceed with a new connection. Since the 
recent match is less than the penalty time they are forgiven their past 
sins and they are back to five connections an hour.

NOTE: using "-j RETURN" instead of "-j ACCEPT" will make this logic work 
without actually accepting the packet, so you can have this kind of gate 
without the gate needing to be the point of authorization. By that I 
mean that the throttling doesn't have to be tied to the details of the rule.

So playing around with timing and other parameters, and using other 
conditions on the call rule can let one gate/throttle be part of many 
sequences without making things hard.

You need one chain per throttle group, but you can have lots of 
different throttles by using different names in the recent and 
[hash]limit stanzas.

Eventually you will hit limits if you've got thousands of entries in the 
limits and whatnot because memory is a finite resource.

But you don't need complex code or any kind of external program to pull 
off these throttles.