can we design a modified fail2ban ?

can we design a modified fail2ban ?

[J. Bakshi]

Dear list,

fail2ban is a popular application to prevent the brute-force attack
against ssh and also against imap, pop3 etc.. But fail2ban actually
blacklist the IP and this is what fail2ban has been designed for. Now a
days we can design the same with iptables. I wonder if iptables can
provide more liberty to match IP as well as port combination so that we
don't need to blacklist the IP but only block the attempts from the IP
based on port. Say more than 3 ssh attempt from IP xxx.xxx.xxx.xxx is
detected and no more ssh attempt from the same ip is no more possible
but pop and imap still works. Is it really possible with iptables ? Any
idea ?

Thanks

-- 
জয়দীপ বক্সী

Re: can we design a modified fail2ban ?

[Jan Engelhardt]

On Friday 2010-04-16 05:57, J. Bakshi wrote:
>
>fail2ban is a popular application to prevent the brute-force attack
>against ssh and also against imap, pop3 etc.. But fail2ban actually
>blacklist the IP and this is what fail2ban has been designed for.
>Now a days [nowadays] we can design the same with iptables.

fail2ban has the ability - if I read its own short description right - to
already use various blocking methods, including not only /etc/hosts.deny
but also iptables.

>I wonder if iptables can
>provide more liberty to match IP as well as port combination so that we
>don't need to blacklist the IP but only block the attempts from the IP
>based on port. Say more than 3 ssh attempt from IP xxx.xxx.xxx.xxx is
>detected and no more ssh attempt from the same ip is no more possible
>but pop and imap still works. Is it really possible with iptables ? Any
>idea ?
>
>Thanks
>
>-- 
>জয়দীপ বক্সী
>

Re: can we design a modified fail2ban ?

[Pascal Hambourg]

Hello,

J. Bakshi a écrit :
> 
> fail2ban is a popular application to prevent the brute-force attack
> against ssh and also against imap, pop3 etc.. But fail2ban actually
> blacklist the IP and this is what fail2ban has been designed for. Now a
> days we can design the same with iptables. I wonder if iptables can
> provide more liberty to match IP as well as port combination so that we
> don't need to blacklist the IP but only block the attempts from the IP
> based on port. Say more than 3 ssh attempt from IP xxx.xxx.xxx.xxx is
> detected and no more ssh attempt from the same ip is no more possible
> but pop and imap still works. Is it really possible with iptables ?

Detected how ? fail2ban detects authentication failures by parsing the
server logs. iptables cannot do this, and IHMO should not do this. This
is just not its purpose.

Re: can we design a modified fail2ban ?

[Alessandro Vesely]

On 16/Apr/10 09:28, Jan Engelhardt wrote:
> On Friday 2010-04-16 05:57, J. Bakshi wrote:
>>
>>fail2ban is a popular application to prevent the brute-force attack
>>against ssh and also against imap, pop3 etc.. But fail2ban actually
>>blacklist the IP and this is what fail2ban has been designed for.
>>Now a days [nowadays] we can design the same with iptables.
>
> fail2ban has the ability - if I read its own short description right - to
> already use various blocking methods, including not only /etc/hosts.deny
> but also iptables.

I don't think it uses netfilter, though. I've read it has to restart a 
daemon in order to unlist an IP --not sure it's still so for the 
current version.

>>I wonder if iptables can
>>provide more liberty to match IP as well as port combination so that we
>>don't need to blacklist the IP but only block the attempts from the IP
>>based on port. Say more than 3 ssh attempt from IP xxx.xxx.xxx.xxx is
>>detected and no more ssh attempt from the same ip is no more possible
>>but pop and imap still works. Is it really possible with iptables ? Any
>>idea ?

The fail2ban doubts I mentioned above are the raison d'etre of ipqbdb. 
It matches IPs for responses, but it leaves it up to the sysadmin to 
configure iptables. For example, an admin may call the NFQUEUE rule 
only for ssh, so as to live pop3 and imap alone; or call NFQUEUE with 
a different queue-num for different services, so as to check client 
IPs against different (Berkeley) databases. Ipqbdb gives a random 
answer for, say, between 2 and 4 attempts, similar to the way stockade 
works. See http://en.wikipedia.org/wiki/Stockade_%28software%29 for a 
short description of such rate limiting approach, 
https://savannah.nongnu.org/projects/ipqbdb/ for ipqbdb itself.

If you are specifically interested in blocking ssh, follow the 
"DenyHosts" link from the "See also" section of that wikipedia page.

Re: can we design a modified fail2ban ?

[Jan Engelhardt]

On Saturday 2010-04-17 18:01, Alessandro Vesely wrote:
> On 16/Apr/10 09:28, Jan Engelhardt wrote:
>> On Friday 2010-04-16 05:57, J. Bakshi wrote:
>>>
>>> fail2ban is a popular application to prevent the brute-force attack
>>> against ssh and also against imap, pop3 etc.. But fail2ban actually
>>> blacklist the IP and this is what fail2ban has been designed for.
>>> Now a days [nowadays] we can design the same with iptables.
>>
>> fail2ban has the ability - if I read its own short description right - to
>> already use various blocking methods, including not only /etc/hosts.deny
>> but also iptables.
>
> I don't think it uses netfilter, though. I've read it has to restart a daemon
> in order to unlist an IP --not sure it's still so for the current version.

Better know than think.

N.B.: If what http://en.wikipedia.org/wiki/Fail2ban says is not
correct, by all means you should correct it.

Besides, if it is accurate, it uses iptables, not directly Netfilter.

Re: can we design a modified fail2ban ?

[Alessandro Vesely]

On 17/Apr/10 19:58, Jan Engelhardt wrote:
> On Saturday 2010-04-17 18:01, Alessandro Vesely wrote:
>>>  fail2ban has the ability - if I read its own short description right - to already use various blocking methods, including not only /etc/hosts.deny but also iptables.
>>
>>  I don't think it uses netfilter, though. I've read it has to restart a daemon in order to unlist an IP --not sure it's still so for the current version.
>
> Better know than think.

The bit I had read is "You currently have to restart the daemon to 
unban." in http://www.fail2ban.org/wiki/index.php/Features#0.9.0

However, reading slightly more carefully, that's about _manually_ 
unbanning an IP (e.g. a misconfigured client that locked out the whole 
office behind its NAT.)

> N.B.: If what http://en.wikipedia.org/wiki/Fail2ban says is not correct, by all means you should correct it.
>
> Besides, if it is accurate, it uses iptables, not directly Netfilter.

Correct. Browsing action.d/iptables.conf one finds

  # Option:  actionban
  # Notes.:  command executed when banning an IP. Take care that the
  #          command is executed with Fail2Ban user rights.
  # Tags:    <ip>  IP address
  #          <failures>  number of failures
  #          <time>  unix timestamp of the ban time
  # Values:  CMD
  #
  actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP

  # Option:  actionunban
  # Notes.:  command executed when unbanning an IP. Take care that the
  #          command is executed with Fail2Ban user rights.
  # Tags:    <ip>  IP address
  #          <failures>  number of failures
  #          <time>  unix timestamp of the ban time
  # Values:  CMD
  #
  actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP

I think the daemon just executes those commands, after replacing the 
tags. I don't know whether fail2ban uses some other storage to 
remember frequently banned IPs.

How would you compare iptables and netfilter? I mean fail2ban actions 
versus looking up a b-tree file, in terms of rough memory consumption 
and responsiveness expectations? For the max number of entries, I 
reckon b-trees can allow to map the entire IPv4 address space within 
1Tb of mass storage. But what might be the difference with usual volumes?

Re: can we design a modified fail2ban ?

[Jan Engelhardt]

On Sunday 2010-04-18 15:46, Alessandro Vesely wrote:
>
> Correct. Browsing action.d/iptables.conf one finds

> actionban = iptables -I fail2ban-<name> 1 -s <ip> -j DROP
> actionunban = iptables -D fail2ban-<name> -s <ip> -j DROP
>
> I don't know whether fail2ban uses some other storage to remember frequently
> banned IPs.

If you are using iptables for actionban, it would not need to.
You can make use of iptables-save or ipset -S on shutdown.

> How would you compare iptables and netfilter?

Like you compare a tree with soil?

> I mean fail2ban actions versus looking up a b-tree file,

Where does that btree file come from, and what should it be useful for?

>in terms of rough
> memory consumption and responsiveness expectations? For the max number of
> entries, I reckon b-trees can allow to map the entire IPv4 address space
> within 1Tb of mass storage. But what might be the difference with usual
> volumes?
>

Re: can we design a modified fail2ban ?

[J. Bakshi]

On 04/17/2010 09:31 PM, Alessandro Vesely wrote:
> On 16/Apr/10 09:28, Jan Engelhardt wrote:
>> On Friday 2010-04-16 05:57, J. Bakshi wrote:
>>>
>>> fail2ban is a popular application to prevent the brute-force attack
>>> against ssh and also against imap, pop3 etc.. But fail2ban actually
>>> blacklist the IP and this is what fail2ban has been designed for.
>>> Now a days [nowadays] we can design the same with iptables.
>>
>> fail2ban has the ability - if I read its own short description right
>> - to
>> already use various blocking methods, including not only /etc/hosts.deny
>> but also iptables.
>
> I don't think it uses netfilter, though. I've read it has to restart a
> daemon in order to unlist an IP --not sure it's still so for the
> current version.
>
>>> I wonder if iptables can
>>> provide more liberty to match IP as well as port combination so that we
>>> don't need to blacklist the IP but only block the attempts from the IP
>>> based on port. Say more than 3 ssh attempt from IP xxx.xxx.xxx.xxx is
>>> detected and no more ssh attempt from the same ip is no more possible
>>> but pop and imap still works. Is it really possible with iptables ? Any
>>> idea ?
>
> The fail2ban doubts I mentioned above are the raison d'etre of ipqbdb.
> It matches IPs for responses, but it leaves it up to the sysadmin to
> configure iptables. For example, an admin may call the NFQUEUE rule
> only for ssh, so as to live pop3 and imap alone; or call NFQUEUE with
> a different queue-num for different services, so as to check client
> IPs against different (Berkeley) databases. Ipqbdb gives a random
> answer for, say, between 2 and 4 attempts, similar to the way stockade
> works. See http://en.wikipedia.org/wiki/Stockade_%28software%29 for a
> short description of such rate limiting approach,
> https://savannah.nongnu.org/projects/ipqbdb/ for ipqbdb itself.
>
> If you are specifically interested in blocking ssh, follow the
> "DenyHosts" link from the "See also" section of that wikipedia page.
> -- 
> To unsubscribe from this list: send the line "unsubscribe netfilter" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>


Hello,

thanks to all of you for your ideas and discussion. I am thank-full to
Richard Horton, who has shown me a great way to design iptables rules
which can work like fail2ban; not only that it can be extended in many
different ways for different purposes. Based on his suggestion I have
made these rule sets and it works as expected, i.e blocking ssh overflow
but the other services i.e. apache, imap etc.. are still accessible.

````````````
# mark packets

iptables -A INPUT -p tcp -m hashlimit --hashlimit-above 2/min
--hashlimit-burst 2 --hashlimit-name hashlimit -m state --state NEW \
-m tcp --dport $SSH_PORT -j MARK --set-xmark 0x1/0xffffffff

# blacklisting interval
iptables -A INPUT -p tcp --dport $SSH_PORT -m recent --rcheck --seconds
60 --name sshoverflow --rsource -j DROP

# blacklist port only based on IP
iptables -A INPUT -p tcp --dport $SSH_PORT -m mark --mark 0x1 -m recent \
--set --name sshoverflow --rsource -j DROP

# accept others
iptables -A INPUT -p tcp -m state --state NEW --dport $SSH_PORT -j ACCEPT

````````````

The only thing which is unusual here is the duration part. what ever
duration is defined here , practically becomes the double. I have tested
with *time* command with different values and every time I have found
that in practical duration becomes double than what is defined . Except
that the rule sets are working well.

Thanks

-- 

জয়দীপ বক্সী

Re: can we design a modified fail2ban ?

[Alessandro Vesely]

On 18/Apr/10 18:44, Jan Engelhardt wrote:
> On Sunday 2010-04-18 15:46, Alessandro Vesely wrote:
>>  actionban = iptables -I fail2ban-<name>  1 -s<ip>  -j DROP
>>  actionunban = iptables -D fail2ban-<name>  -s<ip>  -j DROP
>>
>>  I don't know whether fail2ban uses some other storage to remember frequently banned IPs.
>
> If you are using iptables for actionban, it would not need to.

Right, but rather than deleting by number, it uses the same IP to 
issue the unban command. If the IP is remembered even after unbanning 
it, then it is possible to increase its next ban-period, in case it is 
caught again.

>>  How would you compare iptables and netfilter?
>
> Like you compare a tree with soil?

Yeah, my question was short, but not formally sound...

I heard about some not better characterized performance degradation 
that would occur when successively issuing a huge number of "iptables 
-I" commands. How huge is "huge"? Where is this topic expounded?

>>  I mean fail2ban actions versus looking up a b-tree file,
>
> Where does that btree file come from, and what should it be useful for?

A b-tree can be used to store IPs: One has to remember them, at least 
for some time, one way or another, if different IPs are to be treated 
differently. One can use an "at" command for unbanning, and avoid 
structured storage. Alternatively, one could use ipset, e.g. iptrees 
with timeouts. As yet another alternative, one can use netfilter and 
user-space storage. I mentioned b-trees because I use Berkeley DB with 
the my netfilter daemon. MySQL or anything similar offer similar 
functionality and performance.

When I started to use my daemon, I was afraid that accessing the disk 
on each packet would have been a bottleneck. So, I only filtered new 
connections. Later on, I added a rule to filter all packets, in order 
to also terminate existing connections. Two rules, actually: a 
netfilter rule to set a mark and another one to reject those packets, i.e.

   iptables -t raw -A OUTPUT -p tcp ! --syn -m multiport --sports $X \
      -j NFQUEUE --queue-num 4
   iptables -A OUTPUT -p tcp ! --syn -m mark --mark 4 \
      -j REJECT --reject-with tcp-reset

(For udp and --syn connections, a single netfilter rule is enough.)

The time taken for issuing verdicts used to be much less than that for 
parsing the log file. The former has jumped to about the double of the 
latter, after those additional two rules. Perhaps, I should have stuck 
to using netfilter for new connections only. Existing connections can 
be terminated with short-lived iptables rules, or by calling 
nids_killtcp() or "ngrep -K" directly. In hindsight, that looks fairly 
obvious. However, the above quoted fail2ban's actions don't seem to be 
concerned with such kind of optimizations. Thoughts?