[SOLVED] iptables does not appear to be blocking IP

mfoley · 09-22-2019, 01:08 AM

I have the following iptables rule:

Code:

> iptables -L -v -n --line-numbers | grep 45.136
14       0     0 DROP       all  --  *      *       45.136.108.0/24      0.0.0.0/0

Yet a persistant attacker on that IP continues to get through from that IP on port 1913 trying to brute force logins.

Code:

> nc -lv -p 1913
listening on [any] 1913 ...
45.136.108.10: inverse host lookup failed: Unknown host
connect to [192.168.0.2] from (UNKNOWN) [45.136.108.10] 60552
+&�Cookie: mstshash=hello

Why? How?

ferrari · 09-22-2019, 01:24 AM

Can you show the complete iptables rules? If too lengthy, at least show us all the rules in the chain. If a rule is matching earlier in the chain then your custom rule will never take effect.

mfoley · 09-22-2019, 01:42 AM

I'll show you what I have. Unfortunately, I felt the need to pull the plug on port 1913. The attacker had tried to brute force 67,642 times in the past 15 1/2 hours. This necessitated restarting iptables. However, I do keep track of the blocked IP and they get put into the bad_people chain when iptables is restarted. Here's what I've now got:

Code:

> iptables -L -v -n --line-numbers | more
Chain INPUT (policy DROP 41 packets, 1640 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1     1147 65141 ACCEPT     tcp  --  eth1   *       (not 45. ....)        0.0.0.0/0            tcp dpt:22
2        0     0 ACCEPT     tcp  --  eth1   *       (not 45. ....)        0.0.0.0/0            tcp dpt:22
3    29330 4426K bad_people  all  --  eth1   *       0.0.0.0/0            0.0.0.0/0           
4    26180 4582K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            state RELATED,ESTABLISHED
5      326 21507 ACCEPT     all  --  lo     *       0.0.0.0/0            0.0.0.0/0           
6      618 35548 ACCEPT     tcp  --  eth1   *       0.0.0.0/0            0.0.0.0/0            tcp flags:0x17/0x02
7     3741  936K ACCEPT     udp  --  eth1   *       0.0.0.0/0            0.0.0.0/0           
8        0     0 ACCEPT     icmp --  *      *       192.168.0.0/24       0.0.0.0/0            icmptype 8 state NEW,RELATED,ESTABLISHED
9        0     0 checkcount  tcp  --  eth1   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22 flags:0x17/0x02
10       0     0 ACCEPT     tcp  --  eth1   *       0.0.0.0/0            0.0.0.0/0            tcp dpt:22 flags:0x17/0x02 limit: avg 1/sec burst 3

Chain bad_people (2 references)
num   pkts bytes target     prot opt in     out     source               destination         
1       24  1440 DROP       all  --  *      *       181.230.0.0/16       0.0.0.0/0           
2       24  1440 DROP       all  --  *      *       202.29.0.0/16        0.0.0.0/0           
3       24  1440 DROP       all  --  *      *       122.55.0.0/16        0.0.0.0/0           
4       28  1680 DROP       all  --  *      *       211.75.0.0/16        0.0.0.0/0      
:
:
and another 438 DROP entries ...

All of the INPUT chain rules are automatically loaded and should have been the same as before I restarted. The bad_people chain is added to by intrusion detection scripts, which is how 45.136.108.15 got added in the first place (OP).

Interestingly, after I turned off port 1913 and listened to that port with nc, the attacker got no response to his "hello" request and, after disconnecting, hasn't retried since (that is, in the past hour).

I know the actual iptables rule-set from the attack is gone, but unless there were somehow a rule permitting 45.136.108.0/24 can you think of any way it could continually get past iptables?

ferrari · 09-22-2019, 02:13 AM

Quote:

I know the actual iptables rule-set from the attack is gone, but unless there were somehow a rule permitting 45.136.108.0/24 can you think of any way it could continually get past iptables?

No, the only logical explanation is that a matching rule occurs earlier in the netfilter chain, (likely your rule for allowing incoming connections on port 1913).

mfoley · 09-22-2019, 09:31 AM

Quote:

Originally Posted by ferrari

No, the only logical explanation is that a matching rule occurs earlier in the netfilter chain, (likely your rule for allowing incoming connections on port 1913).

The bot is back! This time trying another port.

The rule that allowed incoming connections on 1913 is:

Code:

iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 1913 -j DNAT --to-destination 192.168.0.52:3389
iptables -A FORWARD -i eth1 -d 192.168.0.52 -p tcp --dport 3389 -j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -d 192.168.0.52 -p tcp --dport 3389 -j SNAT --to-source 192.168.0.2
iptables -t nat -A POSTROUTING -o eth1 -d 192.168.0.52 -p tcp --dport 3389 -j SNAT --to-source 192.168.0.2

This rule was also given to me by kind soul on LQ (here, in fact: https://www.linuxquestions.org/quest...ng-4175650559/). Does it look like it would override the DROP in the bad_people table?

I've now blocked that ip again with '/usr/sbin/iptables -I bad_people -s 45.136.108.0/24 -j DROP'. Running tcpdump shows continued attempts even though I've done that. They are getting through to the target host, so the DROP is definitely not working.

Code:

> tcpdump -n -l -i eth1 port 1914 and 'tcp[13] & 2 = 2'
:
10:31:06.330429 IP 45.136.108.17.55145 > 192.168.0.2.1914: Flags [SEW], seq 3410885181, win 8192, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
10:31:06.330889 IP 192.168.0.2.1914 > 45.136.108.17.55145: Flags [S.], seq 2877431362, ack 3410885182, win 64000, options [mss 1460,nop,wscale 0,nop,nop,sackOK], length 0
:

I've also just added 'iptables -I INPUT -s 45.136.108.0/24 -j DROP', so it is the first rule in the INPUT chain:

Code:

Chain INPUT (policy DROP 1 packets, 40 bytes)
num   pkts bytes target     prot opt in     out     source               destination         
1        0     0 DROP       all  --  *      *       45.136.108.0/24      0.0.0.0/0

Hackbot still at it. Still getting the above tcpdump output.

I'll delete the nat rule(s) and see what happens.

rknichols · 09-22-2019, 09:36 AM

tcpdump is going to pick up packets before they are presented to iptables. Unless you are also seeing your system sending a response to those packets, your DROP rule is working.

mfoley · 09-22-2019, 10:25 AM

Quote:

Originally Posted by rknichols

tcpdump is going to pick up packets before they are presented to iptables. Unless you are also seeing your system sending a response to those packets, your DROP rule is working.

Is not the 2nd line in that tcpdump the response? (192.168.0.2 is the local host, 45.136.108.17 is the bad actor)

Code:

10:31:06.330889 IP 192.168.0.2.1914 > 45.136.108.17.55145: Flags [S.], seq 2877431362, ack 3410885182, win 64000, options [mss 1460,nop,wscale 0,nop,nop,sackOK], length 0

In any case, the DROP was not working. I was able to see login attempts as it continued to try two ports (one example):

Code:

[2019/09/22 11:14:17.806429,  2] ../source4/auth/ntlm/auth.c:475(auth_check_password_recv)
  auth_check_password_recv: sam authentication for user [(null)\USER5] FAILED with error NT_STATUS_NO_SUCH_USER, authoritative=1

As threatened, I then tried deleting the nat rule:

Code:

Chain POSTROUTING (policy ACCEPT 3 packets, 156 bytes)
num   pkts bytes target     prot opt in     out     source               destination
:
22       0     0 SNAT       tcp  --  *      *       192.168.0.0/24       192.168.0.60         tcp dpt:3389 to:192.168.0.2
23    1377 71604 SNAT       tcp  --  *      eth1    0.0.0.0/0            192.168.0.60         tcp dpt:3389 to:192.168.0.2
:

rules 22 and 23 were from 'iptables -t nat ...' rules in my previous post; in this case to host 192.168.0.60. Notice the large number of packets (just since I restarted iptables not long ago).

When I deleted rule 23, I stopped getting the authentication failure messages (above) and tcpdump stopped showing the return ack packets, and less than 5 minutes later the bot stopped trying altogether.

That's the good news. The bad news is that somehow those nat rules override the DROP rules, which is very, very bad. I need to be able to block these hacks!

Can someone help me refine these rules? Again, for reference:

Code:

iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 1913 -j DNAT --to-destination 192.168.0.52:3389
iptables -A FORWARD -i eth1 -d 192.168.0.52 -p tcp --dport 3389 -j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -d 192.168.0.52 -p tcp --dport 3389 -j SNAT --to-source 192.168.0.2
iptables -t nat -A POSTROUTING -o eth1 -d 192.168.0.52 -p tcp --dport 3389 -j SNAT --to-source 192.168.0.2

ferrari · 09-22-2019, 06:02 PM

I'm not sure why the blacklisting is working as expected. From what has been shared so far, you seem to have implemented in the usual fashion...

http://logic.edchen.org/how-to-block...t-in-iptables/

I did wonder whether you need rules in both the INPUT and FORWARD chains though...

Code:

iptables -I INPUT -m set --match-set bad_people src -j DROP
iptables -I FORWARD -m set --match-set bad_people src -j DROP

rknichols · 09-22-2019, 09:16 PM

Indeed, since port 1913 is being forwarded to another machine, those packets never traverse the INPUT chain. That chain is only for packets destined for the local machine.

ferrari · 09-22-2019, 09:51 PM

I concur.

mfoley · 09-24-2019, 04:41 PM

Quote:

Originally Posted by rknichols

Indeed, since port 1913 is being forwarded to another machine, those packets never traverse the INPUT chain. That chain is only for packets destined for the local machine.

Ah ha! Makes sense. How do I deal with the problem? Is there not something I can do on the machine doing the forwarding?

ferrari: is your '-m set --match-set' suggestion something to accomplish this? If not (or so) what is that supposed to do?

I've checked your link and my "BLACKLIST" chain is "bad_people". Otherwise, I nothing different stands out.

later ...

OK, I've done some searching on '-m set --match-set' (of which I've found precious little!) and it does appear that this parameter will instruct the FORWARD to check the bad_people chain. I'd like to try that. Where should I put these two rules? The order in which I run the iptables command is:

Code:

    $IPTABLES -P INPUT ACCEPT
    $IPTABLES -P FORWARD ACCEPT
    $IPTABLES -P OUTPUT ACCEPT

    $IPTABLES -t nat -P PREROUTING ACCEPT
    $IPTABLES -t nat -P POSTROUTING ACCEPT
    $IPTABLES -t nat -P OUTPUT ACCEPT

    $IPTABLES -t mangle -P PREROUTING ACCEPT
    $IPTABLES -t mangle -P INPUT ACCEPT
    $IPTABLES -t mangle -P FORWARD ACCEPT
    $IPTABLES -t mangle -P OUTPUT ACCEPT
    $IPTABLES -t mangle -P POSTROUTING ACCEPT

    $IPTABLES -t raw -P PREROUTING ACCEPT
    $IPTABLES -t raw -P OUTPUT ACCEPT

    $IPTABLES -F
    $IPTABLES -F -t nat
    $IPTABLES -F -t mangle
    $IPTABLES -F -t raw

    $IPTABLES -X
    $IPTABLES -X -t nat
    $IPTABLES -X -t mangle
    $IPTABLES -X -t raw
   $IPTABLES -P INPUT DROP
    $IPTABLES -N bad_people

Then I add the saved list of bad_people blocks, then I add those workstation nat/PREROUTING, FORWARD rules.

Should I put the '-m set --match-set' rules after creating the bad_people chain and before adding IPs to that chain, or after adding to the chain and before adding the workstation routing rules?

ferrari · 09-24-2019, 08:24 PM

Quote:

ferrari: is your '-m set --match-set' suggestion something to accomplish this? If not (or so) what is that supposed to do?

Yes, it was just to illustrate adding a suitable rule to your FOWARD chain (as well as the INPUT chain).

Quote:

I've checked your link and my "BLACKLIST" chain is "bad_people". Otherwise, I nothing different stands out.

Just shared to help with clarification.

later ...

Quote:

OK, I've done some searching on '-m set --match-set' (of which I've found precious little!) and it does appear that this parameter will instruct the FORWARD to check the bad_people chain.

Yes, that's exactly why I posted that rule. For more info

Code:

man iptables

Quote:

I'd like to try that. Where should I put these two rules? The order in which I run the iptables command is:

In general, they should be put as early in the respective chain as possible, but in practice as long there is no rule which might match they absolute position does not matter. The '-I' option will add to the beginning, unless a specific rule number position is explicitly stipulated....

Quote:

-I, --insert chain [rulenum] rule-specification
Insert one or more rules in the selected chain as the given rule number. So, if the rule number is 1, the rule or rules are inserted at
the head of the chain. This is also the default if no rule number is specified.

mfoley · 09-24-2019, 10:21 PM

I've stuck those two '-m set' rules right after creating the bad_people chain and before any IP DROPs, before any workstation FORWARDs and before any other INPUT rules. My erstwhile attacker has gone away (for now) so I'll have to wait to test.

Or will I? While searching for things I came across 'ipset test' which appears to let me test the rules. Unfortunately, I can no longer find that example, nor any other example for 'ipset test'. Maybe I'm just a very poor googler! Can I use that command to test with my known bad actor's IP? I've looked at the ipset man page which has 'ipset test SETNAME TEST-ENTRY [ TEST-OPTIONS ]'. It doesn't really say what a 'set' is, but I'm guessing from your (farrari) rule examples, "bad_people" is a set? How about "FORWARD"? I tried:

Code:

> ipset test bad_people 45.136.108.17

Which gave, "The set with the given name does not exist." So perhaps I don't understand what a set is. Perhaps this isn't even the tool for this.

(btw - I don't know about your iptables version, but my man page, iptables (1.6.0), iptables-extensions, has nothing about --match-set. iptables-extensions says -m specifies the module name -- in this case "set", but where is the documentation on the "set" module? Doing 'iptables -m set -h' gives only "--match-set name flags [--return-nomatch]")

astrogeek · 09-25-2019, 12:27 AM

You will have to create an ipset set before you can add/delete/match addresses with it. It is very well worth learning and easy to use. Iptables 1.6.0 supports it (if built with ipset support).

From man iptables-extensions:

Code:

   set
       This module matches IP sets which can be defined by ipset(8).

       [!] --match-set setname flag[,flag]...
              where flags are the comma separated list of src and/or dst specifications and  there  can  be  no
              more than six of them. Hence the command

               iptables -A FORWARD -m set --match-set test src,dst

              will  match  packets, for which (if the set type is ipportmap) the source address and destination
              port pair can be found in the specified set. If the set type  of  the  specified  set  is  single
              dimension  (for  example ipmap), then the command will match packets for which the source address
              can be found in the specified set.

...
   SET
       This module adds and/or deletes entries from IP sets which can be defined by ipset(8).

       --add-set setname flag[,flag...]
              add the address(es)/port(s) of the packet to the set

       --del-set setname flag[,flag...]
              delete the address(es)/port(s) of the packet from the set
...

See man 8 ipset for the ipset options, including set creation and management.

Sets must be created before iptables can match or make other use of them, of course. So if you did not create a bad_people set you will see the message you posted.

An example of creating a set with support for networks and IPs and counts the number of times an address matches might be like this (you need to read man 8 ipset to understand the different set types and write a command for your own case):

Code:

ipset create myset hash:net hashsize 1024 counters

ipset add myset 111.222.0.0/16

ipset list myset
Name: myset
Type: hash:net
Revision: 4
Header: family inet hashsize 1024 maxelem 65536 counters
Size in memory: 440
References: 0
Members:
111.222.0.0/16 packets 0 bytes 0

ipset test myset 111.222.123.45
111.222.123.45 is in set myset.

You can create a set, add, delete and test without actually making use of it in your iptables rules, so it is easy to learn your way around the sets. When you want to use them in your iptables rules, add the set creation to you rules script before it is referenced in any rule.

Everything you need to know is in the man pages for ipset(8) and iptables-extensions, but you will need to read through them and understand them.

UPDATE: Added following for more complete suggestion of use...

If you use an ipset set to keep banned IPs/networks, you can easily script loading them from a simple text file or using ipset save/restore to initialize the set from a previous state.

Then, assuming your set is named bad_people, you might make this your first rule:

Code:

iptables -t raw -I PREROUTING -m set --match-set bad_people src -j DROP

Anything in that set will never be be seen again...

(I wrote the above rule from memory, fix errors according to man pages)

mfoley · 09-30-2019, 10:48 AM

Quote:

Originally Posted by astrogeek

Everything you need to know is in the man pages for ipset(8) and iptables-extensions, but you will need to read through them and understand them.

Astrogeek: Thanks for that extensive discourse and examples. Athough, I don't know if I totally agree with your assertion that "everything you need to know is in the man pages." I looked for information on understanding, for example, what the various datatypes meant and found little either in the man pages or on the web, although both referenced man pages were highly useful for syntax, if not in-depth explanation. I did find a pretty useful document at https://www.netdevconf.org/1.1/proce...g-iptables.pdf, which was part of the proceedings of a NetDev conference in Spain, 2016. There were a few other helpful posts as well.

Quote:

If you use an ipset set to keep banned IPs/networks, you can easily script loading them from a simple text file or using ipset save/restore to initialize the set from a previous state.

Yes, I do that with my normal iptables rules, which I periodically clear. However, I did find that ipsets persist even when iptables is terminated, which is good. I suppose unless I do save ipsets they would not persist through reboot.

Quote:

Then, assuming your set is named bad_people, you might make this your first rule:

Code:

iptables -t raw -I PREROUTING -m set --match-set bad_people src -j DROP

Anything in that set will never be be seen again...

Here's what I've done, in my iptables script:

Code:

ipset -exist create blockrdc hash:net hashsize 1024 counters

iptables -I INPUT -m set --match-set blockrdc src -j DROP
iptables -I FORWARD -m set --match-set blockrdc src -j DROP

That follows your earlier examples, but using the setname "blockrdc". Your latest example uses only "-t raw -I PREROUTING" as the iptables rule. Would it be better/correct to use that rule instead of what I've got? Or add that rule to what I have? Recall that the iptables rules escaping my normal DROPs in my bad_people chain were of the form:

Code:

iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 1912 -j DNAT --to-destination 192.168.0.58:3389
iptables -A FORWARD -i eth1 -d 192.168.0.58 -p tcp --dport 3389 -j ACCEPT
iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -d 192.168.0.58 -p tcp --dport 3389 -j SNAT --to-source 192.168.0.2
iptables -t nat -A POSTROUTING -o eth1 -d 192.168.0.58 -p tcp --dport 3389 -j SNAT --to-source 192.168.0.2

When I detect an attempted break-in (via a script) I will then do:

Code:

ipset add blockrdc $range

where "$range" is like: 184.57.60.0/24. I did 'ipset test blockrdc 184.57.111.128' after adding that range and got back, "184.57.111.128 is in set blockrdc." So that worked.

I guess my only remaining question has to do with your PREROUTING example and whether my --match rules are adequate or correct.