Why does interface specific iptables rule not work

anindyameister · 08-06-2014, 04:10 AM

I have a RHEL6 box with two NICs where I plan to install oracle database and wish to use eth0 as a dedicated interface for the database traffic and eth1 for everything else. I have assigned static IPs 192.168.0.10 and 192.168.0.11 to eth0 and eth1 respectively. The default iptables rules on RHEL6 initially allow all established traffic on all interfaces then allows new connections to port 22 and then rejects everything else. I added a similar rule like port 22 for port 1521 and i was able to connect over ssh and oracle client to both eth0 and eth1. When I added the extra option of "-i eth1" to port 22 rule and "-i eth0" to port 22 rule, ssh was unable to connect to either interface. Ultimately I modified the rules to check for the destination IP instead of mentioning interface. Now my question is why do the rules not work when configured against specific interfaces ?

Here's My rules of what I was trying to do

Code:

:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -i eth1 -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A INPUT -i eth0 -m state --state NEW -m tcp -p tcp --dport 1521 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

The above configuration didn't work so I had to modify the rules to check destination IP

Code:

:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -p icmp -j ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dst 192.168.0.11 --dport 22 -j ACCEPT
-A INPUT -m state --state NEW -m tcp -p tcp --dst 192.168.0.10 --dport 1521 -j ACCEPT
-A INPUT -j REJECT --reject-with icmp-host-prohibited
-A FORWARD -j REJECT --reject-with icmp-host-prohibited

I want to know what I am doing wrong.

keefaz · 08-06-2014, 04:37 AM

Maybe on the 1, you have to duplicate
-A INPUT -m state --state ESTABLISHED,RELATED
for both interfaces eg

Code:

-A INPUT -i eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -i eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT

edit,

You could set the eth1 IP at the ListenAddress directive in /etc/ssh/sshd_config

Also, " iptables -L -v -n " can be usefull

anindyameister · 08-06-2014, 07:20 AM

Quote:

Originally Posted by keefaz

Maybe on the 1, you have to duplicate
-A INPUT -m state --state ESTABLISHED,RELATED
for both interfaces eg

Code:

-A INPUT -i eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT
-A INPUT -i eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT

edit,

You could set the eth1 IP at the ListenAddress directive in /etc/ssh/sshd_config

Also, " iptables -L -v -n " can be usefull

But the man page states that without -i the rule will apply to all interfaces.

Code:

 [!] -i, --in-interface name
              Name  of an interface via which a packet was received (only for packets entering the INPUT, FORWARD and PREROUTING chains).  When the "!" argument is used before the interface name, the sense is inverted.  If the interface name ends in a "+", then any interface which  begins with this name will match.  If this option is omitted, any interface name will match.

And making sshd listen on eth1 doesn't prevent any other traffic from arriving at eth0

eSelix · 08-06-2014, 07:47 AM

Are the interface names correct, is there any error when you manually invoke command? Like:

Code:

iptables -A INPUT -i eth1 -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT

I see on Ubuntu there is no error if I pass incorrect interface name, so check it for example by "ifconfig" or "ip".

anindyameister · 08-06-2014, 07:55 AM

Quote:

Originally Posted by eSelix

Are the interface names correct, is there any error when you manually invoke command? Like:

Code:

iptables -A INPUT -i eth1 -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT

I see on Ubuntu there is no error if I pass incorrect interface name, so check it for example by "ifconfig" or "ip".

ifconfig shows three interfaces "eth0", "eth1" and "lo"

grim76 · 08-06-2014, 08:53 AM

Can you give us the output of:

Code:

ip addr

anindyameister · 08-06-2014, 09:03 AM

Here it is

Code:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:23:7d:33:fa:1c brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.10/24 brd 192.168.0.255 scope global eth0
    inet6 fe80::223:7dff:fe33:fa1c/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:23:7d:33:fa:5c brd ff:ff:ff:ff:ff:ff
    inet 192.168.0.11/24 brd 192.168.0.255 scope global eth1
    inet6 fe80::223:7dff:fe33:fa5c/64 scope link
       valid_lft forever preferred_lft forever

keefaz · 08-06-2014, 09:06 AM

Quote:

Originally Posted by anindyameister

And making sshd listen on eth1 doesn't prevent any other traffic from arriving at eth0

No, but the SSH connection problem can be at software level, not at kernel packet handling level

Eg, with no address specified for ListenAddress in sshd_config, how can you know sshd will listen to the right address (IP of eth1, 192.168.0.11)? By default it will listen to one address if you don't specify multiple ListenAddress lines.

anindyameister · 08-06-2014, 09:10 AM

Quote:

Originally Posted by keefaz

No, but the SSH connection problem can be at software level, not at kernel packet handling level

Eg, with no address specified for ListenAddress in sshd_config, how can you know sshd will listen to the right address (IP of eth1, 192.168.0.11)? By default it will listen to one address if you don't specify multiple ListenAddress lines.

Well in it's initial state, i was able to connect over ssh using putty to both IP addresses.

keefaz · 08-06-2014, 09:15 AM

Quote:

Originally Posted by anindyameister

Well in it's initial state, i was able to connect over ssh using putty to both IP addresses.

You mean with no iptables rules, you can connect with ssh to both addresses?

anindyameister · 08-06-2014, 09:17 AM

Quote:

Originally Posted by keefaz

You mean with no iptables rules, you can connect with ssh to both addresses?

Yes. I can connect with no rules and also if I remove the --dst or -i options from the existing rules.

keefaz · 08-06-2014, 09:24 AM

Could you log the rejected packet for testing, eg use:

Code:

iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -i eth1 -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -i eth0 -m state --state NEW -m tcp -p tcp --dport 1521 -j ACCEPT
iptables -A INPUT -j LOG --log-prefix "[Rejected] "
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited

run 'tail -f /var/log/syslog' loggued as root in server and try some ssh connection with a client

anindyameister · 08-06-2014, 09:58 AM

Quote:

Originally Posted by keefaz

Could you log the rejected packet for testing, eg use:

Code:

iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -p icmp -j ACCEPT
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -i eth1 -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -i eth0 -m state --state NEW -m tcp -p tcp --dport 1521 -j ACCEPT
iptables -A INPUT -j LOG --log-prefix "[Rejected] "
iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited

run 'tail -f /var/log/syslog' loggued as root in server and try some ssh connection with a client

I added the log statement and what I got was really weird.

Code:

Aug  6 20:08:55 db1 kernel: [Rejected] IN=eth0 OUT= MAC=00:23:7d:33:fa:1c:00:22:4d:a5:56:fd:08:00 SRC=192.168.0.30 DST=192.168.0.10 LEN=48 TOS=0x00 PREC=0x00 TTL=128 ID=9204 DF PROTO=TCP SPT=60915 DPT=22 WINDOW=8192 RES=0x00 SYN URGP=0 
Aug  6 20:08:46 db1 kernel: [Rejected] IN=eth0 OUT= MAC=00:23:7d:33:fa:1c:00:22:4d:a5:56:fd:08:00 SRC=192.168.0.30 DST=192.168.0.11 LEN=44 TOS=0x00 PREC=0x00 TTL=128 ID=9149 DF PROTO=TCP SPT=60912 DPT=22 WINDOW=8192 RES=0x00 SYN URGP=0

even though the destination addresses are different, they are received on eth0.

And eth1 is receiving only the below traffic one in a while

Code:

Aug  6 20:15:09 db1 kernel: [Rejected] IN=eth1 OUT= MAC=01:00:5e:00:00:01:00:23:7d:33:65:ac:08:00 SRC=0.0.0.0 DST=224.0.0.1 LEN=32 TOS=0x00 PREC=0xC0 TTL=1 ID=0 DF PROTO=2

I tried running arping from another box and it returned the same MAC for both IPs. Now i'm confused.

Code:

# arping 192.168.0.10
ARPING 192.168.0.10 from 192.168.0.191 eth0
Unicast reply from 192.168.0.10 [00:23:7D:33:FA:1C]  0.725ms
Unicast reply from 192.168.0.10 [00:23:7D:33:FA:1C]  0.688ms

# arping 192.168.0.11
ARPING 192.168.0.11 from 192.168.0.191 eth0
Unicast reply from 192.168.0.11 [00:23:7D:33:FA:1C]  0.796ms
Unicast reply from 192.168.0.11 [00:23:7D:33:FA:1C]  0.689ms

anindyameister · 08-06-2014, 10:07 AM

Here is my routing table.

Code:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.0     *               255.255.255.0   U     0      0        0 eth0
192.168.0.0     *               255.255.255.0   U     0      0        0 eth1
link-local      *               255.255.0.0     U     1002   0        0 eth0
link-local      *               255.255.0.0     U     1003   0        0 eth1
default         192.168.0.1    0.0.0.0          UG    0      0        0 eth0

smallpond · 08-06-2014, 10:40 AM

If you want to use two NICs on the same subnet, you need to do policy based routing using the iproute2 tools. iptables alone won't do what you want.

http://kindlund.wordpress.com/2007/1...utes-in-linux/