[SOLVED] iptables troubleshooting icmp and best place to log /var/log/messages or /var/log/iptables
Linux - SecurityThis forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
From there, using tail -f messages with the following:
Code:
tail -f /var/log/messages
Currently, the logs are going into /var/log/messages. I've seen others setup logging into /var/log/iptables.
Do I have to create a directory under /var/log for this if I want to switch to this? Create files too, or are those created automatically?
In addition to modifying /etc/syslog.conf with the following
Code:
kern.=debug /var/log/iptables
Is there anything, such as level of logging that needs to be added?
And then I've added a drop rule at the bottom of RH-Firewall-1-INPUT
Code:
iptables -A RH-Firewall-1-INPUT -j DROP
Once I add this, I'm unable to ping servers in my subnet and the various servers and users can't ping back to this server. All other services (SSH/Oracle) work ok.
Here are the rules
Code:
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 26433 3489k RH-Firewall-1-INPUT all * * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
num pkts bytes target prot opt in out source destination
1 0 0 RH-Firewall-1-INPUT all - * * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 27333 packets, 3733k bytes)
num pkts bytes target prot opt in out source destination
1 10 840 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 icmp type 8
2 85 23268 ACCEPT tcp -- 8 eth0 0.0.0.0/0 0.0.0.0/0 multiport sport 22,1158,1521 state ESTABLISHED
Chain RH-Firewwall-1-INPUT (2 references)
num pkts bytes target prot opt in out source destination
1 28712 4091k ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
2 46 5767 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 limit: avg 1/min burst 5 LOG flags 0 level 4 prefix ' ***IPTABLES LOGGED ****'
3 0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 icmp type 0
4 1 100 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22
5 131 9644 ACCEPT tcp -- eth0 * 192.168.10.0/24 0.0.0.0/0 multiport dports 22,1158,1521 state NEW, ESTABLISHED
6 0 0 ACCEPT tcp -- * * xxx.xx.xxx.xx 0.0.0.0/0 state NEW,ESTABLISHED tcp dpt:1521
7 613 59410 DROP all -- * * 0.0.0.0/0 0.0.0.0/0
UPDATE: The logging is picking up alot of broadcast traffic. I believe this traffic is normal, correct?
Last edited by JockVSJock; 01-21-2016 at 03:24 PM.
And then I've added a drop rule at the bottom of RH-Firewall-1-INPUT
Code:
iptables -A RH-Firewall-1-INPUT -j DROP
Once I add this, I'm unable to ping servers in my subnet and the various servers and users can't ping back to this server. All other services (SSH/Oracle) work ok.
Best easiest way to post a firewall rule set is to post 'iptables-save' output as any rule file may not reflect the rules currently loaded in memory. Best easiest way to troubleshoot firewall problems is to make specific "-j LOG" rules precede any DROP or REJECT rules. A rule like '-A RH-Firewall-1-INPUT -m limit --limit 1/min -j LOG --log-prefix " *** IPTABLES LOGGED ***"' is not placed logically (though sure you may just want to log everything) and has no filter (remote address, protocol, port, etc, etc) so will catch way too much. Your rule set could be improved I think.
Best easiest way to troubleshoot firewall problems is to make specific "-j LOG" rules precede any DROP or REJECT rules. A rule like '-A RH-Firewall-1-INPUT -m limit --limit 1/min -j LOG --log-prefix " *** IPTABLES LOGGED ***"' is not placed logically (though sure you may just want to log everything) and has no filter (remote address, protocol, port, etc, etc) so will catch way too much. Your rule set could be improved I think.
I thought that putting this rule upfront would be the best that way I could see what packets were hitting up against it.
The odd thing too is that there is alot of broadcast traffic that is getting picked up in /var/log/messages.
I'm always open to feedback for IPTables, how else does one improve?
:INPUT ACCEPT [2699:278262]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A OUTPUT -p icmp -m icmp --icmp--type 8 -j ACCEPT
-A OUTPUT -s 192.168.10.0/255.255.255.0 -o eth0 -p tcp -m multiport --sports 22,1158,1521 -m state --state ESTABLISHED -j ACCEPT
-A OUTPUT -d xxx.xxx.xxx.xxx -p tcp -m state --state NEW,ESTABLISHED -m tcp --dport 1521 -j ACCEPT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -s 192.168.10.0/255.255.255.0 -i eth0 -p tcp -m multiport --dports 22,1158,1521 -m state --state NEW,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -s xxx.xx.xxx.xx -p tcp -m state --state NEW,ESTABLISHED -m tcp --dport 1521 -j ACCEPT
-A RH-Firewall-1-INPUT -m limit --limit 1/min -j LOG --log-prefix " *** IPTABLES LOGGED **** "
COMMIT
Come to find out, it wss an issue with VMWare. We have a NIC that faces a public internet, and this system is in a test environment. Daily we turn on the NIC and once we are done for the day, we turn off the NIC. Doing this seems to cause an issue. So we are going to turn off the interface and then the NIC and vice versa daily to see if that will fix the issue.
Come to find out, it wss an issue with VMWare. We have a NIC that faces a public internet, and this system is in a test environment. Daily we turn on the NIC and once we are done for the day, we turn off the NIC. Doing this seems to cause an issue.
Because it going into our production environment and this server is in our development environment. We want to keep both separate and pure and so when we are doing working with this server, we turn off the NIC in vCenter that points into production. The "xxx.xxx.xxx.xxx" represents the IP address from prod to dev and we want to regulate traffic to only that server only. We are pulling data from production to dev. I've been running Wireshark to monitor the traffic to make sure no one or processes can't get in.
Hopefully thought, you can look at the IPTables that I wrote up and provide feedback.
Last edited by JockVSJock; 02-02-2016 at 01:36 PM.
Sure but as I have posted a (what I think would be a sane rule set) rule set I have to first ask you to post your motivation for changing it the way you did.
-A OUTPUT -p icmp -m icmp --icmp--type 8 -j ACCEPT #allow icmp echo-reply traffic
-A OUTPUT -s 192.168.10.0/255.255.255.0 -o eth0 -p tcp -m multiport --sports 22,1158,1521 -m state --state ESTABLISHED -j ACCEPT #allow for ssh and oracle traffic connections in the development env.
-A OUTPUT -d xxx.xxx.xxx.xxx -p tcp -m state --state NEW,ESTABLISHED -m tcp --dport 1521 -j ACCEPT #allow for only oracle traffic from this one server in our production env. I've since channged NEW, ESTABLISHED to ESTABLISHED, RELATED
-A RH-Firewall-1-INPUT -i lo -j ACCEPT #allow loopback traffic
-A RH-Firewall-1-INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT #allow icm echo-request
-A RH-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT #allow new ssh connections
-A RH-Firewall-1-INPUT -s 192.168.10.0/255.255.255.0 -i eth0 -p tcp -m multiport --dports 22,1158,1521 -m state --state NEW,ESTABLISHED -j ACCEPT #from the dev environment, allow ssh and oracle traffice
-A RH-Firewall-1-INPUT -s xxx.xx.xxx.xx -p tcp -m state --state NEW,ESTABLISHED -m tcp --dport 1521 -j ACCEPT #from production, only allow oracle traffic from this one ip address
-A RH-Firewall-1-INPUT -m limit --limit 1/min -j LOG --log-prefix " *** IPTABLES LOGGED **** " #log iptable to /var/log/messages, however I'm finding this is even logging broadcast traffic and other traffic from other servers
COMMIT
Thanks for the insight. Here's my rule set explained:
Code:
# 3 lines below are default policies:
:INPUT ACCEPT [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
# 1 line below is a custom chain:
:RH-Firewall-1-INPUT - [0:0]
# this line is typically stock RHEL / CentOS. Serves no purpose here, could filter everything in the "INPUT" chain to simplify things...
-A INPUT -j RH-Firewall-1-INPUT
# You're not (knowingly?) forwarding anything, so let remote hosts know you're not accepting:
-A FORWARD -j REJECT
# Netfilter uses "first rule match" so best order in what traffic is expected to hit the table the most often.
# In essence that's not the loop back device but once you get "lo" out of the way you need not specify the Ethernet
# device name on consecutive rules if you're only running one:
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
# Get any connections already known to Netfilter out of the way:
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# New connections to allow:
-A RH-Firewall-1-INPUT -s 192.168.10.0/255.255.255.0 -p tcp -m multiport --dports 22,1158,1521 -m state --state NEW -j ACCEPT
-A RH-Firewall-1-INPUT -s xxx.xx.xxx.xx -p tcp -m state --state NEW -m tcp --dport 1521 -j ACCEPT
# Ping and pong:
-A RH-Firewall-1-INPUT -s 192.168.10.0/255.255.255.0 -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A RH-Firewall-1-INPUT -s 192.168.10.0/255.255.255.0 -p icmp -m icmp --icmp-type 8 -j ACCEPT
# Log any ICMP traffic only that's not type 0 or 8:
-A RH-Firewall-1-INPUT -s 192.168.10.0/255.255.255.0 -p icmp -m limit --limit 1/sec -j LOG --log-prefix "other_ICMP"
# DROP anything else in the chain:
-A RH-Firewall-1-INPUT -j DROP
# OUTPUT chain is empty as default policy already is ACCEPT:
COMMIT
If there's two things you should take away from this it's that rule ordering matters and that you need to apply filters carefully. This rule is what you could call a "greedy match" filter as it doesn't limit to either remote addresses or protocols and therefore matches everything:
- dropped reject statement from the Forward Rule, because it makes no sense to reply to something that is not being forwarded
However, more questions.
- it sounds like the important traffic (22, 1521), those rules should be closer to the top, correct?
- the rules for loopback and icmp, should be closer to the bottom. The traffic is important, however it can lower to the bottom, correct?
- A INPUT -j RH-Firewall-1-INPUT. I'm confused. I believe that I've asked this question on here, however it sounds like the RH-Firewall-1-INPUT rule is redundant and can be handled by INPUT. If so, how does one fix this in the future? Can you just delete the RH-Firewall-1-INPUT and all rules that were under this, just move under the INPUT rule? Also deleted any references of RH-Firewall-1-INPUT in FORWARD and OUTPUT?
- I don't understand your statement on the logging that I have setup, can you please clarify?
thanks
Last edited by JockVSJock; 02-08-2016 at 03:20 PM.
RH-Firewall-1-INPUT is Redhat's way of trying to be fancy. You can move all your rules to INPUT and then delete the entire chain of RH-Firewall-1-INPUT from your rules.
The first rule should always be ESTABLISHED,RELATED as this is the rule that is always hit after a connection is established. I try to order my rules in numerical order make for finding things a lot easier. I usually place the lo rule right after the ESTABLISHED,RELATED rule followed by the ICMP rules as they is no established connection for them.
it sounds like the important traffic (22, 1521), those rules should be closer to the top, correct?
Yes but below the loopback and the ESTABLISHED,RELATED rule.
Quote:
Originally Posted by JockVSJock
the rules for loopback and icmp, should be closer to the bottom.
Not if you look at my rule set. Loop back should be first. I explained that. If you can't wrap your head around that just take it as a given.
Quote:
Originally Posted by JockVSJock
it sounds like the RH-Firewall-1-INPUT rule is redundant and can be handled by INPUT.
Yes, as said by lazydog. BTW "move" just means 'sed -i "s|RH-Firewall-1-||g" /etc/sysconfig/iptables && iptables-restore < /etc/sysconfig/iptables;' (your choice of making a backup or not).
Quote:
Originally Posted by JockVSJock
Also deleted any references of RH-Firewall-1-INPUT in FORWARD and OUTPUT?
What do you not "get" from the "OUTPUT chain is empty as default policy already is ACCEPT" explanation? If you accept all traffic then explicit "ACCEPT" rules make no sense. They accomplish nothing.
Quote:
Originally Posted by JockVSJock
I don't understand your statement on the logging that I have setup, can you please clarify?
You are interested in ICMP messages. However your logging rule does not filter anything. If you do not filter then you get all traffic.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.
