Problem with port forwarding (NAT) on FC 5

heberrdacruz · 07-26-2006, 01:46 PM

Quote:

Originally Posted by blackhole54

Did you remember to change the line

Code:

EXTIF_IP="200.17.x.x"

to reflect your actual IP address?

Yes, but thanks anyway for the reminder.

Quote:

If you have and it still doesn't work, you should look for log entries containing "FOWARD DROP: "

I have and there were none.

Quote:

These reflect the increased security of your firewall. In these cases it blocked packets sent to it at ports tcp/8080 and udp/610. If your log is getting swamped by dropped packet entries, you can limit the number of logged packets by appending something like

This made me realized that I failed to mention other services that this machine provides to the internal network. Would those lines do what is needed?

Code:

# SMTP on internal interface:
$IPTABLES -A INPUT -p TCP -i $INTIF -s $INTIF_NET --dport 25 \
-m state --state NEW -j ACCEPT

# HTTP on internal interface:
$IPTABLES -A INPUT -p TCP -i $INTIF -s $INTIF_NET --dport 80 \
-m state --state NEW -j ACCEPT

# squid on internal interface:
$IPTABLES -A INPUT -p TCP -i $INTIF -s $INTIF_NET --dport 8080 \
-m state --state NEW -j ACCEPT

I don’t know what udp/610 would be used for.

Quote:

Code:

--limit 4/minute

to the the command(s) with -j LOG in it(them). Of course, you will then lose information about the packets that were blocked but not logged. See the iptables man page for more info.

Thank you for the information.

heberrdacruz · 07-26-2006, 03:36 PM

As you suggested, I played with tcpdump. The odd thing I noticed on the external interface was that inquires were made by machines on the LAN about other machines on the LAN. They received no reply except when it was regarding the machine itself.

Code:

16:57:15.432063 arp who-has 192.168.30.187 tell 192.168.30.186
16:57:16.927388 arp who-has 192.168.30.15 tell 192.168.30.187
16:57:16.927410 arp reply 192.168.30.15 is-at 00:50:2c:xx:xx:xx
16:57:27.645786 arp who-has 192.168.30.188 tell 192.168.30.187
16:57:31.661218 arp who-has 192.168.30.99 tell 192.168.30.14

The output format is a little different from yours.
On the internal interface, things were as I expected

Code:

17:04:04.289561 arp who-has 192.168.30.15 tell 192.168.30.160
17:04:04.289580 arp reply 192.168.30.15 is-at 00:01:01:xx:xx:xx
17:04:04.350302 arp who-has 192.168.30.15 tell 192.168.30.181
17:04:04.350321 arp reply 192.168.30.15 is-at 00:01:01:xx:xx:xx
17:04:04.543113 arp who-has 192.168.30.20 tell 192.168.30.15
17:04:04.543219 arp reply 192.168.30.20 is-at 00:01:01:xx:xx:xx

Somewhere I read about setting net.ipv4.conf.all.log_martians to 1. I did that and it has produced quite a bit of output. Here is a sample.

Code:

Jul 25 18:17:52 newton kernel: martian source 200.17.our.broadcast from 200.17.another.university, on dev eth1
Jul 25 18:17:53 newton kernel: martian source 200.17.our.university from 200.17.also-our.university, on dev eth1
Jul 25 18:18:45 newton kernel: martian source 192.168.30.99 from 192.168.30.14, on dev eth0
Jul 25 18:18:47 newton kernel: martian source 192.168.30.255 from 192.168.30.136, on dev eth0
Jul 25 18:18:47 newton kernel: martian source 192.168.30.255 from 192.168.30.136, on dev eth0
Jul 25 18:18:51 newton kernel: martian source 192.168.30.255 from 192.168.30.136, on dev eth0
Jul 26 12:05:39 newton kernel: martian source 200.17.self.xxx from 200.17.self.xxx, on dev eth1
Jul 26 12:05:39 newton kernel: martian source 200.17.core.builder from 200.17.self.xxx, on dev eth1
Jul 26 17:08:35 newton kernel: martian source 192.168.30.190 from 192.168.30.15, on dev eth0
Jul 26 17:09:08 newton kernel: martian source 192.168.30.181 from 192.168.30.15, on dev eth0

I hope the reformatting I did is clear and safe. I found it puzzling.

blackhole54 · 07-27-2006, 02:05 AM

Quote:

Originally Posted by heberrdacruz

As you suggested, I played with tcpdump. The odd thing I noticed on the external interface was that inquires were made by machines on the LAN about other machines on the LAN. They received no reply except when it was regarding the machine itself.

This is correct. (Note: I am not sure about correct terminology for ARP packets. I'll just call them by names that mimic tcpdump output.) The "who has" packets must be broadcast so that all interfaces see them, so that the the correct one can respond. However, the sender of the "arp reply" packet knows who to send the response to, so the response is not broadcast and no other interface sees it. (Check definitions of broadcast, multicast, and unicast if you are not familiar.)

Quote:

Somewhere I read about setting net.ipv4.conf.all.log_martians to 1. I did that and it has produced quite a bit of output.

That was a good idea. These "martians" that are getting logged are the packets that rp_filter is knocking out. This is apprently why there were no "FORWARD DROP" log entries -- the packets didn't get that far. Check my thinking here, but are we now seeing 100% of the packets coming in on the wrong interface?!! The cables aren't reversed are they?

One of the things you can do with itpables is check how many packets have matched a particular rule. (These are cummulative until manually reset.) Give this command with the options "-nvL" and pipe the result to less. The first number in each line is the number of packets that have matched that rule. (The second number is the number of cummulative bytes these packets had.) So you can, for example, check to see if any packets actually matched the SNAT rule. If not, there is either an error in the rule, the packets didn't make it to the server in the first place, they were knocked out by rp_filter, or they were knocked out by another rule in the firewall. The POSTROUTING chain is the last thing that happens before the packet leaves the box.

If rp_filter is knocking out all of your packets, your only choices are to wait until you have things coming in on the right interfaces, or turn rp_filter off. (Note to win32sux: I am not suggesting this is a good thing to do. It is a workaround for a bad situation.) However, if things are coming in on the wrong interface, even if you turn off rp_filter, the packets will still get blocked by the firewall rules unless you impliment my suggestions in post #9.

Quote:

This made me realized that I failed to mention other services that this machine provides to the internal network. Would those lines do what is needed?

The lines you showed look right to me. In post #15 it sounded like you were also trying to ssh into this box. If so, you will need a rule for that also. The standard ssh port is tcp/22. BTW, I don't know if you are aware of the file /etc/services which lists standard ports for a bunch of services. That may give you a clue about udp/610. Also, the Internet Storm Center provides a search box where you can enter a port number and it will tell you standard services for that port, known malware that uses that port, and show you the activity reported for that port from firewall logs that are submitted to DSHIELD.

Also in #15 you said something about "home page." I am not sure what you were talking about, but maybe this is another rule you need to create for your firewall?

heberrdacruz · 07-27-2006, 04:11 PM

Quote:

Originally Posted by blackhole54

The cables aren't reversed are they?

I afraid they are not. As the switches are interconnected I don’t know if this means a lot.
It seems that I did something very wrong when I added rules to win32sux's script. When I run the ssh session I had froze and I had to restart the machine to get it working again. I have gone back to the previous situation. I will have to check my modifications carefully. As I said earlier, without my modifications it seemed OK except for some services that I forgot to mention.
Although NAT runs (on occasions) I found nothing under the SNAT rule on iptables statistics.

Code:

[root@newton ~]# iptables -nvL
Chain INPUT (policy ACCEPT 2437K packets, 832M bytes)
 pkts bytes target     prot opt in     out     source               destination


Chain FORWARD (policy DROP 9214 packets, 1275K bytes)
 pkts bytes target     prot opt in     out     source               destination

1126K 1641M ACCEPT     all  --  eth0   eth1    0.0.0.0/0            0.0.0.0/0
        state RELATED,ESTABLISHED
 784K   45M ACCEPT     all  --  eth1   eth0    0.0.0.0/0            0.0.0.0/0

 9214 1275K LOG        all  --  *      *       0.0.0.0/0            0.0.0.0/0
        LOG flags 0 level 4

Chain OUTPUT (policy ACCEPT 2574K packets, 957M bytes)
 pkts bytes target     prot opt in     out     source               destination

Thank you for the information on /etc/services and ISC. I had a look on /etc/services and there are several services I had forgotten about. I am still working on that. On this matter I have some questions.
I noticed there were no rules for udp only for tcp. Why?
This machine runs internally NFS and NIS. Which ports are needed?

Quote:

Also in #15 you said something about "home page."

I meant that the http service provided by this server was no longer accessible.
Thank you very much for your comments.

blackhole54 · 07-27-2006, 05:49 PM

My mistake about listing the packet counts. As you can see, the command I told you to run lists the INPUT, OUTPUT, and FORWARD chains. To view PREROUTING and POSTROUTING you must run

Code:

iptables -t nat -nvL

This is how you see the counts for SNAT. Sorry about that.

I repeat, if you want to ssh into this box, you must provide a rule for it in win32sux's script.
If you need to ssh and access port 80 from your LAN, for right now I would add the following lines to that script.

Code:

$IPTABLES -A INPUT -p TCP -s $INTIF_NET --dport 80 \
    -m state --state NEW -j ACCEPT
$IPTABLES -A INPUT -p TCP -s $INTIF_NET --dport 22 \
   -m state --state NEW -j ACCEPT

Note that I removed "-i $INTIF." This might make this less secure that the original script, but with the problems you are having with packets coming into the wrong interfaces it appears to be the only way to allow the packets through. You should go back to the way the original script did it when you get packets coming in on the correct interfaces. I believe (again, for the time being) you need to set net.ipv4.conf.default.rp_filter to zero in /etc/sysctl.conf.

blackhole54 · 07-27-2006, 09:32 PM

Hi heberrdacruz,

We've been discussing a lot of details, and I think we have kind of gotten lost in the forrest. So in this post, I would like to back up a little, summarize where we have been, and propose a firewall script for your temporary use.

You originally presented a problem where you were successfully SNATing packets from your LAN to the Internet, and receiving and FORWARDing the responses. But after a time this would stop working. You could start it working again by issuing a network restart on your server.

It was soon determined that the problem was that packets were coming into the server on wrong interfaces. The exact reason for this, and why it could be straightened out by the network restart, was never determined, but it turned out a contributing factor might be what can most charitably be called an unusual network configuration. This configuration is expected to be corrected in the near future when new switches are purchased.

Also, early on I commented that the firewall script you were using was permitting unlimited access to your box on the INPUT chain from both the Internet and your LAN. I suggested a script for tightening this down some. Win32sux joined the conversation and proposed another script which refined and extended the previous scripts. This script was never successfully used, apparently because of the packets coming in on the wrong interfaces and because some needed services had not been provided for.

----

I believe that is where we are now. You can pursue any additional ideas you have about correcting what interfaces the packets are coming in on. But to move this forward, I am listing below yet another script that I hope will work as a stopgap measure. It does not check which interface packets come in on, and it turns off rp_filter. As I note in its comments, you should correct your situation as soon as possible, and when you do, you should go to a more secure script.

I have also changed the way the script specifies which ports on your server may be accessed. These ports are now listed in the variables EXT_TCP and INT_TCP to make adding new allowed TCP services as easy as editing these two lines. (As you have specified no udp ports that needed access, I did not bother with that.) If you want a service available both from the Internet and LAN, include it in both lists as I have done with port 80.

I have tested portions of this script, but I do not have a convenient way to test it as a whole. If there are errors you can't figure out, let me know.

Code:

#!/bin/sh

#  Firewall script

#  >>>  NOTE:  This script is designed for ***temporary*** use to  <<<
#  >>>     overcome problems caused by packets not coming in on    <<<
#  >>>     their proper interfaces.                                <<<

#  For this reason this script disables rp_filtering and does *not* expect
#  packets to be on a particular ethernet interface.  When the underlying
#  problem is corrected, a more restrictive script should be used.

#  Define interfaces and IP addresses for Internet and LAN

EXTIF="eth0"
EXTIF_IP="200.17.x.x"

INTIF="eth1"
INTIF_NET="192.168.30.0/24"

#  Define the TCP ports the should be accessable from Internet and LAN
#  These should be space seperated, quoted strings.

#  Internet Access:
#      80   http
#      25   smtp  (mail)
#     

#  Internal LAN Access:
#     110   POP3
#     143   IMAP
#      22   ssh
#      80   http
#      25   smtp  (mail)
#    8080   squid

EXT_TCP="80 25"
INT_TCP="110 143 22 80 25 8080"

#  Command definitions:

IPTABLES="/sbin/iptables"
MODPROBE="/sbin/modprobe"

#  Turn rp_filter off

for f in /proc/sys/net/ipv4/conf/*/rp_filter; do
   echo 0 > $f
done

#  Set policies for each chain

$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -P FORWARD DROP
$IPTABLES -t nat -P PREROUTING ACCEPT
$IPTABLES -t nat -P POSTROUTING ACCEPT
$IPTABLES -t nat -P OUTPUT ACCEPT
$IPTABLES -t mangle -P PREROUTING ACCEPT
$IPTABLES -t mangle -P OUTPUT ACCEPT

#  Flush all standard chains and delete all user chains

$IPTABLES -F
$IPTABLES -F -t nat
$IPTABLES -F -t mangle

$IPTABLES -X
$IPTABLES -X -t nat
$IPTABLES -X -t mangle

#  Load modules for iptables

$MODPROBE ip_conntrack_ftp
$MODPROBE ip_nat_ftp
$MODPROBE ip_conntrack_irc
$MODPROBE ip_nat_irc

#  --------------  INPUT Chain Firewall Rules ---------------
#
#  Accept all established connections and all loopback packets

$IPTABLES -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A INPUT -i lo -j ACCEPT

#  Rules which allow new Internet connections

for port in $EXT_TCP; do
   $IPTABLES -A INPUT -p TCP -s $EXTIF_IP --dport $port \
      -m state --state NEW -j ACCEPT
done

#  Rules which allow new LAN connections

for port in $INT_TCP; do
   $IPTABLES -A INPUT -p TCP -s $INTIF_NET --dport $port \
      -m state --state NEW -j ACCEPT
done

#  Policy drops everything else.  But first log them.

$IPTABLES -A INPUT -j LOG --log-prefix"INPUT DROP: "

#  --------------  FORWARD Chain Firewall Rules ---------------
#
#  Accept establish connections.
#  Accept new connections from $LAN that are destined for the Internet
#  Log any packets that will be dropped

$IPTABLES -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -o $EXTIF -s $INTIF_NET \
   -m state --state NEW -j ACCEPT
$IPTABLES -A FORWARD -j LOG --log-prefix "FORWARD DROP: "


#  --------------  POSTROUTING Chain Firewall Rules ---------------
#
#  SNAT all LAN originated packets destined for the Internet.


$IPTABLES -t nat -A POSTROUTING -o $EXTIF -s $INTIF_NET \
   -j SNAT --to $EXTIF_IP

heberrdacruz · 07-28-2006, 07:32 AM

Never mind about the iptables command. The output I obtained is below

Code:

Chain PREROUTING (policy ACCEPT 59681 packets, 4501K bytes)
 pkts bytes target     prot opt in     out     source               destination

Chain POSTROUTING (policy ACCEPT 92108 packets, 6731K bytes)
 pkts bytes target     prot opt in     out     source               destination
 7475  554K SNAT       all  --  *      eth0    192.168.30.0/24      0.0.0.0/0           to:200.17.xxx.xxx

Chain OUTPUT (policy ACCEPT 92108 packets, 6731K bytes)
 pkts bytes target     prot opt in     out     source               destination

Quote:

Originally Posted by blackhole54

You should go back to the way the original script did it when you get packets coming in on the correct interfaces.

OK I will have to save this information for future use.

Quote:

I believe (again, for the time being) you need to set net.ipv4.conf.default.rp_filter to zero in /etc/sysctl.conf.

This seems to have made an effect. Last night I set net.ipv4.conf.default.rp_filter to zero on newton. This morning, NAT was working.
The installation default is to set this variable to one. The server with RH 9 (called magnon) also has this setting and NAT always seemed to work. I have now looked more closely at its log and I noticed occasional messages of the same type.

Code:

Jul 28 08:46:36 magnon kernel: IN=eth1 OUT=eth1 SRC=212.58.235.174 DST=192.168.30.118 LEN=48 TOS=0x00 PREC=0x00 TTL=44 ID=34643 DF PROTO=TCP SPT=80 DPT=1405 WINDOW=64240 RES=0x00 ACK SYN URGP=0

I verified that magnon also shows the behavior that when lots of messages of this type appear NAT doesn’t work. However, contrary to what was happening with newton, it recovers itself. The messages stop by themselves and NAT works again. Repeating, magnon has net.ipv4.conf.default.rp_filter = 1. newton is showing the same behavior as magnon. I ping a computer at our computer center every minute. The first 33 came back, then 5 didn’t and up to the 65th it keeps coming back. The failure coincided with many messages on the log.

heberrdacruz · 07-28-2006, 08:47 AM

I am most grateful for your help.
I think it is great the way you implemented the port addition. As this machine is the internal NIS and NFS I need to know about what ports are needed before installing your script otherwise no one will be able to use our network. The command rpcinfo -p localhost shows lots of ports. Shall add all of them? Searching the internet I found a page saying that NFS uses random ports. Is that so?
When looking at /etc/services I noticed that services, like http, use both tcp and upd ports. Should I add both?
One recommendation I stumbled upon was to add the following lines to /etc/sysconfig/network:

Code:

YPSERV_ARGS="-p 834"
YPXFRD_ARGS="-p 835"

and add the following IPTables rules to enforce which network the server listens to for these ports:

Code:

iptables -A INPUT -p ALL -s! 192.168.0.0/24  --dport 834 -j DROP
iptables -A INPUT -p ALL -s! 192.168.0.0/24  --dport 835 -j DROP

Should I do that also?
Looking on this page http://wiki.linuxquestions.org/wiki/...f_port_numbers I understood that I will also need Internet access to TCP and UDP port 53 (DNS) and TCP port 443 (HTTPS) and Internal LAN Access to TCP and UDP port 53 (DNS), TCP and UDP port 123 (NTP), TCP and UDP port 2049 (NFS) and TCP and UDP port 6000 (X11). Is that correct?

blackhole54 · 07-28-2006, 02:47 PM

I am afraid I made the same mistake I made in post #7! (At least I am consistent!

)

Code:

#  --------------  INPUT Chain Firewall Rules ---------------
#
#  Accept all established connections and all loopback packets

$IPTABLES -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A INPUT -i lo -j ACCEPT

#  Rules which allow new Internet connections

for port in $EXT_TCP; do
   $IPTABLES -A INPUT -p TCP -s $EXTIF_IP --dport $port \
      -m state --state NEW -j ACCEPT
done

#  Rules which allow new LAN connections

for port in $INT_TCP; do
   $IPTABLES -A INPUT -p TCP -s $INTIF_NET --dport $port \
      -m state --state NEW -j ACCEPT
done

The highlighted line above should be replaced by:

Code:

    $IPTABLES -A INPUT -p TCP ! -s $INTIF_NET --dport $port \

I have read your last posts and will try to reply to them tonight (I think your local time is 4 hours ahead of mine).

As far a the
iptables -nvL and
iptables -t nat -nvL
commands go, I was providing them for your use as diagnostic tools. What you posted showed that there were indeed packets that matched the rule.

blackhole54 · 07-30-2006, 02:23 AM

heberrdacruz,

Your server is doing a lot more than I realized. As I alluded to in an early post, your network might have firewalls upstream, in which case your server might not need as stringent of a firewall as we have been talking about. Do you have a local network administrator you can talk to about this?

Quote:

Originally Posted by heberrdacruz

The command rpcinfo -p localhost shows lots of ports. Shall add all of them? Searching the internet I found a page saying that NFS uses random ports. Is that so?

Quote:

One recommendation I stumbled upon was to add the following lines to /etc/sysconfig/network:

Code:

YPSERV_ARGS="-p 834"
YPXFRD_ARGS="-p 835"

and add the following IPTables rules to enforce which network the server listens to for these ports:

Code:

iptables -A INPUT -p ALL -s! 192.168.0.0/24 --dport 834 -j DROP
iptables -A INPUT -p ALL -s! 192.168.0.0/24 --dport 835 -j DROP

Should I do that also?

I am learning about NFS and NIS myself. Aparrently the RPCs that NFS and NIS use listen by default on random ports and report these ports to the portmapper, which in turn is polled by machines wanting to use these services. The RH article I believe you are quoting above talks about nailing down the ports used by the ypxfrd and ypserv services. This article talks about about nailing down some other ports and has a link to yet another article. If you chose to use any of these methods, I would recommend checking the output of rpcinfo and making sure the ports you specified were actually used.

The iptables lines (above) in the RH article should not be necessary in the firewalls we are talking about because we are restricting access with a (default) policy of DROP anyway. If you do decide to use such rules, you need to remember that the order of rules matter. Check the iptables man pages for details, but roughly speaking, in any given chain "the first match wins." But there can be complications. You would also need to subsitute your LAN network address (INTIF_NET) for 192.168.0.0/24.

I assume you are not offering NFS or NIS over the Internet. (I believe that would be quite dangerous.) So even if you allow your firewall to accept all LAN connections, as I discuss below, but block all but certain specified Internet connections, you would still be accomplishing what the above rules do.

So the firewall does need to provide for the services listed with rpcinfo to your LAN. Your options are to open up everything to the LAN (as I discuss below), nail down all the RPC ports and enumerate them in the firewall script, or have the script determine the RPC ports each time it starts. I believe once started the RPC services won't change ports. So in the last case, the correct startup sequence would be

portmap
xinetd (for RPC services)
Firewall script

You could start an interim firewall prior to starting portmap that subsequently is overwitten by the new script. You just wouldn't have NFS or NIS (and possibly other things) until the final firewall script was run. In the last case, if any RPC services are restarted, the firewall script would need to be run again, just like restarting portmap require the rpc services to be restarted.

Here is an excercise that gives userful information for modifying my last script. Copy and paste the following into a terminal window.

Code:

rpc_tcp=$(rpcinfo -p localhost | \
   sed -n "s/^.*tcp[[:space:]]*\([[:digit:]]*\)[[:space:]].*$/\1/p" | \
   sort | uniq | tr "\n" " ")
rpc_udp=$(rpcinfo -p localhost | \
   sed -n "s/^.*udp[[:space:]]*\([[:digit:]]*\)[[:space:]].*$/\1/p" | \
   sort | uniq | tr "\n" " ")
echo "rcp_tcp = $rcp_tcp"
echo "rcp_udp = $rcp_udp"

(The sed commands are doing substitutions based on regular expressions. These are kind of like wildcards in the shell, but much more powerful. sort and uniq working together remove duplicate entries) I believe you should see these echo statements list the tcp and udp ports (respectively) that rpcinfo outputs listed in the format used by my last script (verify this). You can include the first two commands in the script and use what I show next to append them to existing strings. (My script didn't do anything with udp, but you can easily add it by mimicking what I did for tcp.)

If you want to add onto an existing string variable in a bash script (output highlighted):

Code:

[surfcity@Vectra surfcity]$ testme="aa bb cc"
[surfcity@Vectra surfcity]$ echo $testme
aa bb cc
[surfcity@Vectra surfcity]$ testme="$testme ee ff"
[surfcity@Vectra surfcity]$ echo $testme
 aa bb cc ee ff
[surfcity@Vectra surfcity]$ newvar="append this string"
[surfcity@Vectra surfcity]$ echo $newvar
 append this string
[surfcity@Vectra surfcity]$ testme="$testme $newvar"
[surfcity@Vectra surfcity]$ echo $testme
 aa bb cc dd ee ff append this string

Play with this sort of thing in a terminal window until you feel comfortable about how it works (assuming your shell is bash).

Another option is instead of enumerating all of the services for your LAN, you simply allow packets claiming to come from the LAN complete access to this machine. While potentially less secure, you can be certain you won't deny any essential services to machines on your LAN. To do this, replace

Code:

for port in $INT_TCP; do
   $IPTABLES -A INPUT -p TCP -s $INTIF_NET --dport $port \
      -m state --state NEW -j ACCEPT
done

with

Code:

$IPTABLES -A INPUT -p TCP -s $INTIF_NET \
      -m state --state NEW -j ACCEPT

This allows any packet claiming to come from the LAN to access anything.

Quote:

Looking on this page http://wiki.linuxquestions.org/wiki/...f_port_numbers I understood that I will also need Internet access to TCP and UDP port 53 (DNS) and TCP port 443 (HTTPS) and Internal LAN Access to TCP and UDP port 53 (DNS), TCP and UDP port 123 (NTP), TCP and UDP port 2049 (NFS) and TCP and UDP port 6000 (X11). Is that correct?

Realize that the firewall rules in the INPUT chains control incoming connections, not outgoing connections. (OK, technically it also controls what responses can come back, but we have a rule that ACCEPTs everything on established connections.) I doubt that you want external machines connecting to a local X11 server. If you want to run X sessions between machines, you should consider doing it with X11 forwarding in ssh. It is much more secure -- and encrypted!

The only reason you would need to allow incoming DNS connections is if you are running a DNS server or DNS proxy on this box. If this runs a DNS server for the rest of the Internet to poll, you certainly need to allow Internet access. If you are just running a proxy for your LAN you probably don't want to allow Internet access. In either case you would want to allow LAN access.

I am not an expert on these other services. You can find out what programs are actively listening on their respective TCP ports by running
netstat -nap as root. This will also show you UDP connections, but UDP is a stateless protocol so it won't tell you "LISTEN." But a process waiting for a connection will show a local address of 0.0.0.0. You can use this to see what services you might have forgotten about and their port numbers. (If netstat shows you a process number (pid) but no name, that process is completely swapped out. You can find out what it is using
ps <pid>). You then need to decide which of these service should be available to the Internet and/or your LAN.

heberrdacruz · 08-04-2006, 09:32 AM

Thank you very much for your reply. I am sorry I didn’t reply earlier. I was with flu and out of action for a few days, but I am fine now.
Yesterday afternoon I installed the much awaited new switch. I have linked to it only the cables from the external network interfaces and the university network cable. So far the network has been working without any problems. It really seems the mixture of traffic on the same switch was the cause of the problems we were having.
This morning I reintroduced

Code:

net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.log_martians = 1

Soon after, two of those odd messages appeared, but nothing else up to now (Fri Aug 4 11:33:10 BRT 2006).

Code:

Aug  4 09:41:20 newton kernel: IN=eth0 OUT=eth0 SRC=69.64.44.92 DST=192.168.30.11 LEN=1500 TOS=0x00 PREC=0x00 TTL=48 ID=52622 DF PROTO=TCP SPT=80 DPT=48321 WINDOW=778 RES=0x00 ACK URGP=0
Aug  4 09:41:22 newton kernel: IN=eth0 OUT=eth0 SRC=68.235.149.97 DST=192.168.30.118 LEN=316 TOS=0x00 PREC=0x00 TTL=109 ID=38958 DF PROTO=TCP SPT=443 DPT=2318 WINDOW=64271 RES=0x00 ACK PSH URGP=0

As eth0 is the external interface, this packet correctly came this ways. I don’t understand why it would go out through it as well.
I talked to the university network administrator and he told me that their firewall blocks all ports, except the standard ones, i.e., the ones that are known to be used the university computers.
In which way do you believe that blocking ports on the INTERNAL interface would really increase security? Our LAN computers have very limited access: professors and their research students and staff. The option of opening all ports in the LAN interface is surely much simpler.

Quote:

I assume you are not offering NFS or NIS over the Internet.

You assumed correctly.

Quote:

I doubt that you want external machines connecting to a local X11 server.

You again assumed correctly.

Quote:

If you want to run X sessions between machines, you should consider doing it with X11 forwarding in ssh.

What I do is ssh –X machine. Is that what you mean?
Thank you very much for all the information.

blackhole54 · 08-05-2006, 03:36 AM

Quote:

Originally Posted by heberrdacruz

Yesterday afternoon I installed the much awaited new switch. I have linked to it only the cables from the external network interfaces and the university network cable. So far the network has been working without any problems. It really seems the mixture of traffic on the same switch was the cause of the problems we were having.
This morning I reintroduced

Code:

net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.log_martians = 1

Soon after, two of those odd messages appeared, but nothing else up to now (Fri Aug 4 11:33:10 BRT 2006).

Code:

Aug  4 09:41:20 newton kernel: IN=eth0 OUT=eth0 SRC=69.64.44.92 DST=192.168.30.11 LEN=1500 TOS=0x00 PREC=0x00 TTL=48 ID=52622 DF PROTO=TCP SPT=80 DPT=48321 WINDOW=778 RES=0x00 ACK URGP=0
Aug  4 09:41:22 newton kernel: IN=eth0 OUT=eth0 SRC=68.235.149.97 DST=192.168.30.118 LEN=316 TOS=0x00 PREC=0x00 TTL=109 ID=38958 DF PROTO=TCP SPT=443 DPT=2318 WINDOW=64271 RES=0x00 ACK PSH URGP=0

As eth0 is the external interface, this packet correctly came this ways. I don?t understand why it would go out through it as well.

I am glad the new switch helped and things seem to be going well.

The interface a packet is sent out should be determined by the routing table. (Early on I had you check this table with route -n.) Earlier in this thread I assumed your routing table was static, i.e. that it didn't change after the computer had booted. I am now wondering if you are running a routing daemon such as routed, gated, or Zebra. I am not familiar with these, but my understanding is that they will (potentially) alter routing tables as they run. Maybe another poster can shed some more light on this mystery.

Quote:

I talked to the university network administrator and he told me that their firewall blocks all ports, except the standard ones, i.e., the ones that are known to be used the university computers.
In which way do you believe that blocking ports on the INTERNAL interface would really increase security? Our LAN computers have very limited access: professors and their research students and staff. The option of opening all ports in the LAN interface is surely much simpler.

First let me note I am not a security expert. When I first noted that your computer's firewall had no INPUT rules, I said that might not be a problem if you had a good firewall upstream. The script that win32sux posted followed the principle of "deny everything that is not allowed." This is the most secure firewall, where you enumerate every service you want people to have access to and block everything else. The other approach is to block things you don't want people to have access to and hope you haven't forgotten anything. It might be acceptable to you to allow LAN users access to evertything on the box. Where this might be less secure involves scenarios where one or more boxes have become compromised and you are trying to limit damages, or somebody on your LAN is trying to do something malicious. If all of the services your box is running (including ssh) should be accessable to everybody on the LAN, this restrictive approach might provide only marginal more security. It is a tradeoff of security on one hand and difficulty and possible problems on the other hand. You (perhaps in consultation with your network administator) have to make that call. (My first script porposal, in fact, allowed LAN users access to everything.)

When I first noted that your machine was wide open I didn't know how many services this computer provided, and was not aware of the complexity of your network. I feared this box might be exposed directly to the Internet. I originally just noted this as an aside that you might want to look at while we tried to solve your original complaint. Instead, it took on a life of its own and became a major focus on this thread.

If you do decide to allow LAN users access to everything, I would suggest you review all of the services on newton and turn of any you don't need. (Always a good security practice.) An easy way to see what services you are running is running netstat as root:

Code:

netstat -nap | less

Quote:

What I do is ssh -X machine. Is that what you mean?

Yes, the -X enables X11 forwarding (if the server you are connecting to allows it). Note that you can also enable this in your client config file (~/.ssh/config or, for system defaults, /etc/ssh/ssh_config) such that you wouldn't need the -X on the command line.

heberrdacruz · 08-07-2006, 03:32 PM

Thank you for your reply.

Quote:

Originally Posted by blackhole54

Earlier in this thread I assumed your routing table was static

You assumed correctly. This is our simple routing. I don’t know where the 169 ... comes from.

Code:

[root@newton ~]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
200.17.xxx.xxx  0.0.0.0         255.255.255.240 U     0      0        0 eth0
192.168.30.0    0.0.0.0         255.255.255.0   U     0      0        0 eth1
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth1
0.0.0.0         200.17.xxx.yyy  0.0.0.0         UG    0      0        0 eth0
[root@newton ~]#

I have talked to the university network administrator and we agreed that it would be not be a significant security risk to open all ports to the LAN.

Quote:

If you do decide to allow LAN users access to everything, I would suggest you review all of the services on newton and turn of any you don't need.

I will recheck to see if anything was left out.

Quote:

Yes, the -X enables X11 forwarding (if the server you are connecting to allows it). Note that you can also enable this in your client config file (~/.ssh/config or, for system defaults, /etc/ssh/ssh_config) such that you wouldn't need the -X on the command line.

Thank you for the tip.
When customizing your firewall I had a doubt. Which one of the 3 lines below shall I use in the new and improved switch situation?

Code:

   $IPTABLES -A INPUT -p TCP ! -s $INTIF_NET --dport $port \
   $IPTABLES -A INPUT -p TCP -s $EXTIF_IP --dport $port \
   $IPTABLES -A INPUT -p TCP -i $EXTIF -s $EXTIF_IP --dport $port \

blackhole54 · 08-08-2006, 01:01 AM

Quote:

Originally Posted by heberrdacruz

When customizing your firewall I had a doubt. Which one of the 3 lines below shall I use in the new and improved switch situation?

Code:

   $IPTABLES -A INPUT -p TCP ! -s $INTIF_NET --dport $port \
   $IPTABLES -A INPUT -p TCP -s $EXTIF_IP --dport $port \
   $IPTABLES -A INPUT -p TCP -i $EXTIF -s $EXTIF_IP --dport $port \

I believe this is the line that I had so much trouble getting right in the past, so your confusion is understandable! The first line was what I was suggesting when things were not coming in on the correct ports. Since you got that straightened out, what I would suggest now is below. I am including the lines around it for context and to make sure we are not miscommunicating.

Code:

#  Rules which allow new Internet connections

for port in $EXT_TCP; do
   $IPTABLES -A INPUT -p TCP -i $EXTIF --dport $port \
      -m state --state NEW -j ACCEPT
done

heberrdacruz · 08-08-2006, 08:21 AM

Thank you very much for the rules which allow new Internet connections.
Please check below the script I used.

Code:

#!/bin/sh

#  Firewall script

#  Define interfaces and IP addresses for Internet and LAN

EXTIF="eth0"
EXTIF_IP="200.17.xxx.xxx"

INTIF="eth1"
INTIF_NET="192.168.30.0/24"

#  Define the TCP ports the should be accessible from Internet and LAN
#  These should be space separated, quoted strings.

#  Internet Access:
#      TCP UDP protocol
#      80      http
#     443      https
#      25      smtp  (mail)
#      53  53  DNS 

EXT_TCP="80 443 25 53"
EXT_UDP="53"

#  Internal LAN Access: all ports open


#  Command definitions:

IPTABLES="/sbin/iptables"
MODPROBE="/sbin/modprobe"

#  Set policies for each chain

$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -P FORWARD DROP
$IPTABLES -t nat -P PREROUTING ACCEPT
$IPTABLES -t nat -P POSTROUTING ACCEPT
$IPTABLES -t nat -P OUTPUT ACCEPT
$IPTABLES -t mangle -P PREROUTING ACCEPT
$IPTABLES -t mangle -P OUTPUT ACCEPT

#  Flush all standard chains and delete all user chains

$IPTABLES -F
$IPTABLES -F -t nat
$IPTABLES -F -t mangle

$IPTABLES -X
$IPTABLES -X -t nat
$IPTABLES -X -t mangle

#  Load modules for iptables

$MODPROBE ip_conntrack_ftp
$MODPROBE ip_nat_ftp
$MODPROBE ip_conntrack_irc
$MODPROBE ip_nat_irc

#  --------------  INPUT Chain Firewall Rules ---------------
#
#  Accept all established connections and all loopback packets

$IPTABLES -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A INPUT -i lo -j ACCEPT

#  Rules which allow new Internet connections

for port in $EXT_TCP; do
   $IPTABLES -A INPUT -p TCP -i $EXTIF --dport $port \
      -m state --state NEW -j ACCEPT
done


for port in $EXT_UDP; do
   $IPTABLES -A INPUT -p UDP -i $EXTIF --dport $port \
      -m state --state NEW -j ACCEPT
done

#  Rules which allow new LAN connections

$IPTABLES -A INPUT -p TCP -s $INTIF_NET \
      -m state --state NEW -j ACCEPT

#  Policy drops everything else.  But first log them.

$IPTABLES -A INPUT -j LOG --log-prefix "INPUT DROP: "

#  --------------  FORWARD Chain Firewall Rules ---------------
#
#  Accept establish connections.
#  Accept new connections from $LAN that are destined for the Internet
#  Log any packets that will be dropped

$IPTABLES -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
$IPTABLES -A FORWARD -o $EXTIF -s $INTIF_NET \
   -m state --state NEW -j ACCEPT
$IPTABLES -A FORWARD -j LOG --log-prefix "FORWARD DROP: "


#  --------------  POSTROUTING Chain Firewall Rules ---------------
#
#  SNAT all LAN originated packets destined for the Internet.


$IPTABLES -t nat -A POSTROUTING -o $EXTIF -s $INTIF_NET \
   -j SNAT --to $EXTIF_IP

After running it, I am afraid I had some problems. I detected that a LAN computer could no longer get an address resolved:

Code:

C:\Documents and Settings\heber>ping www.xx.yyyy.zz
Ping request could not find host www.xx.yyyy.zz. Please check the name and try again.

From newton’s log:

Code:

Aug  8 09:51:44 newton kernel: INPUT DROP: IN=eth1 OUT= MAC=00:01:01:c7:28:68:00:xx:xx:xx:xx:xx:xx:xx SRC=192.168.30.118 DST=192.168.30.15 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=65036 PROTO=UDP SPT=1037 DPT=53 LEN=40
Aug  8 09:51:56 newton kernel: INPUT DROP: IN=eth1 OUT= MAC=00:01:01:c7:28:68:00:xx:xx:xx:xx:xx:xx:xx SRC=192.168.30.118 DST=192.168.30.15 LEN=40 TOS=0x00 PREC=0x00 TTL=128 ID=65060 PROTO=TCP SPT=2384 DPT=8080 WINDOW=0 RES=0x00 RST URGP=0
Aug  8 09:51:58 newton kernel: INPUT DROP: IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:00:xx:xx:xx:xx:xx:xx:xx SRC=192.168.30.118 DST=192.168.30.255 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=65064 PROTO=UDP SPT=137 DPT=137 LEN=58

newton no longer answered to pings:

Code:

C:\Documents and Settings\heber>ping 192.168.30.15

Pinging 192.168.30.15 with 32 bytes of data:

Request timed out.
Request timed out.
Request timed out.
Request timed out.

Ping statistics for 192.168.30.15:
    Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),

From newton’s log:

Code:

Aug  8 09:56:25 newton kernel: INPUT DROP: IN=eth1 OUT= MAC=00:01:01:c7:28:68:00:xx:xx:xx:xx:xx:xx:xx SRC=192.168.30.118 DST=192.168.30.15 LEN=62 TOS=0x00 PREC=0x00 TTL=128 ID=1451 PROTO=UDP SPT=1037 DPT=53 LEN=42
Aug  8 09:58:26 newton kernel: INPUT DROP: IN=eth1 OUT= MAC=00:01:01:c7:28:68:00:xx:xx:xx:xx:xx:xx:xx SRC=192.168.30.118 DST=192.168.30.15 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=2034 PROTO=ICMP TYPE=8 CODE=0 ID=512 SEQ=3584
Aug  8 09:58:31 newton kernel: INPUT DROP: IN=eth1 OUT= MAC=00:01:01:c7:28:68:00:xx:xx:xx:xx:xx:xx:xx SRC=192.168.30.118 DST=192.168.30.15 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=2035 PROTO=ICMP TYPE=8 CODE=0 ID=512 SEQ=3840
Aug  8 09:58:37 newton kernel: INPUT DROP: IN=eth1 OUT= MAC=00:01:01:c7:28:68:xx:xx:xx:xx:xx:xx:xx SRC=192.168.30.118 DST=192.168.30.15 LEN=60 TOS=0x00 PREC=0x00 TTL=128 ID=2164 PROTO=ICMP TYPE=8 CODE=0 ID=512 SEQ=4096

In fact, there were many INPUT DROP messages in the log from eth1, when I expected none.
What did I do wrong?