configure 2 NICs on linux firewall
 

Maybe this is too much detail, but... I have a problem that I have spent weeks trying to resolve...

I had a perfectly running network using CentOS3, but I fresh installed CentOS5 to get USB support...now I can not get my network working the
way I had it working before!

I have my linux box connected to the the FIOS modem via eth1 and my internal network of PCs connected to the linux box via eth0.

From the PCs, I can not get to the internet. It seems that internal packets are not getting routed to eth1 on the box to the modem.

Obviously,I have a routing problem, but I am too 'linux new' to figure it out and need someones help. And, after days of searching, I can't seem to find a good example of my setup. This should be a common setup and should be in a wiki...

Here is the setup:

modem is 10.1.3.2
eth1 is 10.1.3.1
eth0 is 10.1.2.1

[root@ooorahhome etc]# lspci
00:00.0 Host bridge: Intel Corporation 82845 845 (Brookdale) Chipset Host Bridge (rev 11)
00:01.0 PCI bridge: Intel Corporation 82845 845 (Brookdale) Chipset AGP Bridge (rev 11)
00:1d.0 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #1 (rev 02)
00:1d.1 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #2 (rev 02)
00:1d.2 USB Controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) USB UHCI Controller #3 (rev 02)
00:1d.7 USB Controller: Intel Corporation 82801DB/DBM (ICH4/ICH4-M) USB2 EHCI Controller (rev 02)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 82)
00:1f.0 ISA bridge: Intel Corporation 82801DB/DBL (ICH4/ICH4-L) LPC Interface Bridge (rev 02)
00:1f.1 IDE interface: Intel Corporation 82801DB (ICH4) IDE Controller (rev 02)
00:1f.3 SMBus: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) SMBus Controller (rev 02)
00:1f.5 Multimedia audio controller: Intel Corporation 82801DB/DBL/DBM (ICH4/ICH4-L/ICH4-M) AC'97 Audio Controller (rev 02)
02:00.0 Ethernet controller: ADMtek NC100 Network Everywhere Fast Ethernet 10/100 (rev 11)
02:05.0 VGA compatible controller: ATI Technologies Inc Rage XL (rev 27)
02:0a.0 Ethernet controller: Intel Corporation 82557/8/9 [Ethernet Pro 100] (rev 0d)
02:0c.0 RAID bus controller: Promise Technology, Inc. PDC20276 (MBFastTrak133 Lite) (rev 01)

[root@ooorahhome etc]# mii-tool
eth0: negotiated 100baseTx-FD flow-control, link ok
eth1: negotiated 100baseTx-FD flow-control, link ok

[root@ooorahhome Desktop]# cat /etc/sysconfig/networking/devices/ifcfg-eth0
# Intel Corporation 82557/8/9 [Ethernet Pro 100]
DEVICE=eth0
BROADCAST=10.1.2.255
HWADDR=00:0D:61:35:53:BF
IPADDR=10.1.2.1
NETMASK=255.255.255.0
NETWORK=10.1.2.0
ONBOOT=yes
BOOTPROTO=none
GATEWAY=10.1.3.2
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
PEERDNS=yes
[root@ooorahhome Desktop]# cat /etc/sysconfig/networking/devices/ifcfg-eth1
# ADMtek NC100 Network Everywhere Fast Ethernet 10/100
DEVICE=eth1
BROADCAST=10.1.3.255
HWADDR=FE:FF:FF:FF:FF:FF
IPADDR=10.1.3.1
NETMASK=255.255.255.0
NETWORK=10.1.3.0
ONBOOT=yes
BOOTPROTO=none
GATEWAY=10.1.3.2
TYPE=Ethernet
USERCTL=no
IPV6INIT=no
PEERDNS=yes

[root@ooorahhome Desktop]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:0D:61:35:53:BF
inet addr:10.1.2.1 Bcast:10.1.2.255 Mask:255.255.255.0
inet6 addr: fe80::20d:61ff:fe35:53bf/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:165691 errors:0 dropped:0 overruns:0 frame:0
TX packets:14457 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:47963547 (45.7 MiB) TX bytes:1507408 (1.4 MiB)

eth1 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
inet addr:10.1.3.1 Bcast:10.1.3.255 Mask:255.255.255.0
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:192379 errors:0 dropped:0 overruns:0 frame:0
TX packets:32679 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:78241013 (74.6 MiB) TX bytes:5534915 (5.2 MiB)
Interrupt:169 Base address:0x2000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:5904 errors:0 dropped:0 overruns:0 frame:0
TX packets:5904 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:724193 (707.2 KiB) TX bytes:724193 (707.2 KiB)

[root@ooorahhome Desktop]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.1.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
0.0.0.0 10.1.3.2 0.0.0.0 UG 0 0 0 eth1

Also, why does 169.254.0.0 get auto created whenever I restart the network?

[root@ooorahhome etc]# iptables -L -v
Chain INPUT (policy DROP 24910 packets, 22M bytes)
pkts bytes target prot opt in out source destination
2547 315K ACCEPT all -- lo any anywhere anywhere
26018 22M ACCEPT all -- eth0 any anywhere anywhere
9534 6222K ACCEPT all -- any any anywhere anywhere state RELATED,ESTABLISHED

Chain FORWARD (policy DROP 7413 packets, 597K bytes)
pkts bytes target prot opt in out source destination
1277 90704 ACCEPT all -- eth0 any anywhere anywhere

Chain OUTPUT (policy ACCEPT 15089 packets, 2460K bytes)
pkts bytes target prot opt in out source destination
[root@ooorahhome etc]# iptables -t nat -L -v
Chain PREROUTING (policy ACCEPT 128K packets, 7474K bytes)
pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 560 packets, 91563 bytes)
pkts bytes target prot opt in out source destination
2752 240K MASQUERADE all -- any eth1 anywhere anywhere

Chain OUTPUT (policy ACCEPT 2883 packets, 302K bytes)
pkts bytes target prot opt in out source destination

Again, this seems to be a relatively easy setup. Why is it not working?

Tom
Please make my day!

See if IP forwarding is the problem, try:

been there done...
 

Thank you for the reply. I have done that command several times and restarted the network, as well...

Could there be something wrong with iptables?

Tom

Sorry, that the suggestion didn't help. I have enough items here that numbering them may help. Feel free to save space & typing by just referring to the number.

1. Did you overwrite the CentOS 3 installation when you did the fresh install of 5?
   
   
2. "Obviously" :) someone changed something between 3 & 5,
   1. Do you have access to the changelog(s) to try to figure out if there was a change in the default set-up?
   2. Has CentOS followed Fedora's lead & turned on SELinux by default?
   
   
3. I doubt it matters, but what is a "FIOS modem"?
   
   
4. Quote:

   From the PCs, I can not get to the internet.

   -- I assume that the CentOS box can ping in either direction & surf if you wanted it to.
   
   
5. Does this fairly represent your set-up?:
   Code:

   ---

       (    )
  (  'Net  )
    (    )
      |
      | ISP assigned
+--------------+
| "FIOS modem" |
+--------------+
      | 10.1.3.2
      |
  eth1 | 10.1.3.1
+--------------+
|              |
|  CentOS 5  |
|              |
+--------------+
  eth0 | 10.1.2.1
      |
 +-+-+-+-+-+-+
 | | | | | | |
    PC's

   ---

6. I don't have /etc/sysconfig/networking/devices/ifcfg-ethn files (SimplyMEPIS 6.0), what do they do? Are they merely descriptive or do they prescribe the configuration of the interfaces?
   
   
7. Please edit your post to put the output of route -n in "Code:" blocks -- I can't read it in its current format.
   
   
8. I see no "MASQUERADE" lines in your iptables output,
   1. How were the rules set up? (Shorewall?, GuardDog?, etc.?)
   2. Have you tried, do you dare to, temporarily disabling them?
   3. What does iptables -t nat -L show?

1. Yes. I I reformatted the discs and installed CentOS5

2a. If the change log exists on my system, then I have access to it.

2b. Not positive, but I believe they follow the upstream very closely. So, I would guess yes, but don't know how to determine.

3. Sorry...FIOS is the Verizon fiber feeding high speed access to my home.

4. Yes.

5. Exactly.

6. I found the command online during my search for a solution. I do believe the files list the configuration.

7. Not sure what you mean, "Code:" blocks. I tried to edit it below...

[root@ooorahhome Desktop]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
10.1.3.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth1
0.0.0.0 10.1.3.2 0.0.0.0 UG 0 0 0 eth1

8.
Chain POSTROUTING (policy ACCEPT 560 packets, 91563 bytes)
pkts bytes target prot opt in out source destination
2752 240K MASQUERADE all -- any eth1 anywhere anywhere

8a. I followed a HowTo Wiki off the CentOS website.
8b. The original installation had a more complicated table of rules. Not sure if they were more or less restrictive, but I still had the problem then. I could turn them off for debugging.
8c. That command lists the rules for handling NAT.

Thank you so much for the help.

2. A Google Linux on:

centos selinux

had http://www.linux.com/articles/44663 as the 3rd hit, para. 4: Another Google Linux on:

"turn off" selinux

found: http://www.cyberciti.biz/faq/how-to-...-linux/print/:
Try that & see if it fixes your problem.


Edit:
http://www.linuxquestions.org/questi...or-not-593765/ shows: as the way to find out the status of SELinux.

I added 'selinux=0' to the end of grub.conf and rebooted.

[root@ooorahhome Desktop]# sestatus -v
SELinux status: disabled

I liked the idea, but I still cannot access the internet from the PCs.

I also decided to turn off ipv6:
1 - append "alias net-pf-10 off" to the end of /etc/modprobe.conf
2 - set IPV6INIT=no in /etc/sysconfig/network-scripts/ifcfg-eth0 and /etc/sysconfig/network-scripts/ifcfg-eth1
3 - comment out "::1 localhost6.localdomain6 localhost6" in /etc/hosts
4 - reboot

Thanks for trying. Any other ideas?

I did discover that when I type the following command I cannot ping anything from the PC network...
"echo 1 > /proc/sys/net/ipv4/ip_forward"

I have to restart the network to get back to pinging the CentOS box and not getting the modem...

Odd.

hi,

try delete that bold part in ifcfg-eth0 (you dont need that gateway - that is only for eth1) - and restart network.
your iptables seems ok to me.
169.254 subnet - taking the advantage of MS reserved ip block for networking.

HTH.

Thanks for the idea. This action did not solve my problem...

Tom

Let's go back to your OP:
That is, you did a fresh install of CentOS 5 on the same partition, overwriting the old CentOS 3.

I have have posted various clarification Q's, you have dutifully & uncomplaining answered. I have suggested fixes, they haven't worked; others have suggested other fixes, they haven't worked either.

I'm reasonably sure the cause of the problem is in my Q#2 above. Either a change introduced by CentOS, or by your answers to the installer. If you want track this down, the changelogs I spoke of should be on the CentOS web site.

I have new Q's:

What is this box used for?
-- Particularly, is it a dedicated firewall?
Why is USB support important?

I think your options are:

1. Wait for more suggestions here.
2. Post on a CentOS forum.
3. Re-install CentOS 5 very carefully, looking for options whose answers may have caused the problem.
4. Re-install CentOS 3, trying to get back to where you were.
5. If this is a dedicated firewall & USB is that important, consider IPCop instead. (IPCop 1.4.16 definitely supports USB for saving configuration data.)

If you try C. or D. (or E., for that matter) do it w/o trashing the current install, if at all possible. Use a new partition, a new drive, a different box.


Firewall (FW)/Production Box Notes
For the last 2 or 3 years I have been running SmoothWall Express 2 (SWE 2) as the perimeter FW for my home LAN. It has worked well, but SWE 3 is out & it's time for a change. I'm going about it very conservatively. I have considered my options & my new FW will be either SWE 3 or IPCop. I'm putting SWE 3 on a new FW box & testing it, when I'm happy w/ it I'll swap it into my LAN. Then I'll install IPCop on the old FW box & test it. If I like it better, I'll swap again. After I'm convinced I've made the right choice, I will put the same distro on each box, put the new one into production, & hold the old one as a spare. This isn't just some "What I would do . . .", but what I am actually doing. Your circumstances may be different & YMMV. However, even if your CentOS box is not a dedicated perimeter FW, it looks like a production box & there may be, I hope gentle, lessons here.

Thank you for your words of wisdom and help!

When I installed OS5 I reformatted the disk into one partition unlike my install of OS3. It was just easier.

There was something new during the install. It asked me about the level of security. I chose the default. I think that answer turned on SElinux and set up the default iptables, but I am not sure. Besides, I rewrote the iptables rules and you helped me turn off SElinux.

I use the box as both a firewall and file server for my home network. It has hardware raid and I like the idea of having backed up data...

USB: the (noisy) box is located in a semi-convenient place far from where I wanted the monitor (telephone desk). I run two USB extenders over Cat5 for the keyboard and mouse. OS3 does not have USB support - it barely had mouse wheel support.

I'll search for a solution for a little while longer. I did post on CentOS forum with less response than here. I won't reinstall OS5 - it was painful (newbie) - or OS3 (USB), but thanks for the suggestions. I may go that route instead.

Again, thanks for the support.

Tom

One suggestion on the CentOS forum was to add a rule to the iptable to enable eth1. However, I believe the suggestion would make my firewall useless.

They wrote:
"Quote:
dasto wrote:
At this point could it be my firewall setup?

26018 22M ACCEPT all -- eth0 any anywhere anywhere

Tom

Well yes , you enable only eth0 , what about eth1 ?????"


In my case, if I added " iptables -A INPUT -i eth1 -j ACCEPT", then my firewall would be useless...

Isn't that right?

Tom

Well, that last suggestion got me thinking. I had set the FORWARD chain default to DROP. Once I set it to ACCEPT, it worked. I was able to get internet with the PC network...but now I don't know if my firewall is actually useful.

Any thoughts?

Saving firewall rules to /etc/sysconfig/iptables: [ OK ]
Chain INPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- lo any anywhere anywhere
0 0 ACCEPT all -- eth0 any anywhere anywhere
0 0 ACCEPT all -- any any anywhere anywhere state RELATED,ESTABLISHED

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 ACCEPT all -- eth0 any anywhere anywhere

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain PREROUTING (policy ACCEPT 57648 packets, 3403K bytes)
pkts bytes target prot opt in out source destination

Chain POSTROUTING (policy ACCEPT 139 packets, 13978 bytes)
pkts bytes target prot opt in out source destination
0 0 MASQUERADE all -- any eth1 anywhere anywhere

Chain OUTPUT (policy ACCEPT 1690 packets, 119K bytes)
pkts bytes target prot opt in out source destination


Tom

please help me out with this