Ok guys, here's the real problem for your pleasure.
I had an old server (also a router) and decided to upgrade it the easy way - put an old hard drive to a new machine. Guess what? It didn't work. Kudzu found some new hardware and configured it. The new machine has an on-board network interface. So I got the PCI network card from the old one - the machine is a router, remember? The old one got eth1 pointed outside (dynamic IP via dhcp) while eth0 pointed to a LAN (static - 192.168.1.1). Here's some old configs:

/backup/etc/sysconfig/network-scripts/ifcfg-eth0
# D-Link System Inc Gigabit Ethernet Adapter
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.1.1
NETMASK=255.255.255.0

/backup/etc/sysconfig/network-scripts/ifcfg-eth1
# Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+
DEVICE=eth1
ONBOOT=yes
BOOTPROTO=dhcp
HWADDR=00:50:BA:15:04:5B

/backup/etc/sysconfig/network
NETWORKING=yes
HOSTNAME=something.local

/backup/etc/sysconfig/iptables
# Generated by iptables-save v1.2.11 on Tue Jul 15 09:50:35 2008
*nat
:PREROUTING ACCEPT [2066:139463]
:POSTROUTING ACCEPT [88:5302]
:OUTPUT ACCEPT [712:73423]
-A PREROUTING -i eth0 -p tcp -m tcp --dport 9988 -j DNAT --to-destination 192.168.1.100:9988
-A POSTROUTING -o eth1 -j MASQUERADE
COMMIT
# Completed on Tue Jul 15 09:50:35 2008
# Generated by iptables-save v1.2.11 on Tue Jul 15 09:50:35 2008
*filter
:INPUT ACCEPT [46446:10311121]
:FORWARD ACCEPT [117816:137721556]
:OUTPUT ACCEPT [51627:10042633]
-A INPUT -i eth0 -p tcp -m state --state NEW -m tcp --dport 9988 -j ACCEPT
-A FORWARD -i eth0 -j ACCEPT
COMMIT
# Completed on Tue Jul 15 09:50:35 2008
# Generated by iptables-save v1.2.11 on Tue Jul 15 09:50:35 2008
*mangle
:PREROUTING ACCEPT [228178:153760036]
:INPUT ACCEPT [46446:10311121]
:FORWARD ACCEPT [181589:143441061]
:OUTPUT ACCEPT [51627:10042633]
:POSTROUTING ACCEPT [233216:153483694]
COMMIT
# Completed on Tue Jul 15 09:50:35 2008

Once I swapped the hard drive to a new machine, the only somehow working interface is eth0 (PCI card, for some reason). eth1 located on a motherboard should be misconfigured (say, BOOTPROTO=dhcp while it should be static with an IPADDR=192.168.1.1) so the box is on the internet. If I try to properly assign (static) settings to the eth1 and do `service network restart`, there's no network at all. And the real problem is that this box is 2000 miles away. I have to ask someone on the other end to edit /etc/sysconfig/network-scripts/ifcfg-eth1 to change BOOTPROTO to dhcp and reboot the box to bring it back to life.

The new config files are:

cat /etc/sysconfig/network-scripts/ifcfg-eth0
# Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+
DEVICE=eth0
ONBOOT=yes
BOOTPROTO=dhcp

/etc/sysconfig/network-scripts/ifcfg-eth1
# Intel Corporation 82562EZ 10/100 Ethernet Controller
DEVICE=eth1
ONBOOT=yes
BOOTPROTO=dhcp
#IPADDR=192.168.1.1
#NETMASK=255.255.255.0
#GATEWAY=192.168.1.254
HWADDR=00:0C:6E:A7:04:05

/etc/sysconfig/iptables
the same as above except I've swapped eth0 and eth1

/etc/modprobe.conf
alias usb-controller uhci-hcd
alias eth0 8139too
#alias eth1 sk98lin
alias usb-controller1 ehci-hcd
alias scsi_hostadapter ata_piix
alias eth1 e100
alias snd-card-0 snd-intel8x0
options snd-card-0 index=0
install snd-intel8x0 /sbin/modprobe --ignore-install snd-intel8x0 && /usr/sbin/alsactl restore >/dev/null 2>&1 || :
remove snd-intel8x0 { /usr/sbin/alsactl store >/dev/null 2>&1 || : ; }; /sbin/modprobe -r --ignore-remove snd-intel8x0

A snapshot from /var/log/messages
ifup: Determining IP information for eth0...
kernel: eth0: link up, 100Mbps, full-duplex, lpa 0x41E1
dhclient: DHCPREQUEST on eth0 to 255.255.255.255 port 67
dhcpd: DHCPREQUEST for XXX.XXX.XXX.XXX from 00:50:ba:15:04:5b via eth0: wrong network.
dhcpd: DHCPNAK on XXX.XXX.XXX.XXX to 00:50:ba:15:04:5b via eth0
dhclient: DHCPNAK from XXX.XXX.XXX.XXX
dhclient: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 7
dhcpd: DHCPDISCOVER from 00:50:ba:15:04:5b via eth0
dhcpd: icmp_echorequest 192.168.1.124: Network is unreachable
dhclient: DHCPOFFER from 10.8.0.1
dhclient: DHCPREQUEST on eth0 to 255.255.255.255 port 67
dhcpd: DHCPREQUEST for XXX.XXX.XXX.XXX (10.8.0.1) from 00:50:ba:15:04:5b via eth0: wrong network.
dhcpd: DHCPNAK on XXX.XXX.XXX.XXX to 00:50:ba:15:04:5b via eth0
dhclient: DHCPNAK from XXX.XXX.XXX.XXX
dhclient: DHCPACK from 10.8.0.1
NET: /sbin/dhclient-script : updated /etc/resolv.conf
dhclient: bound to XXX.XXX.XXX.XXX -- renewal in 23319 seconds.
ifup: done.
network: Bringing up interface eth0: succeeded
kernel: ADDRCONF(NETDEV_UP): eth1: link is not ready
kernel: e100: eth1: e100_watchdog: link up, 100Mbps, full-duplex
kernel: ADDRCONF(NETDEV_CHANGE): eth1: link becomes ready
dhcpd: DHCPOFFER on 192.168.1.124 to 00:50:ba:15:04:5b via eth0
network: Bringing up interface eth1: succeeded
dhcpd: receive_packet failed on eth0: Network is down
network: Shutting down interface eth0: succeeded
network: Shutting down interface eth1: succeeded
network: Shutting down loopback interface: succeeded
sysctl: net.ipv4.ip_forward = 0
network: Disabling IPv4 packet forwarding: succeeded
sysctl: net.ipv4.ip_forward = 1
sysctl: net.ipv4.conf.default.rp_filter = 0
sysctl: net.ipv4.conf.default.accept_source_route = 0
sysctl: kernel.sysrq = 0
sysctl: kernel.core_uses_pid = 1
network: Setting network parameters: succeeded
network: Bringing up loopback interface: succeeded

where the XXX.XXX.XXX.XXX - a valid outcide ip address.


netstat -rn
Kernel IP routing table
Destination Gateway Genmask Flags MSS Window irtt Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
xx.xxx.xxx.0 0.0.0.0 255.255.240.0 U 0 0 0 eth0
xxx.xxx.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 76.187.240.1 0.0.0.0 UG 0 0 0 eth0

When I try to connect an internet cable to the onboard interface (eth1), it simply doesn't work.

If it will be really hard to fix, tell me how to start from scratch with networking. I.e. How to gracefully wipe all the settings and configuration in order to start all over again.

Any help is greatly appreciated.

Help me!!! :-)