Server unable to see outside lan

toes · 12-23-2006, 05:10 AM

I've been working all night on getting our small server set back up after an attack disabled it.

All seems to be well except for networking not working correctly. The server hosts dns, web, and email, but it has decided that it can't see anything outside the local network. I can successfully ping both hostname and ip address of the backup dns server which is inside the LAN, and can also ping the ip address of a fileserver inside the LAN.

However, I can't ping ip or hostname outside. The machine lists itself as primary dns server and it partner as secondary. named seems to be running fine, and there are no errors in the named.log file.

Server has run with the same dns config for several months, and has served dns for a few years. Bind 9.2.1. RH9 installation.

I'm certain that this must be a simple configuration problem, but I'm not finding. My bet is that it's a permissions/ownership issue with something restored from backup.

I'm a bit punchy after an all-nighter, so forgive me if I've not provided enough information. Please let me know what else you need.

fordeck · 12-23-2006, 08:01 AM

Have you checked to see if it is a routing issue? If your default route is missing this can have symptoms similar to yours.
Could you post the results from this command:

Code:

 /sbin/route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
169.254.0.0     0.0.0.0         255.255.0.0     U     0      0        0 eth0
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0

The bold line above is an example of a default route. If it is missing you can add it as follows:

Code:

route add default gw <ip-of-gateway-here>

Regards,
Fordeck

toes · 12-23-2006, 08:43 AM

Thanks for the reply! I'm about to head back in after a nice 3 hour nap here at home and will give this a shot. I didn't (couldn't?) think of it last night, but I had that exact problem, a broken route table, on my home system several months ago. I wish I'd have remembered that earlier. I'll post back with results.

toes · 12-23-2006, 09:16 AM

Ok, I've checked the route table and it lines up with what you've posted. Here's what I have:

Code:

66.76.117.0    0.0.0.0        255.255.255.0    U     0    0    0    eth0
169.254.0.0    0.0.0.0        255.255.0.0      U     0    0    0    eth0
127.0.0.0      0.0.0.0        255.0.0.0        U     0    0    0    lo
0.0.0.0        66.76.117.1    0.0.0.0          UG    0    0    0    eth0

The gateway is the same as listed on the backup dns server, so it seem to be right.

I forgot to mention in the first post that all 3 servers are on static addresses. We have the range 66.76.117.24 - 66.76.117.26. The machine on .26 is the problem.

Before I left home to come back up here, I tested ping externally to that server and got no response to either IP or hostname. But I can ping both IP and hostname of the other two addresses, so at least that excludes something broken on the rest of the network and isolates the problem to that server. That was expected, but good to confirm, anyway.

Any follow-up suggestions? Thanks for the help so far!

fordeck · 12-23-2006, 09:31 AM

Quote:

66.76.117.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
127.0.0.0 0.0.0.0 255.0.0.0 U 0 0 0 lo
0.0.0.0 66.76.117.1 0.0.0.0 U 0 0 0 eth0

The route from above:

Quote:

0.0.0.0 66.76.117.1 0.0.0.0 U 0 0 0 eth0

It should have a "G" ( for gateway) next to the "U" in the Flags column if it is to be used as the default gateway.

Have you tried deleting that route and then adding back in with the following command:

Code:

route add default gw 66.76.117.1

toes · 12-23-2006, 09:42 AM

I'm sorry. That was a copy error. It does have the G. Original edited to show this.

I just tried removing and re-adding the default gateway, but the problem still exists.

Something else I experimented with, too, is changing the primary nameserver address to that of my rather than this .26 machine. Making the change on the .25 machine allows proper name resolution. However, changing it on the problem .26 machine still makes no difference. I still can't ping hostnames or IPs.

fordeck · 12-23-2006, 09:57 AM

Is it possible that a firewall is restricting these ports and icmp?

toes · 12-23-2006, 10:05 AM

iptables has no policies. Turning it off doesn't help. There is no physical firewall. I've also tried adding in generic policies to iptables with the gui tool, but that has no effect either.

Thanks for the suggestions. I'm hoping to come across something soon so I can spend some Christmas weekend at home with my wife and daughters!

fordeck · 12-23-2006, 10:08 AM

What is the output from:

Code:

ifconfig eth0

toes · 12-23-2006, 10:17 AM

Hopefully I don't make any typos transferring this...

Code:

eth0   Link encap:Ethernet   HWaddr 00:50:2C:09:00:E7
inet addr:66.76.117.26   Bcast 66.76.117.255   Mask:255.255.255.0
UP  BROADCAST  RUNNING  MULTICAST  MTU:1500  Metric:1
RX packets:848135 errors:0 dropped:0 overruns:0 frame:0
TX packets: 5433 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes: 51007842 (48.6 Mb)  TX Bytes:316325 (308.9kb)
Interrupt:5  Base address:0xe400

fordeck · 12-23-2006, 10:32 AM

So you have the whole public subnet 66.76.117.0-66.76.117.255?

If so then your 24 bit (255.255.255.0) subnet mask is OK, otherwise you would need to have a different subnet mask. For example if you had been given the address space 66.76.117.0-66.76.117.31 then you would be using a 27 bit subnet mask or 255.255.255.224 which for example would yield 30 usable address's.

Just trying to make sure I understand your addressing?

toes · 12-23-2006, 10:46 AM

No, sorry. We only have the three addresses 66.76.117.24, .25, and .26, the last one being the machine with the problem.

I had wondered about the netmask as well, but it's the same as was there before when all was working well, and it's the same as is on the .25 machine which similarly has worked fine for years. I'm not saying it necessarily correct, since I'm not sure, but it has worked.

toes · 12-23-2006, 10:54 AM

At some point throughout yesterday while I was working on this, I know I had network access from this machine. I'm certain that I could successfully ping and was able to access the web pages.

I must have broken some permission or ownership while restoring something for this to be so screwy and for it to have broken after having at least some ability to access the outside world and to be accessed.

I'm going to try restoring my backup to another drive again and see how that comes out. I can't help but think this is user error (mine).

fordeck · 12-23-2006, 11:39 AM

OK so if I understand this, you can ping ip address's of machines on your network (i.e. 66.76.117.0) and even into your LAN from 66.76.117.26? But not beyond your gateway.

Can you ping the gateway 66.76.117.1?

You mentioned:

Quote:

I've been working all night on getting our small server set back up after an attack disabled it.

Is it possible that your ISP is filtering/blocking traffic due to the attack on/from 66.76.117.26?

toes · 12-23-2006, 11:56 AM

I apologize for not being as clear as needed. I've not tried pinging anything else on the 66.76.117.0 range except for those addresses which are ours (.24 and .25, besides the troublesome .26), though that's a good idea that I will test.

The only addresses that I can successfully ping (of those I have tried) from .26 are .25 and .24, and the hostname pings are successful for these as well.

Sometime through the evening/morning I did try pinging the 66.76.117.1 gateway (that's at the ISP) and it failed.

I hadn't considered the option that the ISP has killed the .26 connection on their end, but that may be worth looking into.

I'm booting up as I type with the latest restore from backup, so I'll see how this is going before continuing.