Changed IP on Red Hat box and can't reconnect
 

:newbie:

I'm not a new Unix admin, just new to Red Hat. I had to move a server from one subnet to another and I can't get connected to the new network. Any suggestions? Here's what I know:

The IP address shows up in the ifconfig eth0 output.
The netmask matches those of other servers on this subnet (class c).
The switch it's attached to is a cisco 6509 and the MAC address shows up on the correct port.
The server can ping itself, but nothing on the same subnet, much less outside the subnet. It can't even ping its gateway (returns a network unreachable message).
Both the server and the switch are set to full-100.


Thanks for any help!

K.B.

Since you made this change did you restart the network service??

If not then; service network restart

If still having a problem check the output from the typed commands:

netstat -n
cat /etc/resolve.conf

I actually have restarted the server serveral times. When I first hooked it up, it wasn't registering with the switch. All the pings I'm doing are by IP.

is your gateway configured correctly??

netstat -nr will show it to you.

Okay, the netstat -rn says (no network so it's not cut-n-paste)

Destination Gateway GenMask
10.133.0.0 0.0.0.0 255.255.255.0
169.254.0.0 0.0.0.0 255.255.0.0
127.0.0.0 0.0.0.0 255.0.0.0
0.0.0.0 10.133.0.3 0.0.0.0

The files I edited when changing the IP address were:
/etc/hosts (it only had the localhost address. I added the server's IP address)
/etc/sysconfig/network
/etc/sysconfig/network-scripts/ifcfg-eth0

Did I miss something?

Check the /etc/sysconfig/network-scripts/ifcfg-eth0

Maybe add/edit a line entry like; GATEWAY=XXX.XXX.XXX.XXX

Example:

With RedHat/Fedora, the GATEWAY= entry usually goes in the machine's /etc/sysconfig/network file; in the short run, though, to get birkelbk hooked back up to his network:
where a.b.c.d is the IP address of the default gateway

ought to do the trick. Do that after your "network reload" or however you're stopping & starting the network. I generally use ifup / ifdown and ifconfig; I haven't had the best of luck with or most consistent results in the world from using the /etc/rc.d/init.d/network script except at boot time where it usually works fine...

Good luck,

- Larry

Is your network vlan'ed through the cisco? If so, you would need to change the vlan for that port to that of the new network. Just a thought..

Well, I tried the route add, but it didn't work. Also, when I did the ifdown eth0 then ifup eth0, it came up as half-100. I noticed this when I started working on it, but changed it with mii-tool. How does one make full-100 permanent?

Yes, that was checked first thing. It's in the proper vlan on the switch.

BTW, I'm a she :cool:

At this point, can you reach anything at all from that machine (or can anything else reach it) or no? Can the machine ping itself via the loopback interface (lo) or via eth0?

I started thinking about this along the lines of "it worked before it was moved, what could have changed?" and a few thoughts occurred to me; it doesn't look like you're using a dhcp or bootp server, so that's probably not it. You're sure you've got the correct address and netmask for where you are now, right; you're not on a network that's subdivided so it's not a pure class A/B/C/etc. address?

Also ... if you're using a firewall, have you tried disabling it - briefly - and seeing if you can get anywhere then? If you're using iptables, which is the default for more recent RedHat/Fedora sytems, it's probably service iptables stop and service iptables stop (service iptables [start|stop|status|etc.] is equivalent to /etc/rc.d/init.d/iptables [start|stop|status|etc.], etc.).

What does your routing table look like now (either route or netstat -nr; 6 of one...)? How about your ifconfig?
Check your /etc/sysconfig/network and /etc/sysconfig/network-scripts/ifcfg-eth0 files; generally, that's where settings such as duplex, etc. go. You should pretty much never have to mess with mii-tool.

A lot of distros differ with respect to how they boot up, and how they bring up services like (take a wild guess) networking. With RedHat/Fedora - usually (assuming that no one has "customized" or "enhanced" something - the way things go are as:

/etc/rc.d/init.d/network [start|stop|restart|...] etc. is the System-V style "init script" that runs whenever the system enters a runlevel that it's supposed to run at; chkconfig --list network will show you what levels those are; ntsysv can also be used to control what runs at boot time, too, but to a more limited extent. When you bring up a RedHat/Fedora system, unless you've configured it to do otherwise, the machine's going to boot up to runlevel 3 or 5; runlevel will tell you which one you're at currently, and which one you were last at, if applicable.

The way that script does its thing is by running /etc/sysconfig/network-scripts/ifup, which, in turn, runs /etc/sysconfig/network-scripts/ifup-eth - which runs /etc/sysconfig/network-scripts/ifup-eth[0-9], etc.; and, /etc/sysconfig/network-scripts/ifup-aliases, and so on.

A lot of these scripts also use info from your /etc/sysconfig/network file; a given interface's configuration is going to be set by a combination of the settings in that file, in it's interface specific init file (for example, ifcfg-eth0 for interface eth0) - and by anything that's applicable in udev (/etc/udev): /etc/udev/conf and /etc/udev/rules.d - and by anything that's applicable to a given interface in /etc/modprobe.d.

It sounded like you were familiar with a different distro & were having to manage a RedHat one; hopefully, the info above will be of some help. That's in a perfect world, though; I've seen systems where all sorts of interesting things have been done in various places.

If you hit the point where you think that you've ruled everything else out, there's always the "grab another network adapter and bring it up" approach; at least that'll tell you if something is fundamentally wrong with either your networking setup; there's always the possibility that the card itself died, too - or that it's not compatible with the particular router / concentrator / etc. that you're connecting to now. There are certain DCE/DTE combinations that flat-out will not work properly; I've found that to be the case with two specific routers when I've attempted to use them with a particular brand of laptop's built-in ethernet port.

Good luck with it; I will check back later and see if you've made any headway. I move machines around constantly, and while it admittedly turns into a project some times, it's not generally too painful. I'd like to know what's happened to you before it happens to me...

Oh gosh - I'm sorry; I'm usually very careful about about not using "his", "he" and the like. My apologies...

- Larry

I can't reach anything at the moment and nothing can reach it. I can ping localhost and the local IP address. The outside ping always results in "Destination Host Unreachable" which leads me to believe that I have a network configuration problem somewhere.

No DHCP or bootp. I know I've got the right netmask and address because it's in the same range as our Exchange servers and I doublechecked their netmasks to make sure I was using the right one.

I tried doing service iptables stop, but that didn't change anything either.

The routing table still looks like it did before (route doesn't work due to DNS not being available):


Destination Gateway GenMask
10.133.0.0 0.0.0.0 255.255.255.0
169.254.0.0 0.0.0.0 255.255.0.0
127.0.0.0 0.0.0.0 255.0.0.0
0.0.0.0 10.133.0.3 0.0.0.0

The ifconfig is correct with the IP address, netmask, broadcast and is UP.

Here are the contents of /etc/network:

NETWORKING=yes
HOSTNAME=hostname *changed for security reasons
GATEWAY=10.133.0.3


/etc/network-scripts/ifcfg-eth0 looks like:

DEVICE=eth0
ONBOOT=yes
BOOTPROTO=static
IPADDR=10.133.0.17
NETMASK=255.255.255.0
GATEWAY=10.133.0.3


I don't want to change out the NIC's because first, I don't know if we have another one; second, the likelihood of it breaking when I carried the system across the room is slight; and third, the switch is getting the MAC from it.

Thank you for all the good information about how Red Hat starts up the network. It's a lot more complicated than it needs to be, but I guess that's what you get from an OS designed by committee. :rolleyes:

No worries, not offended, I just like to yank chains occassionally. ;)



Karen

I think I may have it... Nasty little thing, too...

Earlier on, when you tried:
did you get
then, too?

The reason I ask, is that given your gateway address setting, it would generally tend to correspond to an address with a different bitmask than the one you're using; pretty much all the utilities guess at what your netmask, gateway, etc. are given what info you do give them.

Try this:
I have a feeling that will do the trick...

If you want to see what netmask, route, etc. are going to use for a netmask given an ip, and so on, try ipcalc. I think what's happening here is that given the address of your gateway, route is using a netmask of /8 when it goes to add the route, and not /24:
Using that netmask, everything else is just a tad off:
But, if we tell it what the netmask really is, things work out a little bit better:
There was something about the gateway address being 10-dot-anything that kept gnawing at me; that, and the fact that the interface wouldn't come up for you when you tried ifconfig. It finally occurred to me to run the address through ipcalc.

Everything else you have set looks fine. I'll keep my fingers crossed... Thinking back on it, problems like that are why I got into the habit of starting and stopping an interface with ifconfig directly; once I know it works that way, I'll go ahead and automate the process. From a shell prompt, the following:
should always work for you; if it doesn't, then you know you've got a problem. BTW, route -n will work if you're incommunicado with your DNS server; it's pretty much the same as netstat -rn; I can't recall why or when, but I could swear at some point I needed one or the other and only had the other.

My pleasure; I've been glad to have someone else help me out a time or two... It's complexity is a sort of love-hate thing for me; the way it's set up, I can get it to do anything I want it to - just not always what I meant for it to do.

Please post back as to whether or not it worked when you get a chance.

- Larry

Well, Larry, it didn't work. I'm starting to think there's some security setting that our "data security" guys set up that's not letting talk on this new network. Are you a RedHat security expert, too? Anyone else care to guide me in this, too?

Thank you!!!!

You don't even need a gateway to ping machines on the same subnet, so there's a basic problem that needs to be addressed before moving onto the gateway.

Is it possible that you have protected ports configured on that switch?

I don't believe so. The switch this one is attached to is open to our internal network and we can modify the ports at will.

One other thing I found out this afternoon was there is a /etc/rc.local file that has static routes listed in it that don't appear in the netstat -rn. Where are static routes applied in Red Hat?

I didn't see if you mentioned this, but are you at least getting link lights on the switch port and on the server's NIC?

If you are getting link lights, then I would go back and re-address the possibility of it being a VLAN issue, since from what you've described it sounds like you are using VLANS on this switch.

Sorry that didn't work, Karen; this just keeps getting better and better...

That's troubling; they should be getting applied right then and there. "route" won't create what it sees as an impossible route, so if the right interface(s) aren't up, they're not gonna happen.

This doesn't sound so much like a security issue, though it could be; it could just as easily be a bad cable, bad port, or something like that, too... I think that was sort of where alienux was heading with this too; you're absolutely certain that you've got a physical connection, and that this isn't a VLAN issue?

If this were me, I think I would put another computer that I knew worked on that same port; assuming that it worked, I would go through and change its settings to the ones you're planning on using, and see what happens...

Please, as you discover what's going on, post what you're finding out back up here; this is a good one...

- Larry

Yes, I am getting link lights and the MAC address shows up on the switch.

Where does the rc.local get executed in the startup? Does it not apply the static routes if the network isn't available when it tries to do it? I didn't think that would matter, so I would expect to see the static routes in the netstat -rn.

Hi, Karen -

rc.local is started by rc.sysinit, which runs it (rc.local) after it's done with everything else. rc.sysinit is started by inittab, which is read by init and used as the system changes runlevels.

The two other files that come into play a lot are /etc/profile and /etc/bashrc; when bash is started as a login shell, /etc/profile runs your bash profile (~/.bash_profile) and your bashrc (~/.bashrc), then /etc/profile. There may also be files (names and contents vary) in /etc/profile.d that are executed as the shell's environment is initializing. csh (tcsh) has its own, similar, way of doing the same thing.

As far as the routes go, and as to why you're not seeing them in the netstat you did - basically, you'll see the routes in netstat if the system can create them; if what they depend upon doesn't exist, and route determines that they're impossible, then their creation will fail. If they had been created and were there, you would see them.

I still think it's got something to do - directly or indirectly - with the fact that you've got a class A address (10.b.c.d) and a class C subnet mask; I think you're ultimately going to find there's a class C address that's supposed to be getting created - somewhere - which is your ticket onto your router / switch. I could be wrong, though.

Since you have a link, and at least some communication (if the switch can read your MAC address, you have two-way communication, right? Maybe simplifying things a little might help; try something like configure that part of the router / switch / etc. you're using so it's not using VLAN, etc. (so that it's just a basic gateway), then bring up an interface, configure it, verify that you have connectivity... then, go back and add in whatever you had to comment out?

I don't really know what else to suggest that you try... I'll certainly keep my fingers crossed, for whatever good that'll do.

- Larry

Well, after all the head banging and grief, the problem was that it was in the wrong VLAN on the switch. Now, as far as I was concerned, the VLAN was correct, but I found out that I was given the wrong VLAN. After changing the port, the machine was properly accessible.

Thank you all for your suggestions and support. I learned a lot more about Linux from the experience.

Hi, Karen -

For whatever it's worth, I had faith that you would ultimately figure out. I'm glad you did; it was certainly a real gem of a problem. There aren't many things I can think of that are worse than being given bad info, relying on it, and, as a result, watching your own deliverables slip.

Just my opinion, but I think the way you thought the whole thing through, and stuck with it, was commendable.

Ah; isn't that always the case? It certainly seems like things never progress "the easy way", and that it's only through late night and weekend "this darn thing should've worked by now", uh, "study sessions", that I've ended up learning much of anything that's been worth learning. I think that's in part because no sane person would have bothered to delve into some of the more arcane aspects of the O/S unless they had some sort of pressing need to...

At any rate, I'm glad you're up and running, and wish you all the best in both this and in your future endeavors.

- Larry