Could Copying Something From /usr/lib to /lib stop connectivity?
 

Hi,

I have a dedicated server, and one of the techs perfomed the following command:

soon after we were no longer able to connect to the server.

Is it possible that copying a file like that could cause network problems, and prevent us from ssh'ing in?

Thanks.

Depending on your concrete distro and how libs are handled, it can not only break connectivity, but break every single binary that links to libc, which are virtually all of them. Yes, that would render the system unusable. Doing this kind of thing is not usually a good idea, for example, in my system I have these:

The *real* library is /lib/libc-2.10.1.so, the other two are not real libraries. /lib/libc.so.6 is a symlink which points to the lib, breaking it will break any program that relies in finding libc at that location. The file at /usr/lib/libc.so is not even a binary file, but an ld script. Overwriting the real library with it will make no good, however in this case I *guess* that overwriting the symlink with it is safe at the immediate future, though it no doubt will bring future issues.

This much depends on what exactly the layout of the libraries in that server are. If libc is broken then the server wouldn't be able to work at all, so the real question is if you have tested the server locally to see if it's still working.

Hi,

Thanks for your reply.

As i mentioned in the first post this is a dedicated server - so i personally dont have local acess to it. However the providers engineers will be at the datacentre tomorrow.

I can confirm that lib.so.6 was in fact also in /lib so its quite possible that the tech overwrote the real lib with the symlink in /usr/lib.

My question then, is whether there is any solution at all to fix this - even locally?

Note: there was also a copy of lib.so.6 in /lib/i686/nosegneg (untouched)

If libc broke, it won't even boot (and nothing will work from that momment when the lib was overwritten).

Locally everything can be fixed, just boot from another disk or any kind, and manually restore the files from rpm's or whatever.

ok, so just to confirm, if i tell the providers tech's to boot the server from a disk, and restore the lib.so.6 file to /lib - everything should go back to normal

Yes, as long as that's your only problem. I can't know for sure if that's what caused your problem. All I say is that it can be potentially harmful to the point of rendering the whole system unusable, breaking every single binary file on your system.

well as i mentioned, the server only became inaccessible when one of the tech perfomed that command (a few seconds after). So hopefully you are correct in thinking that a replacement will be all that is necessary - and no permenant damage was done.

If I might ask, what inspired the tech to perform this copy in the first place? Is/was there a problem with the server that looked as though it could be solved by doing this?

Sasha

I'm unsure of the exact details, but i believe the tech was having some problems with the rar binary.

It seems that whenever he tried to use the rar command, he recieved the error:

but if he performed the command "/usr/bin/rar" it worked fine.

Also, if he ran the same binary under a different name it worked. i.e.

cp /usr/bin/rar /usr/bin/rar2

using command "rar2" it worked fine.

Removing /usr/bin/rar and placing th ebinary from the source package in /usr/bin also didnt fix the problem, with rar giving the above error.

for some reason the binary wouldnt work by just using "rar"

this was when he subsequently overwrote the file, thinking it was a solution.

However that is the least of my concerns at th emoment, and will just be happy fixing the server.

The above information is worrying, especially on a server.
The reason that it is worrying is that there seems to be an executable called rar which is on your $PATH and is being found before the version at /usr/bin/rar

What is the output of which rar ?
I'll be surprised if it's /usr/bin/rar (which is what it should be)
I hope you have not been running "rar" (which may not be rar at all) as root.

What is the output of which rar ?

- Not sure exactly what you meant by this


I should also note, that "rar" was working fine for several weeks until today. Today a tech tried to incorporate it into an ftpd server that we are running, and somehow during that process he "broke" rar.

Also, no, rar wasnt being run as root.

Something strange happened in there. I'd give rkhunter, chkrootkit and clamav a round just in case. But that's another issue.

About the rar being broken, usually it's a better idea to recompile the binary or download one that matches your glibc, however I think that rar is closed so that might not be an option. A valid solution for this would be to use an alternate libc, via LD_PRELOAD or something. But overwritting the glocal libc (which as I said is used by all the binaries in your system) is never a good idea. Gosh, it's problematic even when you do it in a controlled fashion using your package manager, imagine when you throw a critical library like that in there without even checking that it's compatible with your binaries' ABI.

well i believe that maybe when he was getting the rar binary to work with the ftpd servers, which initself required the copying of the relevant binaries and lib's - as they are run in isolation of the box, and just within the ftpd, he may have inadvertadly messed up how the normal rar binary execution occurred. Also believe he wasnt really paying attention to which lib he was replacing, as yes you are correct in saying that it is extremely risky to mess around wiht such an important library.

I'm not too worried about there being any rootkits, with passwords on accounts changing weekly, and iptables blocking both incoming and outgoing connections except on specific limited ports - some also limited by sourceip and destination ip - the box is fairly safe from such things.