LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-16-2009, 03:14 AM   #1
removed4
LQ Newbie
 
Registered: Oct 2009
Posts: 2

Rep: Reputation: 0
Exclamation Chmod nightmare - biggest mistake of my year


Hello,

I'm new and only decently versed in ubuntu/debian platforms. I'm going to blame the cold medications on this one... I managed to lock myself out of my system entirely. I was working on foswiki permissions and found a decent guide here: http://foswiki.org/Support.SettingFi...ightsLinuxUnix

Now, I had run these commands successfully before. This time I ran them at root(/) as root...

Here is exactly what I put in(I made a small edit on the LocalSite.cfg path):

Code:
find . -type d -print -exec chmod -v 755 {} \;
find data -name '*.txt' -type f -exec chmod -v 644 {} \;
find data pub -name '*,v' -type f -exec chmod -v 444 {} \;
find lib -type f -exec chmod -v 444 {} \;
find locale -type f -exec chmod -v 444 {} \;
find pub -type f -exec chmod -v 644 {} \;
find bin -type f -exec chmod -v 555 {} \;
find bin/logos -type f -exec chmod -v 444 {} \;
find templates -type f -exec chmod -v 444 {} \;
find tools -type f -exec chmod -v 555 {} \;
chmod -v 644 etc/foswiki/LocalSite.cfg
chmod -v 644 data/.htpasswd
chmod -v 644 data/mime.types
chmod -v 644 bin/LocalLib.cfg.txt bin/.htaccess.txt
chmod -v 444 bin/setlib.cfg
chmod -v 444 tools/extender.pl
chmod -v 444 working/tmp/README working/README working/registration_approvals/README
chmod -v 444 working/work_areas/README
chmod -v 660 working/.htaccess
chmod -v 444 AUTHORS COPYING COPYRIGHT index.html INSTALL.html LICENSE pub-htaccess.txt readme.txt
chmod -v 444 robots.txt root-htaccess.txt subdir-htaccess.txt FoswikiHistory.html
chmod -v 444 foswiki_httpd_conf.txt ReleaseNotes01x00.html
I won't pretend to fully understand what is happening above, all I knew is that it worked once. Now I've tried running recovery mode but I keep getting a Kernal Error about finding init but having an error.

Interestingly enough, before I tried to reboot my phpmyadmin was still up and running, I have just lost write privileges on the databases

This happens immediately for normal boot and takes a little longer when trying a recovery boot. Since these failed I decided to load in a Live CD and try using the recovery option. I can't execute a shell on my boot drive, but I can run it in an installer environment.

In the environment I mounted the harddrive and ran a number of (probably idiotic) chmod and chown rules for root to own everything at 777 or 440.

This failed, so I also tried to run a Live CD and attempted the same thing at no avail. I still get the kernal panic.

I don't think I could've crashed my companies primary webserver/fileserver at a worse time. I think I have the flu and am getting groggy, but would rather not lose my job. To answer the obvious - yes I have backups and with the Live CD I can probably recover all of the necessary files (60GB or so) Not a fast, or fun process at midnight.

Does anyone have any suggestions, or perhaps a more specified approach with how to chmod a mounted a drive (I'm not sure if it is even taking).

System: Ubuntu 8.04 amd64.

Please advise.

Sincerely,
PertinaxVir
 
Old 10-16-2009, 04:28 AM   #2
rizhun
Member
 
Registered: Jun 2005
Location: England
Distribution: Ubuntu, SLES, AIX
Posts: 268

Rep: Reputation: 47
Hi PertinaxVir,

I feel for you, I really do.

All it takes is one lapse in concentration and very bad things can happen.

Presumably those commands are meant to be run from the install directory of this 'foswiki' application.

Somebody at my workplace did something very similar last week.

I spent ~6 hours fixing permissions, one-file-at-a-time and still couldn't get it running properly; in the end I restored it from a backup.

Maybe someone else knows some clever bit of software that can help you, but I'd be prepared to restore from the latest backup.

Just for info, the commands that ruined your evening were these:
Code:
$ find . -type d -print -exec chmod -v 755 {} \;
$ find bin -type f -exec chmod -v 555 {} \;
The first command will have recursively set EVERY directory on your system to have 755 permissions.

The second command made everything in /bin 555.

In the future remember; chmod is a VERY powerful tool and you should only run a command as the 'root' user when it is ABSOLUTELY necessary.

And the golden rule: Never run a command you don't understand.

Google & 'man' pages are your friends!

Sorry I can't be more help.

Good luck.
 
Old 10-16-2009, 04:48 AM   #3
Forrest Coredump
Member
 
Registered: Oct 2009
Location: Southwestern United States
Distribution: Redhat Enterprise Linux 4-5 (Current RHCE), Fedora Core 11 (FC11), Arch Linux, BT3 (Current GCIH)
Posts: 42

Rep: Reputation: 16
Um, I would argue that the following was his downfall (at least not allowing him to boot);

find lib -type f -exec chmod -v 444 {} \;

That little gem there changed every file under /lib to -r--r--r-- Good luck booting with those permissions - remember what's in /lib... All of your shared libraries, kernel modules, etc... (most file are ELF, and ELF's REQUIRE the X bit set).

To solve this boot from a live CD, mount the block devices associated with you system (creating the whole system tree under /mnt) "chroot /mnt", su - to set your path, etc... At that point make sure everything in /lib is executable by at minimum root, I would go 755 at that's the majority of file in /lib.

Hope this helps

Last edited by Forrest Coredump; 10-16-2009 at 04:51 AM.
 
Old 10-16-2009, 04:50 AM   #4
i92guboj
Gentoo support team
 
Registered: May 2008
Location: Lucena, Córdoba (Spain)
Distribution: Gentoo
Posts: 4,063

Rep: Reputation: 381Reputation: 381Reputation: 381Reputation: 381
Those commands can really mess up your system, you just need to run them with a slight typo or from the incorrect location to completely break your OS *completely*. Restoring the permissions is not a trivial task, most times it's better to just reinstall or restore a working backup (we all do backups, don't we?).

Code:
find . -type d -print -exec chmod -v 755 {} \;
When running this kind of command, the ideal thing is to understand what's going to happen, and to triple check the command character by character, and the current working directory. Use "pwd" for that if your prompt is confusing, that's what it is for.

In second place, when doing this, it's better to first run the command with an extra "echo", like this:

Code:
find . -type d -print -exec echo chmod -v 755 {} \;
This will "echo" each command instead of running them. Check the commands, check the affected files, and only after you check and agree, remove the echo and run again the command to do the real work.

If you are in the foswiki (whatever that is) directory when running it, all will be fine, but if you are elsewhere, it will affect all the file from the current directory and below. So, if you are at / it will happily screw your whole OS.

Now it's probably late to fix something, but there are ways to backup your permissions, it's easy and relatively non-expensive in processing terms, so you might want to use something like this in a cron script to back them up weekly or something:

Code:
find /whatever/path -type f -or -type d -exec stat --format="%a %n" "{}" \; > chmod.txt
This other oneliner would restore them when run from the same "/whatever/path":

Code:
while read LINE; do PERMS=${LINE%% *}; FILE=${LINE#* }; chmod $PERMS "$FILE"; done < chmod.txt
You might want to save these for future usage and/or reference.
 
Old 10-16-2009, 04:53 AM   #5
i92guboj
Gentoo support team
 
Registered: May 2008
Location: Lucena, Córdoba (Spain)
Distribution: Gentoo
Posts: 4,063

Rep: Reputation: 381Reputation: 381Reputation: 381Reputation: 381
Quote:
Originally Posted by Forrest Coredump View Post
Um, I would argue that the following was his downfall (at least not allowing him to boot);

find lib -type f -exec chmod -v 444 {} \;

That little gem there changed every file under /lib to -r--r--r-- Good luck booting with those permissions - remember what's in /lib... All of your shared libraries, kernel modules, etc... (most file are ELF, and ELF's REQUIRE the X bit set).

To solve this boot from a live CD, mount the block devices associated with you system (creating the whole system tree under /mnt) "chroot /mnt", su - to set your path, etc... At that point make sure everything in /lib is executable by at minimum root, I would go 755 at that's the majority of file in /lib.

Hope this helps
It could be, but only if he ran the command in /, and then half of the commands there would be equally dangerous. The problem when running massive commands like these ones using relative paths like "lib/" instead of "/full/path/to/lib" is that if you are on the wrong "$PWD" you are dead.

That why you must check:
  • the output of pwd
  • the command, byte by byte
  • the list, using echo as said above

And only after that, proceed.
 
Old 10-16-2009, 05:02 AM   #6
rizhun
Member
 
Registered: Jun 2005
Location: England
Distribution: Ubuntu, SLES, AIX
Posts: 268

Rep: Reputation: 47
Quote:
Originally Posted by Forrest Coredump View Post
Um, I would argue that the following was his downfall (at least not allowing him to boot);

find lib -type f -exec chmod -v 444 {} \;
Ohhh... You're not wrong -- didn't spot that little beauty!
 
Old 10-16-2009, 05:09 AM   #7
Forrest Coredump
Member
 
Registered: Oct 2009
Location: Southwestern United States
Distribution: Redhat Enterprise Linux 4-5 (Current RHCE), Fedora Core 11 (FC11), Arch Linux, BT3 (Current GCIH)
Posts: 42

Rep: Reputation: 16
No doubt, ugly little scenario... especially when you consider `ldd /sbin/init` ;-)

Thanks,
FCDump
 
Old 10-16-2009, 05:13 AM   #8
removed4
LQ Newbie
 
Registered: Oct 2009
Posts: 2

Original Poster
Rep: Reputation: 0
You guys are awesome. I had to get out of the office, I think I was spreading the flu around the server room.

I did manage to use shell to chroot and act as root on my actual install(just the tip of the iceberg, as I'm sure you all know). I managed to hammer our permissions for sql and phpmyadmin, but decided 4 hours after a 9 hour day in the office while being sick was too much.

I'll probably end up reinstalling the rig when I have the time. I really do appreciate all of your help above. I'll come back when I'm not about to pass out and write out what I did to get back into my system (for the next unlucky chap).

Thanks again,
PertinaxVir
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: What's OLPC Biggest Mistake? Negroponte Says Sugar LXer Syndicated Linux News 0 07-20-2009 11:42 PM
terrible mistake - chmod -R o+rx /dev ddaas Red Hat 1 08-05-2005 05:38 AM
what is the biggest mistake you've ever made as admin? ddaas General 12 03-09-2005 03:21 PM
Biggest linux event this year Rico16135 General 3 05-30-2004 12:12 AM
Ok i made the biggest mistake anyone could ever make... Soujiro Linux - Software 1 04-24-2004 02:47 AM


All times are GMT -5. The time now is 08:56 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration