[SOLVED] How to recover from chmod 0 /bin/chmod

shruggy · 06-30-2020, 11:50 AM

Recently, user garryjp posted a couple of Linux certification exam questions both here at LQ and on StackExchange. One of them, asked at SE, catched my attention.

Quote:

How would you recover from

Code:

sudo chmod 0000 /bin/chmod

on a running system? Describe all possible alternative options you would use to recover from this command.

There were quite a few solutions suggested in the comments. Not all of them were following the requirement "on a running system" though. So with all due respect to the guys at SE who commented on this, I'll try to recap them here. The actual chmod command can be /bin/chmod on some (older) systems, or /usr/bin/chmod on others, so I'll be referring to it as $CHMOD from now on. You may assign

Code:

CHMOD=$(which chmod)

before trying out the code bits below.

So here it goes.

Fix file permissions directly.
- Another language interface to chmod(2) syscall
  The actual task of changing file permissions is being done by system call chmod(2). The command chmod(1) is just a command line interface to it. So we'll just use another interface to that syscall.
  
  The most straightforward solution is to write, compile and execute a small C program using examples in the chmod(3p) manpage.
  Or, if you happen to have Radare2 installed, perhaps you'll be able to call the library function directly from the command line:
  Code:
  
  rarun2 runlib=/lib64/libc.so.6 runlib.fcn=chmod arg1="$CHMOD" arg2=MODE
  That is, if you figure out how to specify an integer argument value for MODE because I couldn't. %493 mentioned in the rarun2 manpage obviously doesn't work, neither does :01ed. Setting the arg2 to an arbitrary (even empty) string would somehow set file permissions to 1760 on CentOS 8 (glibc 2.28, radare2 4.2.1), to 1700 on Ubuntu 18.04 (libc6 2.27, radare2 2.3.0), but to 2000 on Ubuntu 20.04 (libc6 2.31, radare2 4.2.1).
  
  The green part obviously would be specific to your system. If not sure, find out with:
  Code:
  
  ldd "$(which rarun2)"|awk '/\/libc\.so/,$0=$3'
  The easiest solution probably is this Perl one-liner though:
  Code:
```
perl -e "chmod 0755,'$CHMOD'"
```
  Most languages, both scripting and compiled, include some means of changing file metadata, either in their core parts or in a library/module. Perl has two advantages going for it. 1) chmod is part of the core language, so the syntax is very concise; 2) its ubiquity: Perl is installed on pretty much every Linux system.
  
  PHP may compete on the first account, but not on the second:
  Code:
```
php -r "chmod('$CHMOD',0755);"
```
  Python may compete on the second, but not on the first. Besides, you'll need to figure out if the system in question has python2 or python3 or both and adjust accordingly:
  Code:
```
python2 -c "import os;os.chmod('$CHMOD',0755)"
```
  Code:
```
python3 -c "import os;os.chmod('$CHMOD,0o755)"
```
  Ruby?
  Code:
```
ruby -r fileutils -e "FileUtils.chmod 0755,'$CHMOD'"
```
  Tcl? Well, Tcl is rather verbose compared to others. Besides, it doesn't provide for one-liners, so you'll need to run tclsh interactively, or write a small script:
  Code:
```
#!/usr/bin/tclsh
file attributes /bin/chmod -permissions 0755
```
  Lua is an interesting case. It's conceived as an embedded interpreter for other applications, so it doesn't include much by default. There are external packages like lua-fs that provide required functionality. Luckily, rpm includes Lua interpreter together with everything needed. I tend to have rpm installed even on Debian-based systems for purposes of inspecting, unpacking and converting RPM packages.
  Code:
```
rpm -E "%{lua:posix.chmod('$CHMOD','755')}"
```
- File manager (kudos to rnturn for tipping me on this)
  Let's start with a rather unusual beast here, the mighty Emacs. Its Dired mode practically makes it a file manager of sorts, too. In the examples below, bold parts are key sequences in the usual Emacs notation, so e.g. C-s means Ctrl+S.
  
  Emacs:
  Code:
```
emacs /bin
C-s chmod
M 755
```
  Midnight Commander keybindings are modelled after Emacs, no wonder they're similar:
  Code:
```
sudo mc /bin
C-s chmod
C-x c rwxdbhs
```
  If there are file managers with Emacs-like keyboard shortcuts, there must be ones with Vi-like keybindings, too. So here they are:
  
  ranger:
  Code:
```
sudo ranger /bin
/chmod
755=
```
  Vifm:
  Code:
```
sudo vifm /bin
/chmod
755cp
```
  Speaking of Vi: for quite some time now, there has been a nifty function in Vim, setfperm:
  Code:
```
vim -e --cmd "call setfperm('$CHMOD','rwxr-xr-x')|q"
```
  This one is a one-liner though, so it probably makes Vim a contender for the previous section.
  
  I avoid doing administrative tasks with GUI tools, so there are no examples with GUI file managers (Nautilus, Nemo, Caja, Thunar, Dolphin, Krusader, PCManFM, SpaceFM, whatever). For them to be able to gain root privileges, some PolicyKit adjustments may be required.
  
  But they all are fully capable of changing permissions for files in your home directory, owned by you.
- ACL
  Code:
```
getfacl $(which chown)|setfacl --set-file=- $CHMOD
```
  or just
  Code:
```
setfacl -m u::rx $CHMOD
```
  Also see below NFS ACLs.
- Use package manager facilities
  Some package management systems provide means of either restoring file metadata to the state they were at the installation time:
  Code:
```
rpm --restore $(rpm -qf $CHMOD)
```
  or overriding them:
  Code:
```
dpkg-statoverride --update --add root root 755 $CHMOD
dpkg-statoverride --remove $CHMOD
```
- Edit inode
  This one is a heavyweight . Well, for completeness sake. Things we'll need to know are 1) FS type and device of the FS where $CHMOD is located (df -T $CHMOD) and 2) inode number of $CHMOD (ls -i $CHMOD). Then it's the task of a bin/hex editor or a FS debugging tool. E.g.
  
  debugfs for ext2/3/4
  Don't confuse this ext2/3/4 debugger with an in-kernel pseudo-FS also named debugfs!
  
  Actually, I couldn't make it work for me. Neither this:
  Code:
```
debugfs -wR "sif $CHMOD mode 755" $(df $CHMOD|sed '$!d;s/ .*//')
```
  nor that:
  Code:
```
debugfs -wR "sif <$(stat -c%i $CHMOD)> mode 755" $(df $CHMOD|sed '$!d;s/ .*//')
```
  Any thoughts?
  
  xfs_db for xfs
  Well, this one doesn't work on a running system because the FS must be remounted read-only, and it won't allow me to do so saying "mount point is busy". I didn't pursue it any further.
  Code:
```
fs=$(df $CHMOD|sed '$!d;s/ .*//')
mount -o remount,ro $fs
xfs_db -i -x \
  -c 'type inode' \
  -c "inode $(stat -c%i $CHMOD)" \
  -c 'write mode 755' $fs
```
- Do it remotely
  The first I can think of is
  Code:
```
echo chmod 755 $CHMOD|sftp -b - root@affected_host
```
  This obviously won't work if root is not allowed to ssh, but I'm pretty sure there are many other ways to accomplish this remotely either from the command line or e.g. via an administrative web interface.
Copy content to another file setting the permissions
Any tool capable of moving files around and setting permissions in the process can be abused for the side effect.

The most obvious candidate is install:
Code:
```
install $CHMOD /tmp/
```
Or just cp over another file that happens to have the right permissions:
Code:
```
cp /bin/chown /tmp/chmod
sudo cp $CHMOD /tmp/
```
There are many others as well. E.g. rsync
Code:
```
rsync --chmod=755 $CHMOD /tmp/
```
Or even tar:
Code:
```
tar -C "${CHMOD%/*}" -c --mode=755 "${CHMOD##*/}"|tar x
```
More adventurous among us may even attempt writing the copy back over the original file:
Code:
```
tar -cP --mode=755 $CHMOD|tar x
```
Or how about git?
Code:
```
mkdir repo
cd repo
git init
cp $CHMOD .
sudo git update-index --add --chmod=+x chmod
git checkout chmod
```
Copy to a filesystem that makes the problem disappear
A variation of the previous. We just need a filesystem that enforces certain permissions for all files. FAT is an obvious candidate, but many others will do as well.
Code:
```
mkdosfs -C /tmp/fat.img 180
mount /tmp/fat.img /mnt
cp $CHMOD /mnt
```
Many filesystem types have mount options to set permissions for all files. For some they only work as umask by combining the assigned value with permission bits for the file to make effective permissions. But for some, mostly for such filesystems as FAT that don't keep permission bits for every file, those options set effective permissions themselves. There are even means to enforce file permissions for some types of network FS like CIFS or DAVFS. Interestingly, NFSv4 has its own set of ACLs, completely independent of POSIX ACLs, so if we've copied the file to an NFS share, we can then
Code:
```
nfs4_setfacl -a A::OWNER@:RX copied_file
```
Restore/recreate the file
E.g. by reinstalling the package
Code:
```
dnf reinstall $(rpm -qf $CHMOD)
```
Code:
```
apt reinstall $(dpkg-query -S bin/chmod|cut -d: -f1)
```
But anything goes as long as you'll get an intact copy of the damaged file: backup, ZFS/btrfs snapshots, LVM2 snapshots, VM snapshots if your system is being run inside a VM, an ISO image of the install disk, and so on.
Use a working chmod from another place
- On-system
  I tend to always have busybox installed, just in case. So
  Code:
```
busybox chmod 755 $CHMOD
```
  On Debian-based systems at least, another possibility is extracting chmod from an initrd image.
  Code:
```
mkdir /tmp/initrd
unmkinitramfs /boot/initrd.img /initrd
/tmp/initrd/usr/bin/chmod 755 $CHMOD
```
  Unfortunately, regular initrd images created with dracut don't include chmod, so if your distro uses dracut (most RPM-based distros do, e.g. Fedora, RHEL, CentOS, Oracle, SLE, OpenSUSE) the things get a bit trickier. You'll need a special rescue initrd image. On RHEL/CentOS 7/8, such an image has the word rescue in its name. It likely was created when you system was installed. Hopefully, you didn't remove it to save the space. You'll find it with this command:
  Code:
```
sudo grubby --info=ALL|awk -F\" '/^initrd=/&&/rescue/,$0=$2'
```
  Or just examine the output of grubby --info=ALL for anything that looks like a rescue initrd image. Extract and run chmod from it:
  Code:
```
cd /tmp
rescue=$(grubby --info=ALL|awk -F\" '/^initrd=/&&/rescue/,$0=$2')
lsinitrd --unpack "$rescue" usr/bin/chmod
usr/bin/chmod 755 $CHMOD
rm -r usr
```
  Actually, I like the lsinitrd script from dracut better than unmkinitramfs. So I can even imagine me doing something like this on a Debian-based distro:
  Code:
```
apt install dracut-core
lsinitrd --unpack /boot/initrd.img usr/chmod
```
- Reboot to a rescue environment
  The title says it all. E.g., on a recent Debian-based system
  Code:
```
sudo -i
apt install grml-rescueboot
update-grml-rescueboot
update-grub
reboot
```
  Older Debian systems don't have the update-grml-rescueboot command, so you'll need to download a Grml ISO image manually and put it to /boot/grml/. After reboot into the rescue environment it's a matter of mounting the right FS, so the df command from the inode editing entry above will be a useful prerequisite for this one as well.
- Copy a working chmod from another system
  It even needn't to be the same release as on the affected host. Basically, chmod depends only on glibc, and all we have to check is 1) the architecture is compatible (a 64-bit binary cannot be executed on a 32-bit system), and 2) the chmod we'll copy wasn't linked against a newer glibc than what we have on the affected host. An older chmod will do. Actually, I tried this by copying the chmod binary from a 32-bit CentOS 6 to a 64-bit CentOS 8. Worked like a charm. It goes without saying that I already have had the 32-bit glibc in place on the CentOS 8 system:
  Code:
```
dnf install glibc32
```
  Before attempting this, check the shared object dependencies of both chmod versions with ldd -v. And this bring us to the last item.
Run through dynamic linker
chmod (as almost everything nowadays) is dynamically linked. In the output of ldd you'll see a line with ld-linux.so (or ld-lsb.so or ld.so). This is the dynamic linker.

Now, recall how we run a script. There are two ways to do it.

One is to put shebang as the first line, make the file executable and run it by its name. So, taking a shell script as an example,
Code:
```
sed -i '1i #!/bin/sh' myscript
chmod +x myscript
./myscript
```
Another way is to call the script as an argument to the script intepreter:
Code:
```
sh myscript
```
In this case setting the executable bit for myscript is unnecessary.

For dynamically linked ELF binaries, there is a certain similarity here. Obviously, we don't use shebang lines for them, but we're supposed to make them executable and then run by name. OTOH, we also can call them as arguments to dynamic linker, even if they don't have the executable bit set. Which gives:
Code:
```
$(ldd $CHMOD|awk '/\/ld-/,$0=$1') $CHMOD 755 $CHMOD
```
Or, if you have patchelf installed:
Code:
```
$(patchelf --print-interpreter $CHMOD) $CHMOD 755 $CHMOD
```

Comments are welcome. Have I forgotten anything?

dugan · 06-30-2020, 01:05 PM

I saw this question on Twitter.

The most popular answer was to cat chmod into an existing file that had execute permissions, then move/rename that file to where chmod was.

rnturn · 06-30-2020, 02:09 PM

Quote:

Originally Posted by shruggy

Recently, user garryjp posted a couple of Linux certification exam questions both here at LQ and on StackExchange. One of them, asked at SE, catched my attention.

Comments are welcome. Have I forgotten anything?

Emacs handles this quite nicely. "cd" to the directory, issue "emacs .", scroll down to the 'chmod' listing, and enter "M". You'll be prompted for the mode you want.

shruggy · 06-30-2020, 02:37 PM

Ah yeah, didn't think about file managers. E.g. Midnight Commander: Ctrl+X,c on the file. Or 755= in ranger.

And Vim nowadays also can this:

Code:

vim -e --cmd 'call setfperm("/bin/chmod","rwxr-xr-x")|q'

Update. I amended the top post. Thank you.

syg00 · 06-30-2020, 06:07 PM

Gotta like your doggedness shruggy - that's quite an analysis.

I got sick to death of these arcane, contrived situations in things like certifications/interviews. Anyone that allowed that situation to arise should be fired, not hired.

tinfoil3d · 06-30-2020, 11:30 PM

Love this. Did you find all of those solutions by yourself? Even though far from real world application it's great research there.

shruggy · 07-01-2020, 05:19 AM

Quote:

Originally Posted by tinfoil3d

Did you find all of those solutions by yourself?

No, not all of them. Some were mentioned in the comments on this StackExchange question.

BTW, what I find somewhat amusing is this 0000 in the question. I mean zero is zero is zero, be it binary, octal, decimal, duodecimal, hexadecimal, or how many times you repeat it. Specifying just 0 will clear all permission bits anyway. I'd expect a little bit more insight on the part of folks composing such questions. It feels like an example of

Code:

cat foo|grep bar|awk '{print $1}'

in a book on Unix command line tools: works, but the authors should have known better.

tinfoil3d · 07-01-2020, 05:42 AM

LOL that's a good point too. I'd simply answer reinstall the coreutils package as cpio/tar in package manager would fix all the issues or mount nfs/fuse with chmod/busybox and solve it. I wouldn't went that deep

shruggy · 07-01-2020, 08:59 AM

Well, they asked for "all possible alternative options", and the top comment at SE went on to say that there were at least six, so here we have six distinct options I could think of (or five, if "copy to another file (re)setting permissions in the process" and "copy to a filesystem that makes the problem disappear" are to be considered as two variations of the same theme).

tinfoil3d · 07-01-2020, 09:22 AM

oh okay you mentioned that too, disregard.

pan64 · 07-01-2020, 09:46 AM

looks like we do not need that chmod at all....