Check package integrity ?

Pigi_102 · 07-20-2018, 02:51 PM

Hi all,
I suffered a moderate problem on my pc that lead to some filesystem corruption.
I did the usually things, ext2fsck and so on and ended up with a bunch of files in lost-found.
Fortunately the machine boot no problem and all important stuf runs, but I would like to check and eventually fix the broken packages.

I can't go checking every single file ( they are 1543 ) so my options are to reinstall or check for every package if some files are missing.
I would like to go for the second idea ( checking for files missing in packages and eventually reinstall or remove if not needed ) so I was asking: is there any script or similar that check for packages with missing files ?

Thanks in advance

Pigi

bassmadrigal · 07-20-2018, 03:18 PM

It's probable that someone else has already created this, but if it isn't posted, I think the code below will do it for you. This should run through all files listed in the various /var/log/packages directory and check each file in there to see if it exists. It does ignore .new files since they were likely already handled and we don't need the false positives.

NOTE: I can't currently check that this works (at work without access to my machine), but it is not damaging in any way, so it couldn't hurt to try.

Code:

for i in $(find /var/log/packages); do
  PKGNAM=$(basename $i)
  for j in $(cat /var/log/packages/$PKGNAM); do
    if [ ! -f /$j ] && [ ${j:${#j}-4} == ".new" ]; then
      echo "/$j in package $PKGNAM can't be found"
    fi
  done
done

Darth Vader · 07-20-2018, 03:29 PM

The OP asks of something like "repair installation", similar with the one which is used by Windows and some Windows software?

I am afraid that even RedHat Enterprise Linux does not offer such feature, and it is not possible to have that unless you store checksums for every file and a way to retrieve the original package and partially extract the offended files. For example, can happen to have ZERO length or corrupted files in a corrupted filesystem.

That's really, but really a very long shoot above what the Slackware packages management can offer.

Long story short, I guess that a Linux administrator concerned about the system integrity should reel into backups, even if s/he use RHEL. Always.

No backups? That's really pretty bad, BUT a clean re-install is always at your hand.

OR, in the Slackware case you can even try to do on a updated local mirror:

Code:

upgradepkg --reinstall --install-new slackware/*/*.t?z

That will do the trick, with some patience.

bassmadrigal · 07-20-2018, 04:20 PM

Quote:

Originally Posted by Darth Vader

The OP asks of something like "repair installation", similar with the one which is used by Windows and some Windows software?

If you read more than the title, you'd realize OP was simply looking for something to verify all files in a package exist on the system, not to verify whether the files are corrupt.

Darth Vader · 07-20-2018, 08:06 PM

Yes, I read the original post - but is not enough that checking if a file from a particular package exists in a corrupted filesystem.

From my own experience, that file can exists, but it can be corrupted too.

For example, by being empty and its expected contents being saved instead on a series of files on lost+found. Happened to see with my own eyes this phenomena.

The final goal will be probably repairing this filesystem, and that can be done under Linux only by either restoring from a backup or re-installing the operating system.

glorsplitz · 07-20-2018, 09:15 PM

Quote:

Originally Posted by Pigi_102

but I would like to check and eventually fix the broken packages.

So just go about using your system as you normally would and if you run into a broken package reinstall it.

upnort · 07-20-2018, 10:16 PM

@Pigi_102:

I found the following:

Looking for missing files from packages. The script complains about missing *.new files, but otherwise should help you.

https://www.linuxquestions.org/quest...0/#post1927061

smprms will ensure file permissions are correct. You might want to read the entire thread rather than blindly executing the smprms script.

https://www.linuxquestions.org/quest...1/#post1741073

Edit: Would be an interesting project if some wizard combined both scripts to create a nice recovery tool.

mralk3 · 07-20-2018, 10:19 PM

Quote:

Originally Posted by glorsplitz

So just go about using your system as you normally would and if you run into a broken package reinstall it.

Or to save yourself that hassle you could reinstall your entire system using the following commands with slackpkg:

Code:

slackpkg update gpg
slackpkg update
slackpkg reinstall slackware # For x86 and ARM
slackpkg reinstall slackware64 # For x86_64
slackpkg clean-system
slackpkg new-config

If you have packages from SBo, just issue a full rebuild of everything to be safe. That is trivial if you use sbopkg, first rebuild all queue files for automation. Then run the sbopkg console GUI interface:

Code:

sqg -a
sbopkg

Download your repository again. Then go to -> Queue -> Add - All installed packages to the queue -> Process -> Ok -> Install - Download, Build, and Install -> Start.

This will assure all software sources and packages are re-verified upon installation using the .asc key files and that all possibly corrupt files have their integrity.

If you have a back up of your media files, documents, and dot files, it's best to restore those since you can't be sure about their integrity.

TommyC7 · 07-21-2018, 02:17 AM

bassmadrigal's solution works well for finding if a file exists on your system or not.

However, as previously mentioned, even if the file exists it's not necessarily good and it may even Schroedingly-exist (exist and not exist at the same time) if it's corrupt.

Without a hash sum of the files (or something similar), offline-wise, there's no way to verify the files on your system.

However, if downloading the packages is not a problem you can always download the packages and check the files and replace appropriately (although at that point I'd probably just reinstall the whole package or all the packages).

FlinchX · 07-21-2018, 06:57 AM

Quote:

Originally Posted by TommyC7

[b]Without a hash sum of the files (or something similar), offline-wise, there's no way to verify the files on your system.

this. it's a good motivation to start using a file integrity checker like aide (http://slackbuilds.org/repository/14.2/system/aide/) on a regular basis

Darth Vader · 07-21-2018, 07:04 AM

Use the Backup, Luke!

Pigi_102 · 07-22-2018, 03:19 AM

Sorry for the late answer but while trying to find a solution the filesystem "broke" again

The solution to reinstall wasn't working, as the wget was missing ( The ext4 failed again while reinstalling the wget packages, btw ).

Thus I decided to buy a new sd card ( the machine is a raspberry ) and reinstall, but it would be nice to have such utility.
It's not ( completely ) important to check if the file is fine, but at least to know which is missing.

The bash script posted in the first answer wasn't working.

I've tried to fix and eventually minimized the problems in this way:

Code:

#!/bin/bash 

for i in $(find /var/log/packages/* | grep -v devs); do
  PKGNAM=$(basename $i)
  for j in $(tail -n +19 /var/log/packages/$PKGNAM | egrep -v ".new$x|/$|slack-desc|doinst.sh" ); do
    if [ ! -f /$j ]; then
      echo "/$j in package $PKGNAM can't be found"
    fi
  done
done

I still have some trouble with all those files that has a blank in it ( for example "/usr/share/cmake-3.11/Help/generator/Visual\ Studio\ 10\ 2010.rst" ) as those get expanded in multiple variables at the "for J in " line, but I will try to get fixed later in the future.

Pigi_102 · 03-04-2019, 07:13 AM

Here the script that takes also care of files with a space in it, if interessed

Code:

#!/bin/bash 

SAVEIFS=$IFS
IFS=$(echo -en "\n\b")

for i in $(find /var/log/packages/* | grep -v devs); do
  PKGNAM=$(basename $i)
  #for j in $(tail -n +19 /var/log/packages/$PKGNAM | grep -v ".new$" | grep -v "/$" | grep -v "slack-desc" | grep -v "doinst.sh" ); do
  for j in $(tail -n +19 /var/log/packages/$PKGNAM | egrep -v ".new$x|/$|slack-desc|doinst.sh" ); do
    if [ ! -f "/$j" ]; then
      echo "/$j in package $PKGNAM can't be found"
    fi
  done
done

Pigi