LinuxQuestions.org - Corrupt RPM database

Hi,

I have RedHat 9.0

Version: psa v6.0.2_build031023.13 os_RedHat 9.0
OS: Linux 2.4.20-30.9INLDSmpIPvsDs

My rpm-database is broken.
Things like querying work fine, but installation and upgrade are impossible.

===============
Example 1:
[root@ns downloads]# rpm -Uvh jhead-2.4-2.i386.rpm
error: cannot open Packages index using db3 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
===============

===============
Example 2:
[root@ns downloads]# yum update
Gathering header information file(s) from server(s)
Server: Atomic Rocket Turtle - 9 - Atomic PSA
Server: Atomic Rocket Turtle - 9 - Atomic PSA 6.0
Server: Red Hat Linux 9 base
Server: Fedora Legacy utilities for Red Hat Linux 9
Server: Red Hat Linux 9 updates
Finding updated packages
Downloading needed headers
Resolving dependencies
error: db4 error(13) from db->close: Permission denied
error: db4 error(13) from db->close: Permission denied
Dependencies resolved
I will do the following:
[update: pam-devel 0.75-62.10.legacy.i386]
[update: nfs-utils 1.0.1-3.9.1.legacy.i386]
[update: libtiff 3.5.7-11.2.legacy.i386]
[update: sudo 1.6.6-3.2.legacy.i386]
[update: cvs 1.11.2-25.legacy.i386]
[update: qt 1:3.1.1-8.legacy.i386]
[update: pam 0.75-62.10.legacy.i386]
Is this ok [y/N]: y
Downloading Packages
Running test transaction:
error: db4 error(13) from db->close: Permission denied
error: db4 error(13) from db->close: Permission denied
Test transaction complete, Success!
error: db4 error(13) from db->close: Permission denied
error: db4 error(13) from db->close: Permission denied
error: cannot open Packages index using db3 - Permission denied (13)
error: cannot open Packages database in /var/lib/rpm
Updated: pam-devel 0.75-62.10.legacy.i386 nfs-utils
1.0.1-3.9.1.legacy.i386 libtiff 3.5.7-11.2.legacy.i386 sudo
1.6.6-3.2.legacy.i386 cvs 1.11.2-25.legacy.i386 qt
1:3.1.1-8.legacy.i386 pam 0.75-62.10.legacy.i386
Transaction(s) Complete
[root@ns downloads]#
==================

If I run
rpm -qa
there is no db3* listings..

these are some of the possible relative results:

db4-devel-4.0.14-20
db4-4.0.14-20
db4-utils-4.0.14-20

rpm-4.2-0.69
rpm-devel-4.2-0.69
rpm-build-4.2-0.69
rpm-python-4.2-0.69

yum-2.0.7-1.rh90.art
up2date-3.1.23.2-1RedHat9Stocki386.2

mysql-4.0.21-2.rh90.art

These are the things I have tried to fix it:
==============================================

First I backed up

[root@ns root]# cd /var/lib
[root@ns root]# mkdir rpm-backup
[root@ns root]# rsync -av ./rpm/. ./rpm-backup/.

Second, remove the files which hold lock state information

[root@ns root]# rm -f /var/lib/rpm/__db*
[root@ns root]# nl /etc/rc.d/rc.sysinit | grep rpm
556 rm -f /var/lib/rpm/__db*

Third, I then ran:

[root@ns root]# rpm -vv --rebuilddb

It created a temporary directory to work in at: /var/lib/,
next to the 'parent' one at: /var/lib/rpm/ and attempts a rebuild.
As it gets to the very bottom of the rebuild, it attempted to replace
the new content over top of the old files.

It failed to replace old database with new database.
Thus it also failed to remove directory /var/lib/rpmrebuilddb.4979:
Stating: Directory not empty

this is what I got at the end of the process...

D: closed db index /var/lib/rpm/Packages
D: closed db environment /var/lib/rpm/Packages
D: removed db environment /var/lib/rpm/Packages
D: closed db index /var/lib/rpmrebuilddb.4979/Pubkeys
D: closed db index /var/lib/rpmrebuilddb.4979/Filemd5s
D: closed db index /var/lib/rpmrebuilddb.4979/Sha1header
D: closed db index /var/lib/rpmrebuilddb.4979/Sigmd5
D: closed db index /var/lib/rpmrebuilddb.4979/Installtid
D: closed db index /var/lib/rpmrebuilddb.4979/Provideversion
D: closed db index /var/lib/rpmrebuilddb.4979/Requireversion
D: closed db index /var/lib/rpmrebuilddb.4979/Dirnames
D: closed db index /var/lib/rpmrebuilddb.4979/Triggername
D: closed db index /var/lib/rpmrebuilddb.4979/Conflictname
D: closed db index /var/lib/rpmrebuilddb.4979/Providename
D: closed db index /var/lib/rpmrebuilddb.4979/Requirename
D: closed db index /var/lib/rpmrebuilddb.4979/Group
D: closed db index /var/lib/rpmrebuilddb.4979/Basenames
D: closed db index /var/lib/rpmrebuilddb.4979/Name
D: closed db index /var/lib/rpmrebuilddb.4979/Packages
D: closed db environment /var/lib/rpmrebuilddb.4979/Packages
D: removed db environment /var/lib/rpmrebuilddb.4979/Packages
error: failed to replace old database with new database!
error: replace files in /var/lib/rpm with files from
/var/lib/rpmrebuilddb.4979 to recoverD: removing directory
/var/lib/rpmrebuilddb.4979
error: failed to remove directory /var/lib/rpmrebuilddb.4979:
Directory not empty
D: closed db index /var/lib/rpm/Pubkeys
D: closed db index /var/lib/rpm/Packages
D: closed db environment /var/lib/rpm/Packages

Forth, I ran a check

[root@ns root]# cd /var/lib
[root@ns root]# du | sort -n

this is part of the result:

26588 ./rpmrebuilddb.4979
26588 ./rpmrebuilddb.9956
28636 ./rpm
28932 ./rpm-backup

I also ran df to check for a space issue..

[root@ns lib]# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/hda2 117575692 17061596 94541532 16% /
/dev/hda1 124427 8944 109059 8% /boot
none 511020 0 511020 0% /dev/shm

I then tried:

[root@ns lib]# cd /var/lib/rpm
[root@ns rpm]# /usr/lib/rpm/rpmdb_verify Packages
[root@ns rpm]#

as there was no result... I presume all is ok with the rpmdb_verify
Packages ..??

but if I run

[root@ns rpm]# db_verify Packages
db_verify: Old or incorrect DB version; extraneous errors may result
db_verify: DB->verify: Packages: DB_VERIFY_BAD: Database verification failed

If I run

[root@ns rpm]# rpm -Va

The only potential issue I saw was:
Unsatisfied dependencies for mysql-server-4.0.20-art.2: mysql = 4.0.20

I get loads of missing perl...

missing /usr/lib/perl5/site_perl/5.8.0/......
missing /usr/lib/perl5/vendor_perl/5.8.0/......

yet...

[root@ns rpm]# perl -v

This is perl, v5.8.0 built for i686-linux
This I beleive was a hangover from attempting to install interchange last year.

Summary:
I am not sure on how to sort out this issue...
I hope one of you can help!