First off, whether it is related to my login issue or not, shame on me for letting the / volume become 100% full in the first place. Duly noted, so let's move on.

As the topic states, my / volume filled to 100% capacity while installing a bunch of updates, including a kernel update. The machine is a laptop and I was about to change locations so I shut it down and ever since I have been unable to log in after a normal start up.

The system starts normally (as far as I can tell) but when I enter my username and password at the GUI login screen it throws an invalid password error even though I know the password is correct. If I switch to one of the TTYs and try to log in, the system throws an 'Permission denied' error immediately after entering my username (i.e., I am not even prompted for a password) and returns to the main login prompt. This behaviour is consistent regardless of whether the username entered exists on the system.

I am able to boot into recovery mode and through the recovery mode console I have been able to free several hundred MB off of the / volume by removing older kernels. The volume is no longer starved for space but the login issue persists.

Also using the recovery console I have confirmed that the password is correct by using su to switch to my own account and then running passwd. When passwd prompts me for my current password and I enter it, it proceeds to the next step as expected. When I enter the same password for both New and Verify, the application reports that the password is unchanged, as expected.

I created a new user via the recovery console and the process returns an error, although the user appears to be created normally:

I have searched the internet but most login failure topics seem to deal with desktop sessions that start to load before kicking the user back to the login screen which is considerably different behaviour than I am experiencing.

Since I can only get to a text prompt on the affected machine I have limited access to logs but I have attached the auth.log excerpt that I believe shows my last successful login as well as one from what I believe was the first failed login attempt after the issue presented itself.

I am happy to pull whatever logs might help off of the machine but I am not sure where to start and posting dozens of logs seemed counter-productive.

All of my internet searches thus far have turned up nothing useful and I am at a loss as to how to proceed from here, short of just reinstalling the OS. Any and all suggestions and help are welcome and appreciated.

Attached Files

	auth.failure.log (2.1 KB, 13 views)
	auth.success.log (3.8 KB, 8 views)

In your predicament I would boot off of a Live CD/DVD/USB, grab my data off the drive and then do a full re-install.

The concern is you were halfway through an upgrade so who knows what the state of your system really is.

Moved: This thread is more suitable in <Ubuntu> and has been moved accordingly to help your thread/question get the exposure it deserves.

Booting to a live session and backing up would be a great idea.

Your system, however, should be recoverable. You will need, while in the Live Session to chroot in, clean things up so you have some space, run;
which should sort out a lot of the mess. Would be surprised it got it all. In that case run;
That is the command to fix broken packages and should sort it out.

Any errors after that would best be handled with those same commands but booted to recovery mode.

People wonder why I like big / partitions. This is part of it. I tend to leave /var/cache/apt/archieves alone for a long time. I kind of like to see the history of my system and it is laid out there pretty well. Takes up a lot of space though.

1 members found this post helpful.

Thank you rtmistler and widget for your feedback.

Turns out that the PAM module was messed up, presumably during the failed update install process when I ran out of space.

Here's a breakdown of what I ran through to resolve the issue.

As I mentioned previously, I was able to clean up space on the volume by deleting old kernels. This should have been a straight-forward process but having the system in an inconsistent state caused problems. I couldn't do anything with apt-get or aptitude because one of my installed packages was missing a dependency (kernel update) and there was no space left on the volume to install the missing package. I was finally able to work around this by running the following in the recovery console with networking disabled:

This allowed me to remove versions 3.13.0-32, -35 and -36 of linux-headers, linux-image and linux-image-extra, as well as their generic counterparts. The final argument was required to trick apt-get into thinking I wanted to also install the package that was a missing dependency and the --ignore-missing switch allowed apt-get to proceed to with the removal of the packages I wanted to remove even though the installation failed due to lack of connectivity to the repository.

With almost a gig of space freed by removing the old kernels, I was able to enable networking and fix my broken dependencies.

Once all of that was done I rebooted but found no change in my ability to log in. I was now able to boot successfully to the -39 kernel (prior to the fix a normal boot to -39 failed, for understandable reasons) but I was still getting password errors when trying to log in to my desktop.

I searched the internet some more and found a reference to the "sanity check" application debsums. I installed it (via recovery console) and scanned for hash mismatches.

This returned a single file so I used dpkg to figure out which package that file belonged to (dpkg itself) and the reinstalled that package with aptitude:

After this I confirmed a clean bill of health for both the system files and the config files using debsums and rebooted again; still no joy.

At this point I started to wonder if there was an issue with PAM since everything except authentication seemed to be working normally. I am not terribly familiar with how PAM works, but it seemed like reinstalling the PAM module(s) was a good place to start.

I used dpkg to find a list of packages that I thought might be related to PAM and then reinstalled one of them with aptitude.

As part of the post-package install process, aptitude reported a failure with pam-auth-update due to changes in the the local common files (/etc/pam.d/common-*).

I couldn't think of anything that I had done that would have modified those files so I backed them up and then ran:

I rebooted one last time and confirmed that, at last, I was able to log in successfully.

With the system apparently back in working order, I diffed the contents of /etc/pam.d/common-* against my backups and found that common-account and common-auth had both been blanked out somehow; the backup files were completely empty. I suspect the disk ran out of space while these files were being updated by the initial update process that filled up my / volume and, as a result, the were replaced with blank files. Manually running pam-auth-update restored the files to their default state and now things seem to be working normally again.