My Debian etch system (kernel 2.6.15-1-686-smp) stalls indefinitely during boot. I can press control-c to end the stall. A lot of udevd_event messages flood the screen before and during the stall.

I've tried booting the kernel in recovery mode, but it still hangs at the same point.

Nothing out of the ordinary is logged in dmesg or /var/log/messages.

Here's a chart of boot up processes:
http://67.11.222.152/tmp/bootchart.svg

I think sh and udevd might be involved, since I saw the following message at boot:
I've searched google and came up with nothing. It looks like udevd might be trying to run a shell script, but it fails.

Any ideas what to try next?

Try Sarge. Etch is Testing, Sarge is Stable. If Sarge fails it's probably your hardware, if Sarge succeeds Etch is at fault

That's not really an option. Can anybody help me troubleshoot this in Etch?

Why is it not an option? What's your alternative? What's the use of a system with the "latest and greatest" software that doesn't boot? Well, this is Linux: it's your choise!

Did it boot correctly before or does this problem occur from the first reboot after installation?

Has your network info changed? If so, change it back and see if that does it.

Thanks for the replies.


Tons of Fun,
My network info hasn't changed. The network configuration happens after the stall, and without problems.


Dutch Master,
Because it's not possible to go from testing to stable without reinstalling the system. And since Etch is scheduled to go to stable in a couple of months, it doesn't seem like it's worth the trouble to completely reinstall my system.

I don't think it always did this during boot. I think it happened after I configured udev to work with automount using the following guides:
http://www.debian-administration.org/articles/126
http://www.debian-administration.org/articles/127

...but I can't be sure since I leave my computer running for so long between reboots. Is there any indication of what's wrong from looking at the bootchart?

DutchMaster is a very smart guy, but he's hung up on Sarge as a distro that people should be using. I think Etch is much more satisfactory for most people.

Did you upgrade from Sarge to Etch or do a fresh install?
Did you install udev as part of that tutorial you were following, or was it installed by Etch?
What Kernel are you using?

Ok, I see your point. But, as Etch still takes a few months to become stable you're stuck with an unusable system during that time. As I see it, you have three options:
a) re-install everything (either Sarge or Etch) and forget about udev for the moment
b) uninstall udev from your system, but that might be a PITA (pain in the *ss ) and might take you more time then re-installing the lot
c) use a rescue CD like Knoppix to try to get rid of udev, but I doubt if that will work: dunno if apt/dpkg/aptitude will accept a different target, i.e. your harddrive, to it's own system. AFAIK it won't...

I'm afraid that dmesg won't be of any help, it's data is lost/overwritten with every reboot and I'm not sure you can redirect it's output to a file during the bootprocess. I don't know enough about udev to be of any help there I'm afraid... One more question though: did you set up the automount stuff on udev from the last article you've mentioned? Or did udev go nuts right after you've installed it?

<edit>Rickh: you might be right. But Etch is still in Testing, which means that it can and at some time will be broken! It's less likely to happen comparing with Sid/Unstable, but the stable branch provides, well, just that: a stable distro. But maybe I'm biased as I've got an unusable Etch system before when using an older net-install CD...</edit>

rickh,

I did a fresh install of Etch. (I actually meant to install Sarge, but I grabbed the wrong net-install disc. I decided to stay with Etch after seeing how stable it was.)

I installed udev manually.

I'm using kernel 2.6.15-1-686-smp.


Dutch Master,

I'm very against your first suggestion, to reinstall everything. It's my personal preference to figure out the cause of problems, rather than reinstall as a Windows user would.

About your second and third suggestions, I read that "Etch 2.6 kernels no longer support devfs." http://www.gatago.com/linux/debian/doc/14621687.html

I installed udev and automount several months ago, so it's possible that before I manually installed udev, the Etch kernel supported devfs, but now it doesn't.

I did set up automount from that article. I'm not sure exactly when udev went nuts, because I leave my system running for so long between reboots.


Thanks again for helping me troubleshoot this.

Edit: Of course if you can't boot the system at all this is going to be a problem. Have you tried booting into single user mode?


Shot in the dark: Try it anyway.

dpkg -P hotplug
dpkg-reconfigure linux-image-`uname -r`

Also it probably wouldn't hurt you to get a newer kernel. Udev was still giving intermittent problems until sometime in the 2.6.16 series. Do the above first, tho

I understand your view on re-installing an OS and frankly, I agree! But at some point you must be pragmatic about it: how much time does it take to re-install your system compared to solving the problem by re-tracing your steps (especially as you have such long intervals between reboots) and studying man-pages and such. Ok, I admit that rolling your system back helps you learn a lot on how things work, but is that the purpose of a work-station? (I presume you actually use that machine to do something other then playing Patience )

IMHO a computer is an aid for a human to help him/her to do certain tasks more efficiently or without human intervention, not to spend weeks of tinkering and still it doesn't do what you want it to do... But that's just my

Dutch Master,
I was reluctant to say it, because I knew I'd be scolded for not using Debian stable, but this is a server. So reinstalling isn't an option. I'd have a LOT to reconfigure if I reinstalled this system.

rickh,
I tried your suggestion, and I still have the same exact problem.

I was really surprised that I didn't need hotplug. Does udev replace it?

I took a picture of the screen during the stall. Please look at it to see if anything jumps out at you:
http://67.11.222.152/tmp/udev-boot-messages.jpg

Yeah. udev replaces hotplug.

I don't know what to say. I would still try upgrading the kernel. With any kind of luck the kernel image reconfigure might have cleared up whatever problem you were having there, and it would probably also upgrade udev.

If that doesn't work, a reinstall is quick if you have your /home on a separate partition. While partitioning, just don't reformat /home. You'll get a few warnings about new users being assigned to groups they don't own, but it's insignifficant and irons itself out quickly. I break mine pretty regularly, probably wind up reinstalling every 6 months or so, but it's a 45 minute process to get it running again. Keeps you busy tweaking for a while, but that's the fun part.

here's what finally fixed my problem (this is from a #debian irc bot):
thanks again for all the help!