Attempting to access beyond end of device?

MasterC · 01-20-2003, 09:25 PM

Hello, I am not sure if this is actually a security question, however I haven't had this problem before opening my box up to the internet as a webserver/ftp server. For the last 2 or 3 mornings I've woken up to my box creaping. It's moving very slow, and there is intermittent hard drive churning, very slow. A reboot fixes it, however, here's what /var/log/messages has to say about it:

Code:

Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=883664940, limit=4192933
Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=150237860, limit=4192933
Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=150237861, limit=4192933
Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=150237861, limit=4192933
Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=150237862, limit=4192933
Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=150237862, limit=4192933
Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=150237863, limit=4192933
Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=150237863, limit=4192933
Jan 20 08:32:40 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:40 masterc kernel: 03:01: rw=0, want=150237864, limit=4192933
Jan 20 08:32:50 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:50 masterc kernel: 03:01: rw=0, want=420933836, limit=4192933
Jan 20 08:32:50 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:50 masterc kernel: 03:01: rw=0, want=420933837, limit=4192933
Jan 20 08:32:50 masterc kernel: attempt to access beyond end of device
Jan 20 08:32:50 masterc kernel: 03:01: rw=0, want=420933837, limit=4192933

This repeats for a good 1000+ messages until the reboot. Now I have found some scary info somewhere in the mix of these messages:

Code:

Jan 19 09:17:18 masterc kernel: attempt to access beyond end of device
Jan 19 09:17:18 masterc kernel: 03:01: rw=0, want=115139139, limit=4192933
Jan 19 09:17:18 masterc kernel: attempt to access beyond end of device
Jan 19 09:17:18 masterc kernel: 03:01: rw=0, want=115139140, limit=4192933
Jan 19 09:18:22 masterc sshd[713]: Accepted password for masterc from 198.97.167
.88 port 2742 ssh2
Jan 19 09:18:23 masterc sshd[714]: Could not reverse map address 198.97.167.88.
Jan 19 09:47:17 masterc -- MARK --
Jan 19 10:07:17 masterc -- MARK --
Jan 19 10:27:17 masterc -- MARK --
Jan 19 10:44:53 masterc sshd[797]: Accepted password for masterc from 198.97.167
.88 port 2265 ssh2
Jan 19 10:44:54 masterc sshd[798]: Could not reverse map address 198.97.167.88.
Jan 19 11:07:17 masterc -- MARK --
Jan 19 11:27:17 masterc -- MARK --
Jan 19 11:47:17 masterc -- MARK --
Jan 19 12:07:17 masterc -- MARK --
Jan 19 12:27:17 masterc -- MARK --
Jan 19 12:47:17 masterc -- MARK --

However, I looked back and that was me ssh'ing from work, phew!

Here's where it starts, there is really nothing indicating anything, it's just random:

Code:

Jan 19 23:47:19 masterc gconfd (masterc-1569): starting (version 1.0.9), pid 156
9 user 'masterc'
Jan 20 00:07:17 masterc -- MARK --
Jan 20 00:27:17 masterc -- MARK --
Jan 20 00:47:17 masterc -- MARK --
Jan 20 00:47:19 masterc gconfd (masterc-1569): GConf server is not in use, shutt
ing down.
Jan 20 00:47:19 masterc gconfd (masterc-1569): Exiting
Jan 20 01:07:17 masterc -- MARK --
Jan 20 01:27:17 masterc -- MARK --
Jan 20 01:47:17 masterc -- MARK --
Jan 20 02:07:17 masterc -- MARK --
Jan 20 02:27:17 masterc -- MARK --
Jan 20 02:47:17 masterc -- MARK --
Jan 20 03:07:17 masterc -- MARK --
Jan 20 03:27:17 masterc -- MARK --
Jan 20 03:47:17 masterc -- MARK --
Jan 20 04:07:17 masterc -- MARK --
Jan 20 04:27:17 masterc -- MARK --
Jan 20 04:40:26 masterc kernel: attempt to access beyond end of device
Jan 20 04:40:26 masterc kernel: 03:01: rw=0, want=1103013496, limit=4192933
Jan 20 04:40:26 masterc kernel: attempt to access beyond end of device

I am wondering if this would have anything to do with ssh?

Anyway, if this is not security related, I'd appreciate it being moved to the correct forum and thank you for your time

If anyone has any suggestions or wants me to post other portions of my logs just let me know, I'll be happy to show you.

Thanks again

unSpawn · 01-21-2003, 02:49 AM

some_sw: gimme the nfo!
your_disk: k, it's indexed to be at 1103013496, want it?
your_disk: k, let's see 4192930, let's see 4192931, let's see 4192932, let's see 4192933, (...), 1103013496. 1103013496? I ain't got no steenkin 1103013496! Everything after 4192933 is like, radically bogoid, dude!

Did you run fsck?
// I can move this to general or hardware, whatever you want.

Grim Reaper · 01-21-2003, 02:51 AM

OMG! Somebody else has gotten this problem also! Hehe, I guess that isn't such a _good_ thing for you, but now maybe we'll have more of a chance to get it sorted

I've had this problem also, but it didn't have anything to do with SSH, etc...it was to do with Wine...I'd put my fake C-Drive on a 10 gig HDD, but eventually I'd get those same errors as you, then i noticed files disappearing...not too many, but enough to make games start cracking the shits...

But yeah, they are the same errors i was getting with wine...remember me mentioning something about it corrupting the filesystem, well, thats what it was doing...dunno if it was Wine causing it or not though, i never did find out, i just gave up...

MasterC · 01-21-2003, 09:29 AM

Quote:

Originally posted by unSpawn
some_sw: gimme the nfo!
your_disk: k, it's indexed to be at 1103013496, want it?
your_disk: k, let's see 4192930, let's see 4192931, let's see 4192932, let's see 4192933, (...), 1103013496. 1103013496? I ain't got no steenkin 1103013496! Everything after 4192933 is like, radically bogoid, dude!

Did you run fsck?
// I can move this to general or hardware, whatever you want.

Thanks for the reply unSpawn, yeah from the sounds of it, and Grim's reply, I'd think it's a hardware issue rather than an attempted attack on my system. If you can move it over there, that'd be great thank you.

Some more info:

I got home today and the same thing, it appears to be able to be fixed by dropping back to runlevel 1 and then switching back up to 3. I am also venturing out tonight to see if it's just a filesystem problem, so I am gonna umount each partition, 1 at a time, each night to see if that's that problem. Grim, your response plus there seemed to be something mentioning vfat or something when I dropped back to runlevel 1, so I will try that first.

Thanks

Grim Reaper · 01-21-2003, 09:36 AM

Cool, tell me how ya go

Mik · 01-22-2003, 04:46 AM

Well before you go trying each partition the message indicates this:
kernel: 03:01: rw=0, want=420933837, limit=4192933

03:01 would be major number 3 and minor number 1
If you look through the major numbers and minor numbers of the /dev files that should be /dev/hda1
So that's the partition it's trying to access.

Mik · 01-22-2003, 04:54 AM

Just another thing, next time it happens. Instead of switching to runlevel 1 why don't you try shooting down your processes one by one and see if you can find the culprit that way. Or try to see with a ps listing or top which process is taking up all your resources.

MasterC · 01-23-2003, 01:42 AM

I will Mik, thanks for the tip off on the device!

I have a hard time doing anything once it gets into this problem though. The system literally creeps almost to a halt. Killing X takes a good 5 minutes to complete. Then to su - into root takes another good 10 minutes. By the time init 1 begins it's been ~30 minutes, no exageration. But I would like to pin down what's going on, so I will try a few processes 1 by 1 at least, in a hopeful attempt to hit in on one of those. I tried top, but must not have given enough time, because after about 10 minutes I figured the system had just creeped to a halt.

If it starts up again tonight, I'll have more time tomorrow to pin it down.

Thanks again for the tips.

Cool

MasterC · 01-23-2003, 01:22 PM

Woohoo! Mik, you are the man

It's updatedb giving me problems. I think it's with my mounted /dev/hda1 because I tried umounting it and it would just throw up "device busy" and return me to a prompt.

Then I ran top. At the top of the list updatedb had been running for something like 315 minutes, so I figure that's gotta be it. I kill -15 it's pid and it's back to normal!

So now at least we've identified the problem (maybe) but what to do for the solution? I know I could just umount /dev/hda1 when it's not in use, it's just a win partition anyway and actually I don't think I've actually used it forever.

Woohoo!

Thanks

Mik · 01-24-2003, 03:30 AM

Well if you don't use the partition then you could probably unmount it. A nicer solution might be to prevent updatedb from searching any windows filesystems.
If you read through the man pages of updatedb the default would be to ignore nfs and proc. You could add vfat or dosfs or whatever it's called by using --prunefs or setting the PRUNEFS environment variable. Something like:

updatedb --prunefs="nfs proc dosfs"

Hmm weird I just tried that on a different system but it doesn't have the --prunefs, I guess it's a different version of updatedb. But you could still always use the --prunepaths switch.

Grim Reaper · 01-29-2003, 12:55 AM

w00t!

How is updatedb being run anyway? It can't be set for a cron job because I don't think i have that enabled/installed...

ummm, if you could work out that way to stop it from reading vfat/ntfs partitions that would be unreal...

So does updatedb --prunepaths /files /windows

would that stop updatedb from reading those directories/partitions (because thats where their mounted to..)

Mik · 01-29-2003, 03:43 AM

A lot of distro's do set updatedb to be run as a cron job. I don't know about slackware. But I'm pretty sure slackware would at least include cron.

According to the man pages the syntax would have to be:
updatedb --prunepaths='/files /windows'

MasterC · 01-29-2003, 04:22 AM

Check your /etc/cron.daily folder. There is slocate in mine

(Slack 8.1) It updates once daily, which to me is a good thing. You can change that if you'd like, or remove it completely, or as Mik as shown, have specific folders not included.

BTW, it's been a while now, and working great. Thanks again Mik

Cool

Grim Reaper · 01-29-2003, 05:57 AM

Mad, I'll take a squiz.

Thank you two HEAPPPPS!

That'll stop a few problems! w00t! On with WineX!