LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 07-22-2008, 01:28 PM   #16
userlander
Member
 
Registered: Jul 2008
Distribution: Arch, Debian
Posts: 61

Original Poster
Rep: Reputation: 18

Quote:
Originally Posted by Mr. C. View Post
But you *do* have od on other *nix systems. Transfer the log file! This is what I mean by "Your touching the data - don't walk away". You've walked away again, without an answer.

You have ot make up your mind. Either you want to diagnose the possible problem, or you don't. I hear "HELP" in your Subject, but hear "meehhh, I'll speculate instead" in your responses.

I don't know what you mean about coincidence regarding the graphic; it sounds like you're saying malicious attempts don't come from random IP addresses. This theory of yours is wrong. In fact, that's exactly the pattern seen when a system has been compromised in some way.

I won't speculate any longer here. You have the data, examine it or don't. Is your call.
I'm getting so flustered I didn't even think to transfer the file, how obvious. :-o

Okay, this is what I got, but it means nothing to me! What do I do with this data??

Code:
 $ cat messages |grep eteth0 |od -bc
0000000 112 165 154 040 062 061 040 061 066 072 060 061 072 063 070 040
          J   u   l       2   1       1   6   :   0   1   :   3   8    
0000020 154 157 165 040 153 145 162 156 145 154 072 040 155 157 145 040
          l   o   u       k   e   r   n   e   l   :       m   o   e    
0000040 143 150 145 143 153 111 116 075 145 164 145 164 150 060 040 117
          c   h   e   c   k   I   N   =   e   t   e   t   h   0       O
0000060 125 124 075 145 164 150 061 040 123 122 103 075 061 071 062 056
          U   T   =   e   t   h   1       S   R   C   =   1   9   2   .
0000100 061 066 070 056 060 056 061 060 063 040 104 123 124 075 070 060
          1   6   8   .   0   .   1   1   3       D   S   T   =   8   0
0000120 056 071 062 056 061 060 061 056 062 060 071 040 114 105 116 075
          .   9   2   .   1   0   1   .   2   0   9       L   E   N   =
0000140 061 065 060 060 040 124 117 123 075 060 170 060 060 040 120 122
          1   5   0   0       T   O   S   =   0   x   0   0       P   R
0000160 105 103 075 060 170 060 060 040 124 124 114 075 066 063 040 111
          E   C   =   0   x   0   0       T   T   L   =   6   3       I
0000200 104 075 063 063 061 070 063 040 104 106 040 120 122 117 124 117
          D   =   3   3   1   8   3       D   F       P   R   O   T   O
0000220 075 124 103 120 040 123 120 124 075 070 060 040 104 120 124 075
          =   T   C   P       S   P   T   =   8   0       D   P   T   =
0000240 063 063 070 067 071 040 127 111 116 104 117 127 075 061 060 070
          3   3   8   7   9       W   I   N   D   O   W   =   1   0   8
0000260 040 122 105 123 075 060 170 060 060 040 101 103 113 040 125 122
              R   E   S   =   0   x   0   0       A   C   K       U   R
0000300 107 120 075 060 040 012
          G   P   =   0      \n
0000306


---------------> here's also eth00, eth0IN, and et0, in case those tell anything eteth0 doesn't show. I hope those \n characters don't mean non-printing ones.

Code:
$ cat messages |grep eth00 |od -bc
0000000 112 165 154 040 062 061 040 061 063 072 065 062 072 062 067 040
          J   u   l       2   1       1   3   :   5   2   :   2   7    
0000020 154 157 165 040 153 145 162 156 145 154 072 040 155 157 145 040
          l   o   u       k   e   r   n   e   l   :       m   o   e    
0000040 143 150 145 143 153 111 116 075 145 164 150 060 040 117 125 124
          c   h   e   c   k   I   N   =   e   t   h   0       O   U   T
0000060 075 145 164 150 060 060 040 101 103 113 040 125 122 107 120 075
          =   e   t   h   0   0       A   C   K       U   R   G   P   =
0000100 060 040 012 112 165 154 040 062 061 040 061 066 072 061 064 072
          0      \n   J   u   l       2   1       1   6   :   1   4   :
0000120 061 065 040 154 157 165 040 153 145 162 156 145 154 072 040 155
          1   5       l   o   u       k   e   r   n   e   l   :       m
0000140 157 145 040 143 150 145 143 153 111 116 075 145 164 150 060 060
          o   e       c   h   e   c   k   I   N   =   e   t   h   0   0
0000160 040 117 125 124 075 145 164 150 061 040 123 122 103 075 061 071
              O   U   T   =   e   t   h   1       S   R   C   =   1   9
0000200 062 056 061 066 070 056 060 056 061 060 063 040 104 123 124 075
          2   .   1   6   8   .   0   .   1   1   3       D   S   T   =
0000220 066 070 056 061 061 071 056 061 071 067 056 061 067 065 040 114
          6   8   .   1   1   9   .   1   9   7   .   1   7   5       L
0000240 105 116 075 061 065 060 060 040 124 117 123 075 060 170 060 060
          E   N   =   1   5   0   0       T   O   S   =   0   x   0   0
0000260 040 120 122 105 103 075 060 170 060 060 040 124 124 114 075 066
              P   R   E   C   =   0   x   0   0       T   T   L   =   6
0000300 063 040 111 104 075 063 060 061 061 062 040 104 106 040 120 122
          3       I   D   =   3   0   1   1   2       D   F       P   R
0000320 117 124 117 075 124 103 120 040 123 120 124 075 070 060 040 104
          O   T   O   =   T   C   P       S   P   T   =   8   0       D
0000340 120 124 075 065 065 070 064 061 040 127 111 116 104 117 127 075
          P   T   =   5   5   8   4   1       W   I   N   D   O   W   =
0000360 061 060 070 040 122 105 123 075 060 170 060 060 040 101 103 113
          1   0   8       R   E   S   =   0   x   0   0       A   C   K
0000400 040 125 122 107 120 075 060 040 012
              U   R   G   P   =   0      \n
0000411

$ cat messages |grep eth0IN |od -bc
0000000 112 165 154 040 062 061 040 061 062 072 064 063 072 063 061 040
          J   u   l       2   1       1   2   :   4   3   :   3   1    
0000020 154 157 165 040 153 145 162 156 145 154 072 040 155 157 145 040
          l   o   u       k   e   r   n   e   l   :       m   o   e    
0000040 143 150 145 143 153 111 116 075 145 164 150 060 111 116 075 145
          c   h   e   c   k   I   N   =   e   t   h   0   I   N   =   e
0000060 164 150 060 040 117 125 124 075 145 164 150 061 040 123 122 103
          t   h   0       O   U   T   =   e   t   h   1       S   R   C
0000100 075 061 071 062 056 061 066 070 056 060 056 061 060 063 040 104
          =   1   9   2   .   1   6   8   .   0   .   1   1   3       D
0000120 123 124 075 070 070 056 066 064 056 065 060 056 061 071 066 040
          S   T   =   8   8   .   6   4   .   5   0   .   1   9   6    
0000140 114 105 116 075 061 064 071 062 040 124 117 123 075 060 170 060
          L   E   N   =   1   4   9   2       T   O   S   =   0   x   0
0000160 060 040 120 122 105 103 075 060 170 060 060 040 124 124 114 075
          0       P   R   E   C   =   0   x   0   0       T   T   L   =
0000200 066 063 040 111 104 075 061 065 063 040 104 106 040 120 122 117
          6   3       I   D   =   1   5   3       D   F       P   R   O
0000220 124 117 075 124 103 120 040 123 120 124 075 070 060 040 104 120
          T   O   =   T   C   P       S   P   T   =   8   0       D   P
0000240 124 075 061 067 061 061 061 040 127 111 116 104 117 127 075 061
          T   =   1   7   1   1   1       W   I   N   D   O   W   =   1
0000260 062 064 040 122 105 123 075 060 170 060 060 040 101 103 113 040
          2   4       R   E   S   =   0   x   0   0       A   C   K    
0000300 125 122 107 120 075 060 040 012
          U   R   G   P   =   0      \n
0000310

$ cat messages |grep et0 |od -bc
0000000 112 165 154 040 062 061 040 061 063 072 065 062 072 062 066 040
          J   u   l       2   1       1   3   :   5   2   :   2   6    
0000020 154 157 165 040 153 145 162 156 145 154 072 040 155 157 145 040
          l   o   u       k   e   r   n   e   l   :       m   o   e    
0000040 143 150 145 143 153 111 116 075 145 164 150 060 040 117 125 124
          c   h   e   c   k   I   N   =   e   t   h   0       O   U   T
0000060 075 145 164 060 170 060 060 040 101 103 113 040 125 122 107 120
          =   e   t   0   x   0   0       A   C   K       U   R   G   P
0000100 075 060 040 012 112 165 154 040 062 061 040 061 064 072 060 062
          =   0      \n   J   u   l       2   1       1   4   :   0   2
0000120 072 062 071 040 154 157 165 040 153 145 162 156 145 154 072 040
          :   2   9       l   o   u       k   e   r   n   e   l   :    
0000140 155 157 145 040 143 150 145 143 153 111 116 075 145 164 060 040
          m   o   e       c   h   e   c   k   I   N   =   e   t   0    
0000160 117 125 124 075 145 164 150 061 040 123 122 103 075 061 071 062
          O   U   T   =   e   t   h   1       S   R   C   =   1   9   2
0000200 056 061 066 070 056 060 056 061 060 063 040 104 123 124 075 070
          .   1   6   8   .   0   .   1   1   3       D   S   T   =   8
0000220 070 056 061 067 062 056 062 062 070 056 065 060 040 114 105 116
          8   .   1   7   2   .   2   2   8   .   5   0       L   E   N
0000240 075 061 065 060 060 040 124 117 123 075 060 170 060 060 040 120
          =   1   5   0   0       T   O   S   =   0   x   0   0       P
0000260 122 105 103 075 060 170 060 060 040 124 124 114 075 066 063 040
          R   E   C   =   0   x   0   0       T   T   L   =   6   3    
0000300 111 104 075 063 071 065 065 060 040 104 106 040 120 122 117 124
          I   D   =   3   9   5   5   0       D   F       P   R   O   T
0000320 117 075 124 103 120 040 123 120 124 075 070 060 040 104 120 124
          O   =   T   C   P       S   P   T   =   8   0       D   P   T
0000340 075 064 062 065 067 062 040 127 111 116 104 117 127 075 061 060
          =   4   2   5   7   2       W   I   N   D   O   W   =   1   0
0000360 070 040 122 105 123 075 060 170 060 060 040 101 103 113 040 125
          8       R   E   S   =   0   x   0   0       A   C   K       U
0000400 122 107 120 075 060 040 012 112 165 154 040 062 061 040 061 065
          R   G   P   =   0      \n   J   u   l       2   1       1   5
0000420 072 062 066 072 063 067 040 154 157 165 040 153 145 162 156 145
          :   2   6   :   3   7       l   o   u       k   e   r   n   e
0000440 154 072 040 155 157 145 040 143 150 145 143 153 111 116 075 145
          l   :       m   o   e       c   h   e   c   k   I   N   =   e
0000460 164 060 040 117 125 124 075 145 164 150 061 040 123 122 103 075
          t   0       O   U   T   =   e   t   h   1       S   R   C   =
0000500 061 071 062 056 061 066 070 056 060 056 061 060 063 040 104 123
          1   9   2   .   1   6   8   .   0   .   1   0   3       D   S
0000520 124 075 070 060 056 061 063 070 056 061 071 067 056 064 067 040
          T   =   8   0   .   1   3   8   .   1   9   7   .   4   7    
0000540 114 105 116 075 061 064 070 060 040 124 117 123 075 060 170 060
          L   E   N   =   1   4   8   0       T   O   S   =   0   x   0
0000560 060 040 120 122 105 103 075 060 170 060 060 040 124 124 114 075
          0       P   R   E   C   =   0   x   0   0       T   T   L   =
0000600 066 063 040 111 104 075 067 064 063 066 040 104 106 040 120 122
          6   3       I   D   =   7   4   3   6       D   F       P   R
0000620 117 124 117 075 124 103 120 040 123 120 124 075 070 060 040 104
          O   T   O   =   T   C   P       S   P   T   =   8   0       D
0000640 120 124 075 063 067 071 062 040 127 111 116 104 117 127 075 066
          P   T   =   3   7   9   2       W   I   N   D   O   W   =   6
0000660 064 063 062 040 122 105 123 075 060 170 060 060 040 101 103 113
          4   3   2       R   E   S   =   0   x   0   0       A   C   K
0000700 040 125 122 107 120 075 060 040 012 112 165 154 040 062 061 040
              U   R   G   P   =   0      \n   J   u   l       2   1    
0000720 061 066 072 064 067 072 065 070 040 154 157 165 040 153 145 162
          1   6   :   4   7   :   5   8       l   o   u       k   e   r
0000740 156 145 154 072 040 155 157 145 040 143 150 145 143 153 111 116
          n   e   l   :       m   o   e       c   h   e   c   k   I   N
0000760 075 145 164 060 040 104 120 124 075 064 067 062 071 065 040 127
          =   e   t   0       D   P   T   =   4   7   2   9   5       W
0001000 111 116 104 117 127 075 061 062 064 040 122 105 123 075 060 170
          I   N   D   O   W   =   1   2   4       R   E   S   =   0   x
0001020 060 060 040 101 103 113 040 125 122 107 120 075 060 040 012 112
          0   0       A   C   K       U   R   G   P   =   0      \n   J
0001040 165 154 040 062 061 040 061 066 072 065 064 072 065 063 040 154
          u   l       2   1       1   6   :   5   4   :   5   3       l
0001060 157 165 040 153 145 162 156 145 154 072 040 155 157 145 040 143
          o   u       k   e   r   n   e   l   :       m   o   e       c
0001100 150 145 143 153 111 116 075 145 164 150 060 040 117 125 124 075
          h   e   c   k   I   N   =   e   t   h   0       O   U   T   =
0001120 145 164 060 056 062 060 061 056 061 063 063 040 104 123 124 075
          e   t   0   .   2   0   1   .   1   3   3       D   S   T   =
0001140 061 071 062 056 061 066 070 056 060 056 061 060 063 040 114 105
          1   9   2   .   1   6   8   .   0   .   1   0   3       L   E
0001160 116 075 065 062 040 124 117 123 075 060 170 060 060 040 120 122
          N   =   5   2       T   O   S   =   0   x   0   0       P   R
0001200 105 103 075 060 170 060 060 040 124 124 114 075 064 062 040 111
          E   C   =   0   x   0   0       T   T   L   =   4   2       I
0001220 104 075 061 065 071 065 071 040 104 106 040 120 122 117 124 117
          D   =   1   5   9   5   9       D   F       P   R   O   T   O
0001240 075 124 103 120 040 123 120 124 075 065 062 066 066 066 040 104
          =   T   C   P       S   P   T   =   5   2   6   6   6       D
0001260 120 124 075 070 060 040 127 111 116 104 117 127 075 061 060 063
          P   T   =   8   0       W   I   N   D   O   W   =   1   0   3
0001300 061 067 040 122 105 123 075 060 170 060 060 040 101 103 113 040
          1   7       R   E   S   =   0   x   0   0       A   C   K    
0001320 125 122 107 120 075 060 040 012
          U   R   G   P   =   0      \n
0001330

Last edited by userlander; 07-22-2008 at 01:56 PM.
 
Old 07-22-2008, 02:32 PM   #17
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 63
It is allowing you to see the actual stream of bytes, including any non-printable characters. We only see spaces and a newline, not long streams of unprintables, so the data in the logs actually does contain :

kernel: moe checkIN=eteth0 OUT=eth1 ...

but they should look like:

kernel: IN=eth0 OUT=eth1 ...

So the "moe check" and those shown in earlier posts "server check" are garbage. This output is coming from iptables, via the kernel logger. Since you're seeing either semi-random gibberish at the beginning of some kernel logged messages, and this can result from any of a number of reasons.

This is where I would now eliminate the other things I mentioned: starting with hardware.

I would take the system offline, run a full memory test and quick hardware test. Then, I'd boot into single user and force an fsck of the file systems.
 
Old 07-22-2008, 04:35 PM   #18
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Logging is basically translating of what the kernel sees to ASCII log (in this case) and apart from HW failures this can happen with a system under considerable stress, like disks not keeping up with the amount of data to be logged. That's why for instance with Snort binary logging format is preferred because then you don't have to translate on the fly but only when you need to read it.
 
Old 07-22-2008, 04:49 PM   #19
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 63
I've never seen a log entry corrupted in that fashion; dropped yes, but random, partial merges in the kernel log ring buffer like the OP experiences? No ring buffer algorithm I know of would account for this. It is more like substrings of two strings have been joined/merged. It is like a printf strings:

char *kern="kernel:";
char *in="eth0";
char *out="eth1";

printf ("%s IN=%s OUT=%s...", kernel,in,out,...)

But kern was appended to or its \0 overwritten, making it "kernel: moe check", and the string for in was copied two bytes too far leaving "et" and appending "eth0" making it "eteth0". The way these strings are spliced leaves little doubt in my mind.

Last edited by Mr. C.; 07-22-2008 at 04:50 PM.
 
Old 07-23-2008, 03:59 PM   #20
userlander
Member
 
Registered: Jul 2008
Distribution: Arch, Debian
Posts: 61

Original Poster
Rep: Reputation: 18
Quote:
Originally Posted by Mr. C. View Post
But kern was appended to or its \0 overwritten, making it "kernel: moe check", and the string for in was copied two bytes too far leaving "et" and appending "eth0" making it "eteth0". The way these strings are spliced leaves little doubt in my mind.
Little doubt in your mind that it's an exploit you mean?

Logwatch looked more normal today as far as no mangled entries. Hardware check tomorrow, could be the hard drive is failing, I guess. It is pretty old.

By the way, can anyone tell me what exactly "NEW not SYN?" means? I keep seeing that a lot, not even related to this. I searched around the internet, but I can't figure out if it's good, bad, normal, neutral, undesirable, or what.

Thank you.
 
Old 07-23-2008, 04:13 PM   #21
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 63
No, I'm not concluding its an exploit. I am saying that the way the log line is munged indicates some corruption somewhere. This is why you have to rule all other things out first.

NEW state vs. SYNchronize with existing state.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
hacked? WRSpithead Linux - Security 2 08-30-2006 03:21 AM
Got Hacked! Why? hagen00 Linux - Security 13 08-20-2006 08:44 AM
Could have been hacked! Help! Charles Daniel Linux - Security 28 08-03-2005 06:17 AM
Was I Hacked??? treedstang Linux - Security 2 05-20-2004 09:41 AM
hacked WannaLearnLinux Linux - Newbie 7 10-18-2003 01:34 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 03:21 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration