Using RHEL 6-x86_64.
No sendmail enable or no any other MTA started.
But mailq is generating highly.As my view this server is hacked.
Thing is how to troubleshoot this.

So what does 'mailq -v' actually say?
And how many unprocessed entries does /var/spool/mqueue/ actually hold?
Have you run Logwatch on /var/log/maillog (as in 'logwatch --detail high --logfile maillog')?
And if you say no MTA is enabled does that mean no MTA is running?
Is there anything else to say about the propose of this machine? Is it a mail or a web server?
Are there processes consuming lots of cpu ('\ps ax -eopcpu,pid,args --sort=pcpu|grep -v "^[[:blank:]]\{1,3\}0\.0";')?
Is there an excessive amount of MTA-related traffic ('lsof -Pwln -i tcp:25 -a -i tcp:587')?


What is that statement based on? Is there anything you could add that would shed a light on the situation?

No mail servers running and this is web server.Generated huge mail queue. But it can not relay through the server.How can this happen?

...is basically the question you started out with. To find out the basic reflex should be to check the log files that running services provide for errors or clues, check system statistics like the process table, network connection list, user access records for anomalies, unwanted processes, increased activity or resource usage and check the file system for unwanted files, permission problems, et cetera. That's what those questions were all about. The more nfo you provide the better insight we get. Providing terse or not enough nfo is inefficient and often leads to speculation which should be avoided at all costs.


- Since when (date, week) did this situation start?
- What list of measures have you tried since this started?
- What output does 'mailq -v' show? Attach its output as plain text file.
- How many unprocessed entries are there in /var/spool/mqueue/ and does this still increase?
- Do run Logwatch on the mail and web server log files and attach its output as plain text file.
- If you say no MTA is running does that mean the MTA was stopped to mitigate the situation or does the machine run without an MTA?
- What (web log, forum software, photo gallery, statistics package, shopping cart or other) software runs on top of the web server (names and versions) and are they all the latest version (including any plugins if any)?
- How many virtual hosts does the web server provide?
- How can users access their web content? Only via HTTP(S) or also by SSH or FTP?
- What hardening measures (firewall, mod_evasive, mod_bandwidth, mod_security) and restrictions (.htaccess, php.ini, etc) does the system have right now?
- Is there an excessive amount of HTTP-related traffic ('lsof -Pwln -i tcp:80')?
- Are there processes consuming lots of cpu ('\ps ax -eopcpu,pid,args --sort=pcpu|grep -v "^[[:blank:]]\{1,3\}0\.0";')?
- Are there files in /tmp, /var/tmp, /var/www and user home directories that look have ownership, access rights and time stamps that seem out of place or could be linked to the start of this situation?
- Have you run Linux Malware Detect, ClamAV or other applications on the contents of /tmp, /var/tmp, /var/www and user home directories?