READ ME FIRST
This article is provided as a courtesy. Installing, configuring, and troubleshooting third-party applications is outside the scope of support provided by (mt) Media Temple. Please take a moment to review the Statement of Support.
This guide shows some of the advanced steps to search for and find compromised scripts/hacks/malware that might be running on your DV server.
Our friends at Google have put together a comprehensive guide on recovering from a hack or compromise that may be helpful to you. We strongly encourage you to visit this valuable resource:
This article has the following dependencies:
- If your server has become severely compromised, it is highly recommended to reinstall your DV. This article is intended to help you determine if your server is hacked. It does not show you how to fix it.
- If you currently do not have your root user enabled or our Developer Tools installed, please visit the Root Access & Developer Tools section of your AccountCenter. For instructions, please see: How do I enable root access to my DV? and Install the Developer Tools.
- Whenever installing third-party software, please consult the official documentation. (mt) Media Temple does not support the installation and configuration of software not installed at time of service activation. Please consult our Scope of Support page for further explanation.
- The domain example.com is used as an example. Please be sure to replace this text with the proper information for your site or server.
Sometimes it is not clearly apparent that a server is compromised. Depending on the level of severity, a hacked server may be beyond repair. When an intruder gains root-level access to a system, they can replace system commands and processes to appear to be doing things they're actually not. Under most circumstances, you should immediately perform a reinstall upon determining that your server has become compromised.
Checking your email queue
Check the number of emails in the queue with the following command:
If you have more than 100 emails in the queue, your service may be compromised. Now run the following command:
grep -R "Subject: failure notice" /var/qmail/queue/mess/*
If you see many failure notices, then it is probably the bounceback spam. You will want to clear out your email queue using qmHandle.
Checking your temporary directories
The /tmp and similar directories are a common location for hackers to place scripts on a server. The following command will let you check to see what's inside this directory:
ls -lab /tmp
Be sure to also check ALL of the following' directories:
Please use "ls -lab" for checking directories as sometimes compromised servers will have hidden files that a regular "ls" will not show.
Check your access_log and error_log files
Sometimes your access_log and error_log files will have information regarding who has compromised your service, or changes they may have made to the system. Look for particular patterns of activity.
Check your process tree
Be sure to also check the process tree (ps -auwwxf) for suspicious processes; often times the malware or hack pretends to be an Apache process. To check for possible exploits that are using Remote File Inclusion, you can run the following command:
zgrep "*=http://" /var/www/vhosts/*/statistics/logs/access_log*| awk '/Apr/ && /libww/ && $9 !~/^4/'
This looks for all the remote inclusions (about 90% of exploits use this technique), in the month of April, using libwww user agent (automated scanning), that didn't have 404 error code. To change the month you are searching in you change out the month code using the first 3 letters of the month's name. So for the month of June you would use Jun, September would be Sep. You can remove "/libwww/ &&" to get more results. You can also run the following command to see if you have compromised script someone is posting to, like sending out mail:
zgrep POST /var/www/vhosts/*/statistics/logs/access_log*| awk '/Apr/ && $9 !~/^4/'
Check your mail logs and email headers
If your server was exploited to send out spam, then the spam report in the email headers may contain some useful information such as the username that sent the emails, or the time that the emails were sent. Viewing your email logs may also provide you with some insightful information.
Check your bash history
Often times, you can see what the attacker did by running the following command as the root user:
Remember to replace username with a real username on your DV.
For more information about using the history command, please read: Using the history command.