SERVER LOAD TWEAKS...APACHE/MYSQL/SMTP-POP3/EXIM/IMAP

Q:- How to trace the server load? (Inludes with all application.)

The steps are according to services :-

----------------------------------------------------------------

1. Apache :-

• TOP command >> check for many httpd processes
• Login to WHM >> Check for "Apache ststus",if find anyone downloading mp3,rar,exe,zip files then suspen that account
• Also used to check "cpu/memory/Mysql Usage" option from WHM >> Here we can find actual CPU and Memory usage for particular domain.

----------------------------------------------------------------

2. Mysql :-

• TOP command >> check for many httpd processes
• MySQLadmin process / mysqladmin status
• Also used to check "cpu/memory/Mysql Usage" option from WHM >> Here we can find actual CPU and Memory usage for particular domain.

----------------------------------------------------------------

3. Ftp :-

• ps -aux | grep ftp >> used for checking ftp action taken by user like uploading /downloading files
• tail -f /var/log/secure

----------------------------------------------------------------

4. SMTP / POP3 /IMAP :-

• tail -f /var/log/exim_mainlog >> checking for logs and check which email addresses is continuesly scrolling and confirm is it doing spaming,if confirmed then suspend account
• tail -f /var/log/exim_mainlog | grep public_html >> check for spamming if anybody is using php script for sending mail
• Login to whm and select "Manage Mailqueue" to find the email address which is doing spamming.

----------------------------------------------------------------

First of all check the load and if its above safe limits, we have to settle it down. run deep scripts untill you get the messages "no processes found" for mysql, exim and http. check the load again. restart the services by proper restart scripts
s-http, s-mysql, s-exim. Then proceed for investigation.

* top, shift p, check processes taking load and are in plenty.
* if apache
* quickly go to whm and check apache status and do the needful. if theres nothing in apache
check netstat -n|less . it can be http attack.
* if mysql-
do mysqladmin process and look for processes and queries.
* if exim-
go to /var/log/exim_maillog and check the logs for spam.
this can be done by grepping the logs as follows:
tail -f /var/log/exim_mainlog|grep /tmp
tail -f /var/logs/exim_mainlog |grep public_html
tail -f /var/logs/exim_mainlog |grep sendmail
check mail queue
check for frozen mails and delete frozen mails.
* check io wait if its more.
there can be some reasons for this.
* any user may be downloading heavy files- this you can see in the apache status.
* you can see the heavy cpu consuming processes on the server by ps auxw|grep mvi,mgp,mp3,pkgacct,backup,gzip and you can get the processes. check it and kill it as necessary.
* last is check out for bad processes by ps auxw|grep nobody and kill the bad processes. to know more about what is happening behind the process you can check at /proc/procid.

----------------------------------------------------------------

• 1> ps aux | grep nobody or gzip /backup /fixquota
• 2> TOP / shift + p / shift + m / k =kill
• 3> tail -f /var/log/... | grep ... .avi/.mpg/.rar/.jpg all logs
• 4> cd /proc/pid ls -alh
• 5> Apache status/ cpu mysql memory usage form whm
• 6> netstat -n
• 7> w
• 8> Event Viewer log / Task manager for windows
• 9>tail -f /var/log/exim_mainlog|grep tmp /sendmail /public_html

----------------------------------------------------------------

• 1) top.....to see the process list, then accroding kill the process which is taking load
• 2) ps -aux
• ps -aux | grep gzip, backup, pkg
• tail -f /var/log/exim_maillog | grep sendmail, public, tmp
• 3) w to see whois online
• 4) kill httpd, mysqld, cppop
• 5) netstat
• 6) mysqladmin process :- to see the mysql process
• 7) /scripts/restartsrv_service name:- to restart the service if it goes down
• 8) tail -f /etc/httpd/logs/access_log
• tail -f /etc/httpd/logs/error_log

or you can manage the serverload using WHM

1) under the server status option you can see the
apache status...
CPU/Mysql usages/memory
service status and you can manage the server
2) SQL services under this option you can see the mysqladmin process

3) restart services:- using this option you can restart the services

----------------------------------------------------------------


---
top
---
Will display the processes that are using the maximum processor resources
We can use various options to monitor and control process through top like shift + p, shift + m and k which is used to kill processes. r can be used to renice a process and prioritise a process. In case of high i/o wait we need to check the logs for
high resources using processes.

---------
uptime
---------
It displays the the time since the server has been up and running, number of users logged in and the load average. Similarly we can use 'w'.

------------
ps -auxwf
------------
Will display the process with details like, username, pid, resource usage and child processes. It is very effective in monitoring processes.
We generally use ps -auxwf | grep gzip
ps -auxwf | grep backup
ps -auxwf | grep pkg
For bad processes - ps -auxwf | grep nobody

-------------------
Kill and Kill All
-------------------
used to kill processes or services that are found to be eating up server resources.

--------------
Spamming
--------------
To check spamming we can watch for the mail logs using :
tail -f /var/log/exim_mainlog | grep sendmail
tail -f /var/log/exim_mainlog | grep tmp
tail -f /var/log/exim_mainlog | grep public_html
as spamming can be done from a user's public_html directory using a script or through sendmail. Another way of spamming is using the tmp directory as it is the 'world writable directory'.

--------------
WebServer logs
--------------
We can check for customized logs in the WHM under the Server Status section.
We can trace the user responsible for high web server resource usage by the folowing command
tail -f /etc/httpd/logs/access_log | grep mp3
tail -f /etc/httpd/logs/access_log | grep rar
tail -f /etc/httpd/logs/access_log | grep wav etc

tail -f /etc/httpd/logs/access_log | grep 408 can be used to check for DDOS attacks on the server.

--------
mysql
--------
Apart from top and ps, 'mysqladmin processlist' can be used to check the mysql processes, users and the type of process/query being run by the user.

Killing a proceess is the first option to control server load, restarting the affected server is another option. Still if the load is high we track down the responsible user and suspend him.(This applies for all servers,i.e,apache, mysql, exim etc.)

----------------------------------------------------------------

1. top - check load average, iowait, httpd, mysql, exim etc.
2. P = CPU Usage, M = Memory Usage, K = kill unwanted processes.
3. If load is high, run "deep".
4. Login to WHM of the server and check apache, cpu/memory, mysql status.
5. If any user found downloading gif images, mp3, etc; suspend that particular user.

iowait is high, someone is backing up their files; run
# ps aux | grep pkg
# ps aux | grep gzip
# ps aux | grep backup

Spamming check-
# tail /var/log/exim_mainlog –f | grep public_html
MySQLdump check-
# ps aux | grep mysqldump
# mysqladmin processlist -- mysql status
Bad processes running-
# ps aux | grep nobody

Service restart commands-
#/scripts/restartsrv_mysql
#/scripts/restartsrv_httpd
#/scripts/restartsrv_exim


----------------------------------------------------------------



By using a top command you can find out the process which is causing the load on the server. You can use kill or kilall command to kill that process. OR you can run deep command which will kill all the httpd, exim and mysql process. once the server load comes down restart the service which you have killed.

1. Apache :-

Using top command, we will come to know whether httpd service is eating up high resources on server. if so then kill the httpd service and restart it again when load comes to normal.

You can check if any backup is going on, run the following commands:
# ps aux | grep pkg
# ps aux | grep gzip
# ps aux | grep backup
If any backup process is going on, kill that process.

Also Login to WHM and Check for "Apache ststus", if find anyone downloading mp3,rar,exe,zip files then suspend that account.

the other way to check any download is going on is by using
ps auxw | grep nobody | grep mp3 , jpeg, wmv, mpeg, rar and kill that process.


ALso you can used to check "cpu/memory/Mysql Usage" option from WHM. Here we can find actual CPU and Memory usage for particular domain.

2. Exim Mail :-

Check for spamming by checking the logs for exim

tail -f /var/log/exim_mainlog >> checking for logs and check which email addresses is continuesly scrolling and confirm is it doing spaming,if confirmed then suspend account.

tail -f /var/log/exim_mainlog | grep tmp

Login to whm and select "Manage Mailqueue" to find the email address which is doing spamming.

3. MySQL :-

Use the mysqladmin command as mysqladmin process / mysqladmin status

Also used to check "cpu/memory/Mysql Usage" option from WHM. You can get actual CPU and Memory usage for particular domain.


4. Ftp :-

ps -aux | grep ftp >> used for checking ftp action taken by user like uploading /downloading files
tail -f /var/log/secure

5. ps command :-

Use ps -auxw command ALWAYS to check if there is bad processes running by doing ps aux|grep nobody. You'll sometimes see bad scripts running as nobody. Normally only httpd, merlange chat, and sometimes proftpd are run as user nobody. so if you find any other process is running as user nobody kill that process.

kill -9 pidofproc

----------------------------------------------------------------

Some basic commands to see load average and process running on the server,
1]top :: This command is very useful for system administartion. Basically it gives you summary view of system, including number of users, memory usage, CPU usage and active processes.
Shift+p =>list all processes accourding to maximum CPU usage.
Shift+m=>list all processes accourding to maximum memory usage.

2]w :: This commands gives us information regarding who is logged into server and what processes they are running
w -s , gives you shorter process listing.

3]uptime:: It will also gives us information regarding the number of user logged into server, Current time, time since server is up, load average.

4]ps :: list the current running processes.
ps -aux , gives us information of users, PID, resoure usages like CPU and memory, processes running.

Reasons for increase of load on the server::

Load on server will get increase due to sevral reasons sated below,

1.many httpd processes
2.any user is downloading mp3, exe, zip files
3.email spamming
4.uploading/downloading files via FTP
5.mysql processes and queries run by user

Troubleshooting::

1.If many load increased suddenly, fire "deep" command, which kills httpd, mysql and exim processes.
2. Also Login to WHM and Check for "Apache ststus", if find anyone downloading mp3,rar,exe,zip files then suspend that account.
3.For spamming, you need to fire following commands,
tail -f /var/log/exim_mainlog | grep public_html



----------------------------------------------------------------
1)top
2)w
3)shift+m
4)shift+p
5)tail -f /usr/local/apache/logs/access_log
6)tail -f /usr/local/apache/logs/error_log
7)tail -f /var/log/exim_mainlog


----------------------------------------------------------------


1. Check for server load using top command with following options:
Shift p CPU Usage,
Shift m Memory Usage
& check which process is taking load with the help of above two options.
Kill the responsible process using k option.
2. Check for the downloads using
# ps auxw | grep nobody | grep mp3 , jpeg, wmv, mpeg, rar
# ps auxw | grep gzip, backup, fixquota
Suspend the perticular account who is repeatedly downloading the above mentioned files.
3. Check for access & error logs for following options
# tail -f /etc/httpd/logs/access_log | grep 408, zip
# tail -f /etc/httpd/logs/error_log | grep 203
4. Check mail spamming with following commands.
#tail -f /var/log/exim_mainlog | grep sendmail, public_html, tmp
5. Login to WHM of the server and check apache, cpu/memory, mysql status & check for frozen mails in mail queue manager.
6. Check Mysql errors with
# mysqladmin processlist
check the users, command, time & information fields.
7. If you are making changes to httpd.conf then first run # httpd -configtest before restarting httpd to reduce the downtime.
8. Restart the perticular service causing load tomgo hihg with # /scripts/restartsrv_httpd, exim, mysql

----------------------------------------------------------------

How to repair Mysql database?

Login to ssh with root password >>

mysqlcheck -o -r --all-databases

Cheers!

How to enable InnoDB engine in MySql?

To enable it just remove the line "skip-innodb" from your my.cnf and restart the mysql-server.