Archive for September, 2009

Linux performance tuning tools (vmstat tool)

Wednesday, September 23rd, 2009

Ref: Performance Tuning for Linux® Servers

Ref: System Performance Tuning, Second Edition

Ref: Optimizing Linux® Performance: A Hands-On Guide to Linux® Performance Tools

Linux Performance Tools:

Processor time is organized into four timed modes: system time, user time, I/O wait time, and idle time. The idle time consists of what’s left over when all other portions have had their fill. A program’s normal operating state is user mode, but as it runs, it may generate requests for services that the kernel provides, such as I/O. These requests require the attention of the operating system, so the program switches into system mode, then returns to user mode when the request is complete. The time spent in these two modes is tabulated independently to give the user time and system time values, respectively. These two figures account for the majority of a process’s execution time.

Note that vmstat reports only the user time, system time, and idle time (wait time is summed in with idle time). In order to get separated values for wait time and idle time, use mpstat.

When a process waits for a block device data request to complete, it incurs I/O wait time. This brings up an important fact: when a process is blocked in this fashion, all idle time becomes wait time. If your idle time is zero, as reported by vmstat, the first thing you should check is if your system has I/O throughput problems.

vmstat (Virtual Memory Statistics):

  • How many processes are running
  • How the CPU is being used
  • How many interrupts the CPU receives
  • How many context switches the scheduler performs
vmstat [-n] [-s] [delay [count]]
vmstat 2 5
Column Explanation
r This is the number of currently runnable processes. These processes are not waiting on I/O and are ready to run. Ideally, the number of runnable processes would match the number of CPUs available.

Performance hits: If the run queue is consistently at or more than four times the number of configured processors in the system, you should probably consider increasing the available processors.

b This is the number of processes blocked and waiting for I/O to complete.
forks The is the number of times a new process has been created. Command : vmstat -f
in This is the number of interrupts occurring on the system.
cs This is the number of context switches happening on the system.
us The is the total CPU time as a percentage spent on user processes (including “nice” time).
sy The is the total CPU time as a percentage spent in system code. This includes time spent in the system, irq, and softirq state.
wa The is the total CPU time as a percentage spent waiting for I/O.
id The is the total CPU time as a percentage that the system is idle.

Sample output:

[ezolt@scrffy tmp]$ vmstat
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
1  0 181024  26284  35292 503048    0    0     3     2    6      1  5  1 94  0

What to look for:

One key to focus on is the (wa), as consistent high numbers here is definitely a problem waiting on I/O. Usually seeing consistent (wa) in 10+ will show degradation of the system. Once it reaches 35+ you will not need to look at the statistics as your users will be complaining.

Look for high numbers in either system or user space. If you see consistent high numbers for user space it could be an application that has a process that is consuming too much resources. In this case look at top to see if you can identify the problem process. If you see consistent high numbers for system then you also need to look into what programs are taking the CPU resources and evaluate their status

To be continue…

How to install apache2-php-mysql from source

Thursday, September 10th, 2009

Prerequisite : yum install gcc-c++ gcc make ncurses-devel openssl-devel glibc* libc-*

Packages required for php: yum install libjpeg-devel libpng-devel curl-devel libmcrypt-devel krb5-devel

Apache Server Installation from Source:
Apache installation directory is : /usr/local/apache
a) Download the apache source file from : http://httpd.apache.org/download.cgi
b) Download the source file in to /tmp directory.
c) I am guessing the source file is httpd-2.2.13.tar.gz
d) Cd /tmp
e) tar –xvzf httpd-2.2.13.tar.gz
f) cd httpd-2.2.13

g)

 
./configure  --prefix=/usr/local/apache --with-included-apr --with-php --with-mysql --with-susexec --disable-info --with-mpm=prefork --enable-so --enable-cgi --enable-rewrite --enable-ssl --enable-mime-magic --enable-unique-id --enable-mods-shared="proxy cache ssl all"

h) make
i) make install
j)To restart apache : /usr/local/apache/bin/apachectl start

MySql Server Install from source:

Ref:http://dev.mysql.com/doc/refman/5.1/en/quick-install.html

a)Download my.version.tar.gz from
http://dev.mysql.com/downloads/mysql/5.1.html#source

b)shell> groupadd mysql
c)shell> useradd -g mysql mysql
d)shell> gunzip < mysql-VERSION.tar.gz | tar -xvf – e)shell> cd mysql-VERSION
f)

   ./configure --prefix=/usr/local/mysql --with-ssl --with-plugins=innobase

note:: for mysql 5.1 : to add innodb support its ” –with-plugins=innobase” but for 5.0 its “./configure –with-innodb”
g)shell> make
h)shell> make install
i)shell> cp support-files/my-medium.cnf /etc/my.cnf
j)shell> cd /usr/local/mysql
k)shell> chown -R mysql .
l)shell> chgrp -R mysql .
m)shell> bin/mysql_install_db -–user=mysql
n)shell> chown -R root .
o)shell> chown -R mysql var
p)shell> bin/mysqld_safe -–user=mysql &

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
./bin/mysqladmin -u root password 'new-password'

Php installation from source with GD library Support

http://www.php.net/manual/en/install.unix.apache2.php

a) Download the php source from here : http://www.php.net/downloads.php
b) Download the source file in to /tmp directory
c) Here I am gussing the php version is php-5.3.0.tar.gz
d) Tar –xvzf php-5.3.0.tar.gz
e) Cd php-5.3.0
f)

./configure --with-apxs2=/usr/local/apache/bin/apxs --with-mysql=/usr/local/mysql --enable-mbstring --with-gd --with-zlib --with-jpeg-dir --with-png-dir --with-openssl --with-curl --with-mcrypt --with-imap --with-imap-ssl --with-kerberos --with-mysqli=/usr/local/mysql/bin/mysql_config

g) make
h) make install
i) setup your php.ini : cp php.ini-dist /usr/local/lib/php.ini

j) In httpd.conf file.. check for bellow lines

LoadModule php5_module modules/libphp5.so

j) Add the bellow lines in httpd.conf file to allow .php extension.
add bellow lines under directive

Add php extension

 
<FilesMatch "\.phps$">
          SetHandler application/x-httpd-php-source
      </FilesMatch>
 
 <FilesMatch "\.ph(p[2-6]?|tml)$">
          SetHandler application/x-httpd-php
      </FilesMatch>

j) Stop apache /usr/local/apache/bin/apachectl1 stop
k) Restart apache /usr/local/apache/bin/apachectl1 start

Note :

(a) configure: error: xml2-config not found. Please check your libxml2 installation. : yum install libxml2-devel

(b) configure: error: libpng.(a|so) not found.
configure: error: libjpeg.(a|so) not found.
(c) Error : configure: error: utf8_mime2text() has new signature, but U8T_CANONICAL is missing
yum install libc-client-devel*
So it will try to find accurate rpm for your kernel(32/64)

(d) If you have older httpd daemon running , please stop that daemon,Other wise when you will start apache daemon, it will through an error .You can check by bellow command to make sure you don’t have any other httpd is running in background.

ps aux | grep -v grep | grep httpd

If this returns value that means another httpd daemon is running and you can stop it by executing

service httpd stop

Last Update : 14-09-2010

How to backup linux Server remotely

Monday, September 7th, 2009

Bellow article will show how to backup a Linux server remotely by using RSYNC with public key base authentication.

To make this backup process automatic we need a password less authentication system so that we don’t have to insert username and password before backup starts.

How to create Key base authentication:

Here are the steps you need to do on the computer that acts as the SSH client:

1) Generate your SSH encryption key pair for the filecopy account. Press the Enter key each time you are prompted for a password to be associated with the keys. (Do not enter a password.)

[filecopy@bigboy filecopy]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key
(/filecopy/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in
/filecopy/.ssh/id_dsa.
Your public key has been saved in
/filecopy/.ssh/id_dsa.pub.
The key fingerprint is:
1e:73:59:96:25:93:3f:8b:50:39:81:9e:e3:4a:a8:aa
filecopy@bigboy
[filecopy@bigboy filecopy]#

2) These keyfiles are stored in the.ssh subdirectory of your home directory. View the contents of that directory. The file named id_dsa is your private key, and id_dsa.pub is the public key that you will be sharing with your target server. Versions other than RedHat/Fedora may use different filenames, use the SSH man pages to verify this.

[filecopy@bigboy filecopy]# cd ~/.ssh
[filecopy@bigboy filecopy]# ls
id_dsa  id_dsa.pub  known_hosts
[filecopy@bigboy .ssh]#

3) Copy only the public key to the home directory of the account to which you will be sending the file.

[filecopy@bigboy .ssh]# scp id_dsa.pub filecopy@smallfry:public-key.tmp

Now, on to the server side of the operation.

Configuration – Server Side

Here are the steps you need to do on the computer that will act as the SSH server.

1) Log into smallfry as user filecopy. Create an .ssh subdirectory in your home directory and then go to it with cd.

[filecopy@smallfry filecopy]# ls
public-key.tmp
[filecopy@smallfry filecopy]# mkdir .ssh
[filecopy@smallfry filecopy]# chmod 700 .ssh
[filecopy@smallfry filecopy]# cd .ssh

2) Append the public-key.tmp file to the end of the authorized_keys file using the >> append redirector with the cat command. The authorized_keys file contains a listing of all the public keys from machines that are allowed to connect to your Smallfry account without a password. Versions other than RedHat/Fedora may use different filenames, use the SSH man pages to verify this.

[filecopy@smallfry .ssh]# cat ~/public-key.tmp >> authorized_keys
[filecopy@smallfry .ssh]# rm ~/public-key.tmp

From now on you can use ssh and scp as user filecopy from server bigboy to smallfry without being prompted for a password.

2. Taking Backup by Rsync

Write a Backup script Example : backup.sh in /root directory

cd /root

vi backp.sh

Press I to Insert

then Write like this :

#!/bin/bash

DESTROOT=”/backups”
TODAY=`date ‘+%A’`

rsync -e ssh -avz –delete filecopy@smallfry:/var/www $DESTROOT/backup

#Archive todays files
tar czvf $DESTROOT/archived/${TODAY}-backup.tar.gz $DESTROOT/backup > $DESTROOT/archived/${TODAY}-backup.log

Press :wq [ to Save the file and exit]

then : chmod 700 backup.sh [ so it will make this file executable for root]

Explanation : The Script will connect to the server smallfry via Ssh, then will download all the files from /var/www directory to local pc under /backups Directory.

3. Automate the system by Crontab

crontab -e

Press I to insert
00 3 * * 1-5 /root/backup.sh >/dev/null 2>&1

Press :wq [ to save the file]

Explanation : Now crontab will execute this file First minute 3 am Every day Every month Monday to Friday .