Archive for the ‘Linux’ Category

Linux SVN/Subversion Usefull commands

Thursday, February 25th, 2010

Ref:http://svnbook.red-bean.com/en/1.5/svn.tour.cycle.html#svn.tour.cycle.update

(A) How to install and Import directory into repository:
Click here

(B)How to get working copy from svn repository into local machine?
Goto Your home directory , example : cd /root
Now execute bellow commands

[root@mail ~]# svn checkout file:///svn
A    svn/script
A    svn/script/checkmemory.sh
A    svn/config
A    svn/config/httpd-vhosts
Checked out revision 2.

Now you will see there is a directory call svn has been created in to your home directory and this svn has all the files we added before
Example :

[root@mail ~]# ls
 rpmforge-release-0.3.6-1.el5.rf.i386.rpm
epel-release-5-3.noarch.rpm  script
svn
[root@mail ~]# cd svn
[root@mail svn]# ls
config  script
[root@mail svn]# cd script/
[root@mail script]# ls
checkmemory.sh
[root@mail script]# pwd
/root/svn/script
[root@mail script]#

NOte : If you want to just check out only one directory example script. then command would be :

[root@mail ~]# svn checkout file:///svn/script
A    script/checkmory.sh
Checked out revision 2.

(C) How to get the Updated copy of the working directory

[root@mail script]# svn update
U    checkmemory.sh
Updated to revision 3.
[root@mail script]#

While you are working on a file.if any one changes to that file and you want to download those changes, then svn update will give the up to date file.

(D) How to add directory or file to the repository

[root@mail script]# svn add check_cpu.sh check_memory.sh
A         check_cpu.sh
A         check_memory.sh
[root@mail script]#
 
Note : Dont forget to run bellow command to make the update permanently
svn commit

(E) How to delete file or directory

 [root@mail script]# svn delete check_memory.sh 
D         check_memory.sh

(F) How to view overall changes of local modifications

[root@mail script]# svn status
A       check_ping
M       check_cpu.sh
[root@mail script]#

(G) How to view the details of local modifications

For Every modifications( every files)
[root@mail script]# svn diff
Index: check_ping
===================================================================
--- check_ping  (revision 0)
+++ check_ping  (revision 0)
@@ -0,0 +1 @@
+This is a test insert
Index: check_cpu.sh
===================================================================
--- check_cpu.sh        (revision 4)
+++ check_cpu.sh        (working copy)
@@ -0,0 +1 @@
+This is a test line
[root@mail script]#
 
For a particular file:
 
[root@mail script]# svn diff check_cpu.sh
Index: check_cpu.sh
===================================================================
--- check_cpu.sh        (revision 4)
+++ check_cpu.sh        (working copy)
@@ -0,0 +1 @@
+This is a test line
[root@mail script]#

(H) How to commit changes ( update your working copy with svn repository)

   svn commit

Before committing , it will open vim editor and you need to insert some comments about what you updating for your/Other peoples references.

(I) How to check history ?
Ref :http://svnbook.red-bean.com/en/1.5/svn.tour.history.html

[root@mail script]# svn log
------------------------------------------------------------------------
r5 | root | 2010-02-25 14:10:45 +0000 (Thu, 25 Feb 2010) | 3 lines
 
Added Check_ping file
Modified check_cpu file
 
------------------------------------------------------------------------
r4 | root | 2010-02-25 13:55:01 +0000 (Thu, 25 Feb 2010) | 2 lines
 
Added Check_cpu and Delete check_memory.sh file
 
------------------------------------------------------------------------
r3 | root | 2010-02-25 13:42:52 +0000 (Thu, 25 Feb 2010) | 2 lines
 
Just update
 
------------------------------------------------------------------------
r1 | root | 2010-02-25 11:19:54 +0000 (Thu, 25 Feb 2010) | 2 lines
 
Just Adding a directory
 
------------------------------------------------------------------------

Meaning of A,C,D,M:

A item
 
    The file, directory, or symbolic link item has been scheduled for addition into the repository.
C item
 
    The file item is in a state of conflict. That is, changes received from the server during an update overlap with local changes that you have in your working copy (and weren't resolved during the update). You must resolve this conflict before committing your changes to the repository.
D item
 
    The file, directory, or symbolic link item has been scheduled for deletion from the repository.
M item
 
    The contents of the file item have been modified.

How to install Subversion

Thursday, February 25th, 2010

Ref:http://svnbook.red-bean.com/en/1.5/index.html

(A) To install Subversion : yum install mod_dav_svn subversion
(B) How to create a Repo :

svnadmin create /svn

So it will create a svn directory under / directory

[root@mail /]# pwd
/
[root@mail /]# ls
aquota.group  boot      dev   lib    opt   sbin     svn  usr
aquota.user   conffile  etc   media  proc  selinux  sys  var
bin           data      home  mnt    root  srv      tmp
[root@mail /]#

(C) How to import Directory in svn repos ?

 svn import directory/ file:///svn/directory

output :

[root@mail /]# svn import /root/script/ file:///svn/script
Adding         /root/script/checkmory.sh
 
Committed revision 1.

Note After typing the svn import command it will open a Vim editor ,and will ask you to write some comments about this import
Example: I would of write: I am adding the /root/script directory into svn repository.
Pic:

Svn Asking to input comments before adding directory to repository

Svn Asking to input comments before adding directory to repository

Note : if you see bellow error :

svn: Could not use external editor to fetch log message; consider setting the $SVN_EDITOR environment variable or using the --message (-m) or --file (-F) options
svn: None of the environment variables SVN_EDITOR, VISUAL or EDITOR are set, and no 'editor-cmd' run-time configuration option was found

which means: you need to add “vim ” program path into bash_profile files

How to add vim programm path in .bash_profile file :

vi /root/.bash_profile
add this line :
export SVN_EDITOR="/bin/vi"

Now save the file and logoff and log on agian

(d) How to check list of projects in svn repo

[root@mail /]# svn list file:///svn

Output :

script/

Which means script directory has been added into svn repository.
Now if you want to see what is inside script directory

[root@mail /]# svn list file:///svn/script
checkmemory.sh

nagios script for checking mysql server replication status between 4 servers

Monday, February 22nd, 2010

Ref : http://onlamp.com/pub/a/onlamp/2006/04/20/advanced-mysql-replication.html?page=2

Date:22/02/2010
This Script is still under development.

Purpose:

Develop a nagios script, which would be able to check replication status between 4 Master/Master Server.

This scripy will check following :

#1.Each Mysql servers are online : Stats: Done
#2.If Slave process is running : Status:Done
#3.If Slave IO process is running : Status:Done
#4.If There is any bin log position difference between Master/Slave :Status:Done
#5.Check time in processlist for(Has read all relay log; waiting for the slave I/O thread to update it) for further repliation related problems. Status: Under Development
#6.If problem found,change Master server info in Slave, and connect to differerent Master server. :Status :Under Development
#7. Change Dns record(A record) to stop comming http request to the problematic server:Status:Under Development

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
 
 
#!/bin/bash
################################################################################
#SVN Version: 28
#Script Version 28
#Purpose of this script:
#Check Bellow options between 4 mysql Master/Master Replication
#1.Every Mysql servers are online: Status:Done
#2.If Slave process is running:    Status:Done
#3.If Slave IO process is running: Status:Done
#4.If There is any bin log position difference between Master/Slave: Status:Done
#5.Check time in processlist for:Status: In Development
#6.If problem found change  Master server info in Slave
#7. Change Dns record(A record) to stop comming http request to the problematic sever
#################################################################################
 
#Status check for nagios script
 
STATE_OK=0
STATE_WARNING=1
STATE_CRITICAL=2
STATE_UNKNOWN=3
STATE_DEPENDENT=4
 
 
#Define All the variables
 
declare -rx SCRIPT=${0##*/}
declare -rx CMD_AWK="/bin/awk"
declare -rx CMD_MYSQL="/usr/local/mysql/bin/mysql"
declare -rx CMD_GREP="/bin/grep"
 
##Define value for nagios is to fire for certain conditions for log check
 
CRITICAL_VALUE=100
CRITICAL_VALUE=50
NORMAL_VALUE=0
 
#######################################################################
#####Define All the servers Ip, username and password for mysql server#
#######################################################################
declare -rx SLV_MSTRServerA="Ip.of.your.server"
declare -rx SLV_MSTRServerA_USER="noslave"
declare -rx SLV_MSTRServerA_PASSWD="password"
 
#Slave/Master ServerB (NodeB-web.yourdomian.co.uk)
 
declare -rx SLV_MSTRServerB="ip.of.your.server"
declare -rx SLV_MSTRServerB_USER="noslave"
declare -rx SLV_MSTRServerB_PASSWD="password"
 
#Slave/Master ServerC (NodeC-http.yourdomain.local)
 
declare -rx SLV_MSTRServerC="localhost"
declare -rx SLV_MSTRServerC_USER="noslave"
declare -rx SLV_MSTRServerC_PASSWD="password"
 
 
#Slave/Master ServerD (NodeD-beaver.yourdomain.local)
 
declare -rx SLV_MSTRServerD="ip.of.your.server"
declare -rx SLV_MSTRServerD_USER="noslave"
declare -rx SLV_MSTRServerD_PASSWD="password"
 
#Global Variables
declare -a result
 
 
#Definning Variable for Array
 
declare -a SLV_MSTRServerS=($SLV_MSTRServerA $SLV_MSTRServerB $SLV_MSTRServerC $SLV_MSTRServerD)
declare -a SLV_MSTRServerS_USERS=($SLV_MSTRServerA_USER $SLV_MSTRServerB_USER $SLV_MSTRServerC_USER $SLV_MSTRServerD_USER)
declare -a SLV_MSTRServerS_PASSWD=($SLV_MSTRServerA_PASSWD $SLV_MSTRServerB_PASSWD $SLV_MSTRServerC_PASSWD $SLV_MSTRServerD_PASSWD )
declare -a SLV_ServerAppLS=("SLV_MAIL" "SLV_WEB" "SLV_HTTP" "SLV_BEVR")
declare -a MSTR_ServerAppLS=("MSTR_MAIL" "MSTR_WEB" "MSTR_HTTP" "MSTR_BEVR")
 
 
###########################################################
#Section-1.0:function-Command: My Mysql Servers are online#
###########################################################
 
function FUNC_CHECK_SERVER_ONLINE_CMD
{
 
$CMD_MYSQL -h "$1"  -u"$2" -p"$3" -e "show slave status\G" >/dev/null 2>&1
 
return $?
 
}
 
 
###########################################################
#Section-2.0: Function-Command:If Slave_IO_Running#########
###########################################################
 
function FUNC_CHK_SLV_IO_RUN_CMD
{
 
echo $($CMD_MYSQL -h "$1"  -u"$2" -p"$3" -e "show slave status\G" | $CMD_GREP Slave_IO_Running | awk '{ print $2 }' )
 
}
 
 
##########################################################
#Section-3.0: Function-Command:If Slave_SQL_running#######
##########################################################
 
function FUNC_CHK_SLV_SQL_RUN_CMD
{
 
echo $($CMD_MYSQL -h "$1"  -u"$2" -p"$3" -e "show slave status\G" | $CMD_GREP Slave_SQL_Running | awk '{ print $2 }' )
 
}
 
##########################################################
#Section-4.0: Function-Command:#######
##########################################################
 
function FUNC_CHK_SLV_LOG_POS_CMD
{
 
echo $($CMD_MYSQL -h "$1"  -u"$2" -p"$3" -e "show slave status\G" | $CMD_GREP Read_Master_Log_Pos | awk '{ print $2}' )
 
 
}
 
 
 
##########################################################
#Section-5.0: Function-Command:#######
##########################################################
 
function FUNC_CHK_MSTR_LOG_POS_CMD
{
 
echo $($CMD_MYSQL -h "$1"  -u"$2" -p"$3" -e "show master status" | $CMD_GREP bin | cut -f2 )
 
 
}
 
 
###########################################################
#Section-1.1: If all Mysql Server is Online################
##Implementing Secton 1.0(ref:FUNC_CHECK_SERVER_ONLINE_CMD#
###########################################################
 
function FUNC_CHECK_SERVER_ONLINE()
{
i=0
COUNT=${#SLV_MSTRServerS[*]}
while [ $i -lt $COUNT ]
do
 
if ! $(FUNC_CHECK_SERVER_ONLINE_CMD "${SLV_MSTRServerS[$i]}" "${SLV_MSTRServerS_USERS[$i]}" "${SLV_MSTRServerS_PASSWD[$i]}"  )
 
then
echo " Server IP: ${SLV_MSTRServerS[$i]},is not running "
exit $STATE_CRITICAL
exit 99
fi
 
 
i=$(($i+1))
 
done
 
#echo "All Server are Online"
#exit $STATE_OK
 
}
 
 
###########################################################
#Section-2.1: If Slave_IO_Running IS RUNNING OR NOT########
##Implementing Secton 2.0(ref:FUNC_CHK_SLV_IO_RUN_CMD######
###########################################################
 
 
 
function FUNC_CHK_SLV_IO_RUN()
{
 
i=0
COUNT=${#SLV_MSTRServerS[*]}
while [ $i -lt $COUNT ]
do
result[$i]=$(FUNC_CHK_SLV_IO_RUN_CMD "${SLV_MSTRServerS[$i]}" "${SLV_MSTRServerS_USERS[$i]}" "${SLV_MSTRServerS_PASSWD[$i]}" )
i=$(($i+1))
done
 
 
i=0
while [ $i -lt $COUNT ]
do
if [ ${result[$i]} != "Yes" ]
then
echo "In Server IP: ${SLV_MSTRServerS[$i]},Slave_IO_running is not running "
exit $STATE_CRITICAL
 
fi
i=$(($i + 1 ))
done
#echo "ALL Servers are running fine"
#exit $STATE_OK
 
}
 
###########################################################
#Section-3.1: IF Slave_SQL_Running OR NOT#################
##Implementing Secton 3.0(ref:FUNC_CHK_SLV_SQL_RUN_CMD#####
###########################################################
 
function FUNC_CHK_SLV_SQL_RUN()
 
{
i=0
COUNT=${#SLV_MSTRServerS[*]}
while [ $i -lt $COUNT ]
 
do
result[$i]=$(FUNC_CHK_SLV_SQL_RUN_CMD "${SLV_MSTRServerS[$i]}" "${SLV_MSTRServerS_USERS[$i]}" "${SLV_MSTRServerS_PASSWD[$i]}" )
i=$(($i+1))
done
 
 
i=0
while [ $i -lt $COUNT ]
do
if [ ${result[$i]} != "Yes" ]
then
echo "In Server IP: ${SLV_MSTRServerS[$i]},Slave_SQL_Running is not running "
exit $STATE_CRITICAL
 
fi
i=$(($i + 1 ))
done
 
 
}
 
###########################################################
#Section-4.1: GETTING MASTER LOG POSITION FROM EACH SLAVE##
##Implementing Secton 4.0(ref:FUNC_CHK_SLV_LOG_POS_CMD#####
###########################################################
 
 
function FUNC_CHK_SLV_LOG_POS ()
 
{
i=0
COUNT=${#SLV_MSTRServerS[*]}
while [ $i -lt $COUNT ]
 
do
result[$i]=$(FUNC_CHK_SLV_LOG_POS_CMD "${SLV_MSTRServerS[$i]}" "${SLV_MSTRServerS_USERS[$i]}" "${SLV_MSTRServerS_PASSWD[$i]}" )
i=$(($i+1))
done
 
i=0
while [ $i -lt $COUNT ]
do
eval ${SLV_ServerAppLS[$i]}=${result[$i]}
 
 
i=$(($i + 1 ))
done
 
 
}
 
###########################################################
#Section-5.1: GETTING MASTER LOG POSITION FROM EACH MASTER#
##Implementing Secton 5.0(ref:FUNC_CHK_MSTR_LOG_POS_CMD#####
###########################################################
 
 
 
function FUNC_CHK_MSTR_LOG_POS ()
 
{
i=0
COUNT=${#SLV_MSTRServerS[*]}
while [ $i -lt $COUNT ]
 
do
result[$i]=$(FUNC_CHK_MSTR_LOG_POS_CMD "${SLV_MSTRServerS[$i]}" "${SLV_MSTRServerS_USERS[$i]}" "${SLV_MSTRServerS_PASSWD[$i]}" )
i=$(($i+1))
done
 
 
i=0
while [ $i -lt $COUNT ]
do
eval ${MSTR_ServerAppLS[$i]}=${result[$i]}
 
 
i=$(($i + 1 ))
done
 
 
}
 
#########################################################################################
 
#Section-4.1/5.1: FIND OUT LOG POSITION DIFFERENCE#######################################
##Implementing Secton 4.1 and 5.1(ref:FUNC_CHK_SLV_LOG_POS) AND   FUNC_CHK_MSTR_LOG_POS##
#########################################################################################
 
 
function FUNC_FIND_LOG_POS_DIFF ()
 
{
################################################################
##Calling FUNC_CHK_SRV_LOG_POS and FUNC_CHK_MSTR_LOG_POS########
## To Get All the output from Servers###########################
################################################################
 
FUNC_CHK_SLV_LOG_POS
FUNC_CHK_MSTR_LOG_POS
 
echo "Master log position from each master(show master status)"
echo "mail: $MSTR_MAIL"
echo "web: $MSTR_WEB"
echo "http: $MSTR_HTTP"
echo " beaver: $MSTR_BEVR"
 
 
 
echo "MAster log position from each  Slave(show slave status) "
echo "mail: $SLV_MAIL"
echo "web: $SLV_WEB"
echo "http: $SLV_HTTP"
echo "beaver: $SLV_BEVR"
 
 
 
 
### note : $SLV_MAIL will be equal to $MSTR_BEVR
### Note : $SLV_WEB will be equal to $MSTR_MAIL
### Note : $SLV_HTTP will be equal to $MSTR_WEB
### Note : $SLV_BEVR will be equal to $MSTR_HTTP
 
if [ $SLV_MAIL -ne $MSTR_BEVR ]
 
then
echo "Problem between Server Mail: $SLV_MAIL and Server BEVR:$MSTR_BEVR"
exit $STATE_CRITICAL
fi
 
if [ $SLV_WEB -ne $MSTR_MAIL ]
 
then
 
echo "Problem between Server WEB:$SLV_WEB and Server MAIL:$MSTR_MAIL"
exit $STATE_CRITICAL
fi
 
if [ $SLV_HTTP -ne $MSTR_WEB ]
 
then
echo "Problem between Server HTTP:$SLV_HTTP  and Server WEB:$MSTR_WEB "
exit $STATE_CRITICAL
fi
 
if [ $SLV_BEVR -ne $MSTR_HTTP ]
 
then
echo "Problem between Server BEVR:$SLV_BEVR and Server HTTP:$MSTR_HTTP"
exit $STATE_CRITICAL
fi
 
 
}
 
 
######################################################################
#Calling all function from  section [1.1,2.1,3.1,4.1,5.1]#############
######################################################################
 
FUNC_CHECK_SERVER_ONLINE
FUNC_CHK_SLV_IO_RUN
FUNC_CHK_SLV_SQL_RUN
FUNC_FIND_LOG_POS_DIFF
 
 
#####################################################################
## If there is not any error from section[1.1,2.1,3.1,4.1,5.1]#######
## then show bellow commands to ensure nagios all OK ################
#####################################################################
 
 
 
#echo $SLV_HTTP
#echo $SLV_BEVR
 
#echo $MSTR_MAIL
#echo $MSTR_WEB
#echo $MSTR_HTTP
#echo $MSTR_BEVR
 
echo "ALL Servers are running fine"
exit $STATE_OK

Linux:If Running Kernel Is 32 Or 64 Bit

Thursday, February 18th, 2010

To find out, if Your kernel is 64 bit or 32 bit, run this commands

uname -a

Sample Out put for 64 bit kernel :

Linux  2.6.16.53-070731a #1 SMP Tue Jul 31 10:46:54 CEST 2007 x86_64 x86_64 x86_64 GNU/Linux

here x86_64 GNU/Linux indicates this kernel is 64 bit.

Sample Output for 32 bit kernel :

Linux sandbox.hostname.local 2.6.18-164.11.1.el5 #1 SMP Wed Jan 20 07:39:04 EST 2010 i686 i686 i386 GNU/Linux

here i386 GNU/Linux indicates its a 32 bit kernel, also if you see i386/i486/i586/i686 which indicates its 32 bit kernel

also:
How to find if processor is 32 bit or 64 bit:
ref : http://fosiul.com/index.php/2010/02/linux-how-to-conferm-64bit32bit-capability-of-cpu/

Additional repository list for 32 bit kernel and 64 bit kernel
http://fosiul.com/index.php/2009/12/yum-repo-list-for-centos/

Linux-How to conferm 64bit/32bit capability of CPU

Wednesday, February 17th, 2010

How many CPU in the system :

commands : cat /proc/cpuinfo

 
[root@server ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.40GHz
stepping        : 3
cpu MHz         : 3401.008
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl est cid xtpr
bogomips        : 6805.07
 
processor       : 1
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.40GHz
stepping        : 3
cpu MHz         : 3401.008
cache size      : 2048 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl est cid xtpr
bogomips        : 6799.15
 
processor       : 2
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.40GHz
stepping        : 3
cpu MHz         : 3401.008
cache size      : 2048 KB
physical id     : 3
siblings        : 2
core id         : 3
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl est cid xtpr
bogomips        : 6799.30
 
processor       : 3
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Xeon(TM) CPU 3.40GHz
stepping        : 3
cpu MHz         : 3401.008
cache size      : 2048 KB
physical id     : 3
siblings        : 2
core id         : 3
cpu cores       : 1
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl est cid xtpr
bogomips        : 6799.40
 
[root@server ~]#

From the example above there is 4 processor in the system [ processor 0 …. processor 3]

also :

short cut : cat /proc/cpuinfo  | grep processor
processor       : 0
processor       : 1
processor       : 2
processor       : 3


How to find out if processors are 64bit or 32 bit

From the out put cat /proc/cpuinfo, look at the flags column, if there is a word call lm , that means its
a 64bit processor.

Short cut commands :
grep flags /proc/cpuinfo

output :

[root@server ~]# grep flags /proc/cpuinfo
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl est cid xtpr
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl est cid xtpr
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl est cid xtpr
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe lm constant_tsc pni monitor ds_cpl est cid xtpr

From the output above, all four processors has lm word, which means its 64 bit processors
if you do not see lm word, then its 32 bit processor.

Linux/windows-How to add a printer from cups by using .ppd file

Friday, February 12th, 2010

Recently I was trying to add a Canon LBP3460 Printer in my linux Server, I was trying to use .ppd which was available from canon website, but that file would not work.

so I download the file from : http://openprinting.org/printer_list.cgi , and its works perfectly .

here is the procedure::

Download the .ppd file from http://openprinting.org/printer_list.cgi

Upload the file into /usr/share/cups/model/ Directory.

Note : here I added the printer in windows 2003 server and made this printer Shareable by using port number 9100

example : port name : NPI414B70 and port number 9100

as described in the bellow picture

adding printer in windows 2003 server

adding printer in windows 2003 server

Now open your Cups interface by : https://localhost:631

Goto Administration->add new printer and follow as bellow pictues

Adding a new printer in cups (step1)

Adding a new printer in cups (step1)(Click to view Full image)

Selecting how this printer will be connected to linux

Selecting how this printer will be connected to linux

Definning the url for connecting to this printer

Definning the url for connecting to this printer(step3)(Click to view Full image)

cups5

Selecting model/driver for the printer(Step5) (Click to view Full image)

Selecting model(Step4)(Click to view Full image)

Selecting model(Step4)(Click to view Full image)

now you should be able to print from that printer.

How to install puppet in server and client

Monday, February 8th, 2010

Ref : http://docs.reductivelabs.com/guides/installation.html#open_firewall_ports_on_server_and_client

How to install puppet Client:

If yum can not find puppet software you can add bellow repo :

http://fosiul.com/index.php/2009/12/yum-repo-list-for-centos/

after adding repos

(a) yum install puppet( to install puppet client rpm)

(b) edit /etc/puppet/puppetd.conf and add references of puppet server

server = puppet-server.companydomain.com

Or Execute bellow command to connect to puppet server:
puppet agent –server fosiul.fosiul.lan –waitforcert 60 –test

(c) /usr/sbin/puppetd –verbose ( Start the client for the first time)

it will show below output

[root@pupet-client]# /usr/sbin/puppetd –verbose
warning: peer certificate won’t be verified in this SSL session
notice: Did not receive certificate

(d) Now go to Puppet Server , and type

[root@puppet-server]# puppetca –list
puppet-client.companydomain.com

(e) Now Again in the server execute bellow command, this command will sign the certificate fro the pupppet-client host.

puppetca –sign puppet-client.companydomain.com

Note :

(a) Make sure port 8140 (tcp/udp) open in puppet-server

and you have allow only puppet-client’s ip to connect puppet-server

Example :

-A RH-Firewall-1-INPUT -p tcp -m tcp -s ip-of-puppet-client –dport 8140 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp -s ip-of-puppet-client –dport 8140 -j ACCEPT

Linux print job administration

Friday, January 22nd, 2010

How to find every Printer Status :

lpc status

How to view print job for a particular printer:

lpq -Pprinter-spool-name
[root@Host~]# lpq -PGI_LBP
GI_LBP is ready and printing 
Example:
Rank    Owner   Job     File(s)                         Total Size<br />
active  xxx 231020  s.t-avprnt.0H9                  4096 bytes<br />
1st     xxxx   231023  s.t-avprnt.0HD                  4096 bytes</p>

How to remove a particular print job from a printer :

lprm -PGI_LBP 231020

How to remove all print job from a printer:

lprm -PGI_LBP -


How to sent a print job to a different printer

lpr -P printer-spool-name document
example:
lpr -P GI_LBP wordbook.txt

end_request: I/O error, dev fd0, sector 0 (openfiler)

Tuesday, January 12th, 2010

Error:

When you try to open volume groups page from open filer, either its hang or take too long to open and at the same time you see bellow error log ..

Jan 12 09:07:44 filer2 kernel: end_request: I/O error, dev fd0, sector 0
Jan 12 09:07:44 filer2 kernel: Buffer I/O error on device fd0, logical block 0

Solution for openfiler:

Remove the floppy modules from kernel or disable it from kernel.

How to remove:

# lsmod | grep -i floppy

Output should be :
floppy 95465 0

Now remove the module :
# modprobe -r floppy

How to disable:

Ref :
http://www.cyberciti.biz/faq/linux-end_request-ioerror-dev-fd0-sector0/

realtime network monitoring tools

Thursday, December 24th, 2009
  1. tcptrack :http://www.rhythm.cx/~steve/devel/tcptrack/release/1.3.0/docs/tcptrack.1.html
  2. ngrep : http://www.linux.com/archive/feature/46268
  3. ntop :
  4. mrtg:
  5. vnstat: http://humdi.net/vnstat/

useful apache server documentation link

Tuesday, December 15th, 2009
  1. Prefix for configutraion: http://httpd.apache.org/docs/2.2/en/programs/configure.html#installationdirectories

How to install mod_security from source

Tuesday, December 15th, 2009

Ref :http://www.modsecurity.org/documentation/modsecurity-apache/2.5.11/html-multipage/installation.html

Mod security works with apache. So You will have to define where is your Apache location (if you installed Apache from source)
Here I have installed Apache in /usr/local/apache Directory
Note:
Make sure you have mod_unique_id installed :

run the bellow command to make sure mod_unique_id is installed .

bin/apachectl -l | grep  mod_unique_id.c

if this module is not installed then you will have to recompile your Apache with –-enable-unique-id
Example:

./configure  --prefix=/usr/local/apache --with-included-apr --with-php --with-mysql --with-susexec --disable-info --with-mpm=prefork --enable-so --enable-cgi --enable-rewrite --enable-ssl --enable-mime-magic --enable-unique-id

To install Mod_Security you need bellow rpms :

yum install pcre-devel
yum install apr-devel

Download modsecurity from :http://www.modsecurity.org/download/index.html

Configuring and installing Mod_Security

 
a)Download and upload modsecurity-apache_2.5.12.tar.gz in /tmp directory
 
b) tar -xvzf modsecurity-apache_2.5.12.tar.gz
 
c) cd modsecurity-apache_2.5.11
 
d) cd apache2
 
e) ./configure --with-apxs=/usr/local/apache/bin/apxs --with-pcre=/usr/bin/pcre-config --with-apr=/usr/local/apache/bin/apr-1-config --with-apu=/usr/local/apache/bin/apu-1-config
 
f)make
 
g)make intall

Configure Mod security with Apache:

a)Make a directory named modsecurity    under /usr/local/apache/conf/ and copy all the modsecurity rules there
note:
modsecurity rules will be found in modsecurity source directory "/tmp/modsecurity-apache_2.5.11/rules"  
(b) Insert the bellow lines  line in httpd.conf file(/usr/local/apache/conf/) 
    Include conf/modsecurity/*.conf
 
C)Also insert bellow lines in httpd.conf(/usr/local/apache/conf) 
 
   LoadFile /usr/lib/libxml2.so
   LoadFile /usr/lib/liblua-5.1.so  (optionals)    
          Note: This library is optional and only needed if you will be using         the new Lua engine.In that case you will have to  use      -–with-lua=PATH prefix with mod security installation. Ref : </span>http://www.modsecurity.org/documentation/modsecurity-apache   /2.5.11/html-multipage/installation.html    
   LoadModule security2_module modules/mod_security2.so 
           Note: This line should be automatically inserted while installation of mod security.If not then insert by your self.

Now Stop and restart apache service. and check apache error_log for this kind of entry :

[Tue Dec 15 12:14:10 2009] [notice] ModSecurity for Apache/2.5.11 (http://www.modsecurity.org/) configured.
[Tue Dec 15 12:14:10 2009] [notice] Original server signature: Apache/2.2.14 (Unix) mod_ssl/2.2.14 OpenSSL/0.9.8e-fips-rhel5

Enabling mod_security:

By default , Mod_security rules is enabled, but you can check it from here :
modsecurity_crs_10_config.conf  ( location:/usr/local/apache/conf/modsecurity/)
Make sure bellow line is set to ON as bellow
 SecRuleEngine On,

Adding rules to mod_security :

Copy all the rules from base_rules directory to modsecurity directory 
 cp /tmp/modsecurity-apache_2.5.12/rules/base_rules/* /usr/local/apache/conf/modsecurity/

Note : To test your rules you can set SecRuleEngine DetectionOnly in modsecurity_crs_10_config.conf file ( location:/usr/local/apache/conf/modsecurity/) It will show you how all those rules are performing.

Now Stop and restart apache again, and look at error_log, access_log for modsecurity activity

Prefix for modsecurity installation

-–with-apxs=FILE FILE is the path to apxs; defaults to “apxs”.
-–with-pcre=PATH Path to pcre prefix or config script
-–with-apr=PATH Path to apr prefix or config script
-–with-apu=PATH Path to apu prefix or config script
-–with-libxml=PATH Path to libxml2 prefix or config script
-–with-lua=PATH Path to lua prefix or config script (optional)
-–with-curl=PATH Path to curl prefix or config script (optional)

Extra Notes :
1. Please create a directory “mkdir /usr/local/apache/conf/modsecurity/data”
and add bellow lines in modsecurity_crs_10_config.conf

  SecDataDir /usr/local/apache/conf/modsecurity/data

If this lines is missing you might see this kind of error :
[Thu Dec 10 10:10:54 2009] [error] [client xx.xx.xx.xx] ModSecurity: Unable to retrieve collection (name “ip”, key “xx.xx.xx.xx”). Use SecDataDir to define data directory first. [hostname “xx.xx.xx.xxx”] [uri “/”] [unique_id “SyC7Hn8AAAEAABLHj9gAAAAL”]

Yum repo list for Centos

Monday, December 14th, 2009

For 32 bit kernel

Repo1:

Download :
 
wget -c http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm
 
Install : rpm -Uvh epel-release-5-3.noarch.rpm

Repo2:

Download :
 
wget -c  http://apt.sw.be/redhat/el5/en/i386/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.i386.rpm
 
Install :
 
rpm -Uvh rpmforge-release-0.3.6-1.el5.rf.i386.rpm

For 64bit kernel :

Repo1:

Download :
 
wget -c http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-3.noarch.rpm
 
Install : rpm -Uvh epel-release-5-3.noarch.rpm

Repo2:

Download :
 
wget -c  http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.1-1.el5.rf.x86_64.rpm
Install :
 
rpm -Uvh rpmforge-release-0.5.1-1.el5.rf.x86_64.rpm

How to display security updates by yum

Monday, December 7th, 2009

Ref: http://magazine.redhat.com/2008/01/16/tips-and-tricks-yum-security/

Ref: http://www.cyberciti.biz/faq/redhat-fedora-centos-linux-yum-installs-security-updates/

Install Plugin

Type the following command:
# yum install yum-security

How Do I Display Available Security Updates?

Type the following command:
# yum list-security
Sample Outputs:

Loaded plugins: rhnplugin, security
RHSA-2009:1148-1 security httpd-2.2.3-22.el5_3.2.x86_64
RHSA-2009:1148-1 security httpd-devel-2.2.3-22.el5_3.2.i386
RHSA-2009:1148-1 security httpd-manual-2.2.3-22.el5_3.2.x86_64
RHSA-2009:1148-1 security mod_ssl-1:2.2.3-22.el5_3.2.x86_64
list-security done

To list all updates that are security relevant, and get a reutrn code on whether there are security updates use:
# yum --security check-update
To get a list of all BZs that are fixed for packages you have installed use:
# yum list-security bugzillas
To get the information on advisory RHSA-2009:1148-1 use:
# yum info-security RHSA-2009:1148-1
Sample Outputs:

Loaded plugins: rhnplugin, security

===============================================================================
  RHSA-2009:1148
===============================================================================
  Update ID : RHSA-2009:1148-1
    Release :
       Type : security
     Status : final
     Issued : 2009-07-08 23:00:00
       Bugs : 509125 - None
	    : 509375 - None
       CVEs : CVE-2009-1890
	    : CVE-2009-1891
Description : Important: httpd security update  \The Apache HTTP Server is a
            : popular Web server.  A denial of service flaw was
            : found in the Apache mod_proxy module when it was
            : used as a reverse proxy. A remote attacker could
            : use this flaw to force a proxy process to consume
            : large amounts of CPU time. (CVE-2009-1890)  A
            : denial of service flaw was found in the Apache
            : mod_deflate module. This module continued to
            : compress large files until compression was
            : complete, even if the network connection that
            : requested the content was closed before
            : compression completed. This would cause
            : mod_deflate to consume large amounts of CPU if
            : mod_deflate was enabled for a large file.
            : (CVE-2009-1891)  All httpd users should upgrade to
            : these updated packages, which contain backported
            : patches to correct these issues. After installing
            : the updated packages, the httpd daemon must be
            : restarted for the update to take effect.
      Files : mod_ssl-2.2.3-22.el5_3.2.x86_64.rpm
	    : httpd-devel-2.2.3-22.el5_3.2.i386.rpm
	    : httpd-2.2.3-22.el5_3.2.x86_64.rpm
	    : httpd-devel-2.2.3-22.el5_3.2.x86_64.rpm
	    : httpd-manual-2.2.3-22.el5_3.2.x86_64.rpm
	    : mod_ssl-2.2.3-22.el5_3.2.i386.rpm
	    : httpd-2.2.3-22.el5_3.2.i386.rpm
	    : httpd-manual-2.2.3-22.el5_3.2.i386.rpm
info-security done

Ref:http://www.cyberciti.biz/faq/redhat-fedora-centos-linux-yum-installs-security-updates/

To get an info list of the latest packages which contain fixes for Bugzilla 3595; CVE # CVE-2009-1890 and advisories RHSA-2009:1148-1, use:
# yum --bz 3595 --cve CVE-2009-1890 --advisory RHSA-2009:1148-1 info updates

How Do I Install All The Security Updates Only?

Type the following command to download and install all the available security updates:
# yum update --security

Mysql server master master active active replication

Tuesday, November 24th, 2009

Ref: http://www.howtoforge.com/mysql_master_master_replication

a) Create user name and password for replication on both servers by using this command

GRANT REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO noslave@’host-name’ IDENTIFIED BY ’some-pass’;

b) Configuration for Server 1 To make it primary Server for Server2

vi /etc/my.cnf

log-bin=mysql-bin
binlog-do-db=fosiul # which Database to replicate
binlog-do-db=hesk # Which Database to replicate
binlog-ignore-db=mysql # Which Database to ignore
binlog-ignore-db=test # Which Database to ignore
server-id = 1 # Primary Server id
auto_increment_increment= 2 # to solved the issue for auto indexing problem
auto_increment_offset = 1 # to solved the issue for auto indexing problem

Configuration For server2 to make as Slave for Server1

server-id = 2

master-host = IP_Of_Server1
master-user =noslave
master-password = SomeStrongPassword
master-port = 3306
auto_increment_increment= 2 # Avoid Auto Indexing problem
auto_increment_offset = 2

Now restart both Server and look for bellow report:

For Server 1 (Master Report):

mysql> show master status;
+——————+———-+————–+——————+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+——————+———-+————–+——————+
| mysql-bin.000008 | 565444 | fosiul,hesk | mysql,test |
+——————+———-+————–+——————+
1 row in set (0.00 sec)

For Server2:( Slave report)

mysql> show slave status\G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: IP-Of-Server1
Master_User: noslave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000008
Read_Master_Log_Pos: 565444
Relay_Log_File: web-relay-bin.000092
Relay_Log_Pos: 153971
Relay_Master_Log_File: mysql-bin.000008
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 565444
Relay_Log_Space: 154124
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
1 row in set (0.00 sec)

ERROR:
No query specified

note :

a) Make sure Master_Log_File: mysql-bin.000008 From Slave Report matches with the Master_Log_file name with Master Reports.

b)Make sure Read_Master_Log_Pos: 565444 at Slave Report matches with Position field at Master Reports

c) Make sure Seconds_Behind_Master is always 0(Zero)

Click on the picture to view

Verify Log File

Master(Server1) Slave (Server2)

c)Configuration for server 2 as Master for Server1

#Bellow section for acting as Master for server1

log-bin=mysql-bin

binlog-do-db=fosiul # Which Database to repliacate
binlog-do-db=hesk # Which Database to replicate
binlog-ignore-db=mysql # Which Database to ignore
binlog-ignore-db=test # Which Database to ignore

#Configuration for Server1 to make as slave for Server 2

master-host = IP-Of-Server2
master-user = noslave
master-password = SomeSTrongPassowrd
master-port = 3306
log-slave-updates # To make this Master Server act as slave

d) Now Restart both mysql server and look for bellow report:

Slave Status report for Server1

mysql> show slave status\G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: IP_OF_Sever2
Master_User: noslave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000006
Read_Master_Log_Pos: 106
Relay_Log_File: mail-relay-bin.000025
Relay_Log_Pos: 251
Relay_Master_Log_File: mysql-bin.000006
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 106
Relay_Log_Space: 550
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
1 row in set (0.00 sec)

Master Report for Server2 :

mysql> show master status;
+——————+———-+————–+——————+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB |
+——————+———-+————–+——————+
| mysql-bin.000006 | 106 | fosiul,hesk | mysql,test |
+——————+———-+————–+——————+
1 row in set (0.01 sec)

Note :

a) Make sure Master_Log_File: mysql-bin.000006 From Slave Report matches with the Master_Log_file name with Master Report.

b)Make sure Read_Master_Log_Pos: 106 at Slave Report matches with Position field at Master Report.

c) Make sure Seconds_Behind_Master is always 0(Zero)

Click on the picture to view

Slave(Server1) and Master (Server2)

Slave(Server1) and Master (Server2)

How to install innotop

Thursday, November 19th, 2009

a)Download innotop from http://code.google.com/p/innotop/

b)cd /tmp

c) tar -xvzf innotop-1.7.2.tar.gz

d) cd innotop-1.7.2

e) perl per Makefile.PL

f) Make install

NOte : if you see error like this :

Looks good
Warning: prerequisite DBD::mysql 1 not found.
Warning: prerequisite DBI 1.13 not found.
Warning: prerequisite Term::ReadKey 2.1 not found.
Writing Makefile for innotop

Solution :

yum install perl-DBD-MySQL

yum install perl-TermReadKey

Run innotop : perl /usr/bin/innotop –password “your password”

How to install chkrootkit/rootkit hunter

Thursday, November 19th, 2009

a) Download latest rootkithunter from http://www.chkrootkit.org/download/ (latest version is chkrootkit-0.49 but its has bugs)

mv chkrootkit.tar.gz /usr/local/
cd /usr/local/
tar xvfz chkrootkit.tar.gz
ln -s chkrootkit-0.43/ chkrootkit
(replace 0.43 with the right version number)
cd chkrootkit/
make sense

You will now find the chkrootkit program under /usr/local/chkrootkit. Run it by typing

cd /usr/local/chkrootkit/ && ./chkrootkit

How to install portsentry

Thursday, November 19th, 2009

Install PortsEntry

Portsentry is a tool to detect port scans and log it. Download the sorce package of portsentry from sourceforge.net

wget http://path/to/portsentry-1.2.tar.gz
tar zxf portsentry-1.2.tar.gz
make linux
make install

If you get errors like while compiling

make linux
SYSTYPE=linux
Making
gcc -O -Wall -DLINUX -DSUPPORT_STEALTH -o ./portsentry ./portsentry.c \
./portsentry_io.c ./portsentry_util.c
./portsentry.c: In function ‘PortSentryModeTCP’:
./portsentry.c:1187: warning: pointer targets in passing argument 3 of ‘accept’ differ in signedness
./portsentry.c: In function ‘PortSentryModeUDP’:
./portsentry.c:1384: warning: pointer targets in passing argument 6 of ‘recvfrom’ diffe r in signedness
./portsentry.c: In function ‘Usage’:
./portsentry.c:1584: error: missing terminating ” character
./portsentry.c:1585: error: ‘sourceforget’ undeclared (first use in this function)
./portsentry.c:1585: error: (Each undeclared identifier is reported only once
./portsentry.c:1585: error: for each function it appears in.)
./portsentry.c:1585: error: expected ‘)’ before ‘dot’
./portsentry.c:1585: error: stray ‘\’ in program
./portsentry.c:1585: error: missing terminating ” character
./portsentry.c:1595: error: expected ‘;’ before ‘}’ token
make: *** [linux] Error 1

To fix:

Open portsentry.c and look for the following line. There will be a extra carriage return breaking the line and you have to delete the carriage return and make single line. It should look like below.

printf (“Copyright 1997-2003 Craig H. Rowland <craigrowland at users dot sourceforget dot net>\n”);

Then run make and make install. That should fix it!

To launch portsentry

/usr/local/psionic/portsentry/portsentry -stcp
/usr/local/psionic/portsentry/portsentry -sudp

check the log files /var/log/secure or /var/log/messages on what portsentry is active or not.

Invalid method in request \x80O\x01\x03

Wednesday, November 18th, 2009

Make sure the IP of the server and the Ip in Virutal host (ssl configuraiton ) are same.

<VirtualHost xx.xx.xx.xx:443>

</VirtualHost>

How to configure nagios to work with apache source install(/usr/local/apache)

Monday, November 16th, 2009

Ref:http://nagios.sourceforge.net/docs/3_0/quickstart-fedora.html

Basic nagios setup works well if you install apache server by yum. But if you install and configure Apache from source to run different directory(i.e /usr/local/apache) other then /etc/httpd/conf then then the default nagios web interface would not work because , by default nagios creates nagios.conf file in /etc/httpd/conf.d directory for fedora.

So if you have already installed apache from source then do the following:

Follow step 1 to 5 as documented in nagios website (Except htpasswd section)

(a)

I assume , you have configured your Apache to run from /usr/local/apache , and your apache configuration file is in /usr/local/apache/conf/

copy nagios.conf file from /etc/httpd/conf.d to /usr/local/apache/conf/extra

cp /etc/httpd/conf.d/nagios.conf /usr/local/apache/conf/extra/

Edit httpd.conf which located in /usr/local/apache/conf/httpd.conf and add the bellow line

Include conf/extra/nagios.conf

(b)

Create a nagiosadmin account for logging into the Nagios web interface. Remember the password you assign to this account – you’ll need it later.

/usr/local/apache/bin/htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

Restart Apache to make the new settings take effect.

stop the apache server by /usr/local/apache/bin/apachectl stop

start the Apache server by /usr/local/apache/bin/apachectl start

Then Follow rest of the steps .

Extra note:

if you did install apache by yum , then you might see error like bellow :

[root@web nagios-3.2.0]# make install-webconf
/usr/bin/install -c -m 644 sample-config/httpd.conf /etc/httpd/conf.d/nagios.conf
/usr/bin/install: cannot create regular file `/etc/httpd/conf.d/nagios.conf’: No such file or directory
make: *** [install-webconf] Error 1

Solution : mkdir /etc/httpd/conf.d , now run

make install-webconf

So it will install nagios.conf file in /etc/httpd/conf.d directory. Now follow form (a) to (b)

Note : I am assuming you have configured apache to install on /usr/local/apache directory

How to compile php for GD library

Friday, November 13th, 2009

Install necessary software by yum or from source

yum install gd gd-devel yum install zlib zlib-devel

then

a) Download the php source from here : http://www.php.net/downloads.php
b) Download the source file in to /tmp directory
c) Here I am gussing the php version is php-5.3.0.tar.gz
d) Tar –xvzf php-5.3.0.tar.gz
e) Cd php-5.3.0
f)

 ./configure --with-apxs2=/usr/local/apache/bin/apxs --with-mysql=/usr/local/mysql --enable-mbstring –-with-gd –-with-zlib –-with-jpeg-dir-with-png-dir

g) make
h) make install
i) setup your php.ini : cp php.ini-dist /usr/local/lib/php.ini
j) Stop apache /usr/local/apache/bin/apachectl1 stop
k) Restart apache /usr/local/apache/bin/apachectl1 start

How to check :

create a file phpinfo.php

<?php
phpinfo();
?>

Open the file in your browser , http://localhost/phpinfo.php

and Look for 2 section , GD and ZLIB. You should see something like bellow pictures

Linux performance tuning tools (vmstat tool)

Wednesday, September 23rd, 2009

Ref: Performance Tuning for Linux® Servers

Ref: System Performance Tuning, Second Edition

Ref: Optimizing Linux® Performance: A Hands-On Guide to Linux® Performance Tools

Linux Performance Tools:

Processor time is organized into four timed modes: system time, user time, I/O wait time, and idle time. The idle time consists of what’s left over when all other portions have had their fill. A program’s normal operating state is user mode, but as it runs, it may generate requests for services that the kernel provides, such as I/O. These requests require the attention of the operating system, so the program switches into system mode, then returns to user mode when the request is complete. The time spent in these two modes is tabulated independently to give the user time and system time values, respectively. These two figures account for the majority of a process’s execution time.

Note that vmstat reports only the user time, system time, and idle time (wait time is summed in with idle time). In order to get separated values for wait time and idle time, use mpstat.

When a process waits for a block device data request to complete, it incurs I/O wait time. This brings up an important fact: when a process is blocked in this fashion, all idle time becomes wait time. If your idle time is zero, as reported by vmstat, the first thing you should check is if your system has I/O throughput problems.

vmstat (Virtual Memory Statistics):

  • How many processes are running
  • How the CPU is being used
  • How many interrupts the CPU receives
  • How many context switches the scheduler performs
vmstat [-n] [-s] [delay [count]]
vmstat 2 5
Column Explanation
r This is the number of currently runnable processes. These processes are not waiting on I/O and are ready to run. Ideally, the number of runnable processes would match the number of CPUs available.

Performance hits: If the run queue is consistently at or more than four times the number of configured processors in the system, you should probably consider increasing the available processors.

b This is the number of processes blocked and waiting for I/O to complete.
forks The is the number of times a new process has been created. Command : vmstat -f
in This is the number of interrupts occurring on the system.
cs This is the number of context switches happening on the system.
us The is the total CPU time as a percentage spent on user processes (including “nice” time).
sy The is the total CPU time as a percentage spent in system code. This includes time spent in the system, irq, and softirq state.
wa The is the total CPU time as a percentage spent waiting for I/O.
id The is the total CPU time as a percentage that the system is idle.

Sample output:

[ezolt@scrffy tmp]$ vmstat
procs -----------memory---------- ---swap-- -----io---- --system--
----cpu----
r  b   swpd   free   buff  cache   si   so    bi    bo   in    cs us sy id wa
1  0 181024  26284  35292 503048    0    0     3     2    6      1  5  1 94  0

What to look for:

One key to focus on is the (wa), as consistent high numbers here is definitely a problem waiting on I/O. Usually seeing consistent (wa) in 10+ will show degradation of the system. Once it reaches 35+ you will not need to look at the statistics as your users will be complaining.

Look for high numbers in either system or user space. If you see consistent high numbers for user space it could be an application that has a process that is consuming too much resources. In this case look at top to see if you can identify the problem process. If you see consistent high numbers for system then you also need to look into what programs are taking the CPU resources and evaluate their status

To be continue…

How to install apache2-php-mysql from source

Thursday, September 10th, 2009

Prerequisite : yum install gcc-c++ gcc make ncurses-devel openssl-devel glibc* libc-*

Packages required for php: yum install libjpeg-devel libpng-devel curl-devel libmcrypt-devel krb5-devel

Apache Server Installation from Source:
Apache installation directory is : /usr/local/apache
a) Download the apache source file from : http://httpd.apache.org/download.cgi
b) Download the source file in to /tmp directory.
c) I am guessing the source file is httpd-2.2.13.tar.gz
d) Cd /tmp
e) tar –xvzf httpd-2.2.13.tar.gz
f) cd httpd-2.2.13

g)

 
./configure  --prefix=/usr/local/apache --with-included-apr --with-php --with-mysql --with-susexec --disable-info --with-mpm=prefork --enable-so --enable-cgi --enable-rewrite --enable-ssl --enable-mime-magic --enable-unique-id --enable-mods-shared="proxy cache ssl all"

h) make
i) make install
j)To restart apache : /usr/local/apache/bin/apachectl start

MySql Server Install from source:

Ref:http://dev.mysql.com/doc/refman/5.1/en/quick-install.html

a)Download my.version.tar.gz from
http://dev.mysql.com/downloads/mysql/5.1.html#source

b)shell> groupadd mysql
c)shell> useradd -g mysql mysql
d)shell> gunzip < mysql-VERSION.tar.gz | tar -xvf – e)shell> cd mysql-VERSION
f)

   ./configure --prefix=/usr/local/mysql --with-ssl --with-plugins=innobase

note:: for mysql 5.1 : to add innodb support its ” –with-plugins=innobase” but for 5.0 its “./configure –with-innodb”
g)shell> make
h)shell> make install
i)shell> cp support-files/my-medium.cnf /etc/my.cnf
j)shell> cd /usr/local/mysql
k)shell> chown -R mysql .
l)shell> chgrp -R mysql .
m)shell> bin/mysql_install_db -–user=mysql
n)shell> chown -R root .
o)shell> chown -R mysql var
p)shell> bin/mysqld_safe -–user=mysql &

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
./bin/mysqladmin -u root password 'new-password'

Php installation from source with GD library Support

http://www.php.net/manual/en/install.unix.apache2.php

a) Download the php source from here : http://www.php.net/downloads.php
b) Download the source file in to /tmp directory
c) Here I am gussing the php version is php-5.3.0.tar.gz
d) Tar –xvzf php-5.3.0.tar.gz
e) Cd php-5.3.0
f)

./configure --with-apxs2=/usr/local/apache/bin/apxs --with-mysql=/usr/local/mysql --enable-mbstring --with-gd --with-zlib --with-jpeg-dir --with-png-dir --with-openssl --with-curl --with-mcrypt --with-imap --with-imap-ssl --with-kerberos --with-mysqli=/usr/local/mysql/bin/mysql_config

g) make
h) make install
i) setup your php.ini : cp php.ini-dist /usr/local/lib/php.ini

j) In httpd.conf file.. check for bellow lines

LoadModule php5_module modules/libphp5.so

j) Add the bellow lines in httpd.conf file to allow .php extension.
add bellow lines under directive

Add php extension

 
<FilesMatch "\.phps$">
          SetHandler application/x-httpd-php-source
      </FilesMatch>
 
 <FilesMatch "\.ph(p[2-6]?|tml)$">
          SetHandler application/x-httpd-php
      </FilesMatch>

j) Stop apache /usr/local/apache/bin/apachectl1 stop
k) Restart apache /usr/local/apache/bin/apachectl1 start

Note :

(a) configure: error: xml2-config not found. Please check your libxml2 installation. : yum install libxml2-devel

(b) configure: error: libpng.(a|so) not found.
configure: error: libjpeg.(a|so) not found.
(c) Error : configure: error: utf8_mime2text() has new signature, but U8T_CANONICAL is missing
yum install libc-client-devel*
So it will try to find accurate rpm for your kernel(32/64)

(d) If you have older httpd daemon running , please stop that daemon,Other wise when you will start apache daemon, it will through an error .You can check by bellow command to make sure you don’t have any other httpd is running in background.

ps aux | grep -v grep | grep httpd

If this returns value that means another httpd daemon is running and you can stop it by executing

service httpd stop

Last Update : 14-09-2010

How to backup linux Server remotely

Monday, September 7th, 2009

Bellow article will show how to backup a Linux server remotely by using RSYNC with public key base authentication.

To make this backup process automatic we need a password less authentication system so that we don’t have to insert username and password before backup starts.

How to create Key base authentication:

Here are the steps you need to do on the computer that acts as the SSH client:

1) Generate your SSH encryption key pair for the filecopy account. Press the Enter key each time you are prompted for a password to be associated with the keys. (Do not enter a password.)

[filecopy@bigboy filecopy]# ssh-keygen -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key
(/filecopy/.ssh/id_dsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in
/filecopy/.ssh/id_dsa.
Your public key has been saved in
/filecopy/.ssh/id_dsa.pub.
The key fingerprint is:
1e:73:59:96:25:93:3f:8b:50:39:81:9e:e3:4a:a8:aa
filecopy@bigboy
[filecopy@bigboy filecopy]#

2) These keyfiles are stored in the.ssh subdirectory of your home directory. View the contents of that directory. The file named id_dsa is your private key, and id_dsa.pub is the public key that you will be sharing with your target server. Versions other than RedHat/Fedora may use different filenames, use the SSH man pages to verify this.

[filecopy@bigboy filecopy]# cd ~/.ssh
[filecopy@bigboy filecopy]# ls
id_dsa  id_dsa.pub  known_hosts
[filecopy@bigboy .ssh]#

3) Copy only the public key to the home directory of the account to which you will be sending the file.

[filecopy@bigboy .ssh]# scp id_dsa.pub filecopy@smallfry:public-key.tmp

Now, on to the server side of the operation.

Configuration – Server Side

Here are the steps you need to do on the computer that will act as the SSH server.

1) Log into smallfry as user filecopy. Create an .ssh subdirectory in your home directory and then go to it with cd.

[filecopy@smallfry filecopy]# ls
public-key.tmp
[filecopy@smallfry filecopy]# mkdir .ssh
[filecopy@smallfry filecopy]# chmod 700 .ssh
[filecopy@smallfry filecopy]# cd .ssh

2) Append the public-key.tmp file to the end of the authorized_keys file using the >> append redirector with the cat command. The authorized_keys file contains a listing of all the public keys from machines that are allowed to connect to your Smallfry account without a password. Versions other than RedHat/Fedora may use different filenames, use the SSH man pages to verify this.

[filecopy@smallfry .ssh]# cat ~/public-key.tmp >> authorized_keys
[filecopy@smallfry .ssh]# rm ~/public-key.tmp

From now on you can use ssh and scp as user filecopy from server bigboy to smallfry without being prompted for a password.

2. Taking Backup by Rsync

Write a Backup script Example : backup.sh in /root directory

cd /root

vi backp.sh

Press I to Insert

then Write like this :

#!/bin/bash

DESTROOT=”/backups”
TODAY=`date ‘+%A’`

rsync -e ssh -avz –delete filecopy@smallfry:/var/www $DESTROOT/backup

#Archive todays files
tar czvf $DESTROOT/archived/${TODAY}-backup.tar.gz $DESTROOT/backup > $DESTROOT/archived/${TODAY}-backup.log

Press :wq [ to Save the file and exit]

then : chmod 700 backup.sh [ so it will make this file executable for root]

Explanation : The Script will connect to the server smallfry via Ssh, then will download all the files from /var/www directory to local pc under /backups Directory.

3. Automate the system by Crontab

crontab -e

Press I to insert
00 3 * * 1-5 /root/backup.sh >/dev/null 2>&1

Press :wq [ to save the file]

Explanation : Now crontab will execute this file First minute 3 am Every day Every month Monday to Friday .

How to install mod_security by yum(Redhat-Centos 5)

Friday, August 28th, 2009

1.Download the EPEL repo :

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-3.noarch.rpm

2.Then type the following command :

yum install mod_security

Note : Mod_security require liblua-5.1.so, If you don’t have this , it will throw an error while installing by yum.


--> Processing Dependency: liblua-5.1.so for package: mod_security
--> Finished Dependency Resolution
mod_security-2.5.9-1.el5.i386 from epel has depsolving problems
--> Missing Dependency: liblua-5.1.so is needed by package mod_security-2.5.9- 1.el5.i386 (epel)
Error: Missing Dependency: liblua-5.1.so is needed by package mod_security-2.5.9 -1.el5.i386 (epel)

Solution: You can download the rpm from this website

http://rpm.pbone.net/index.php3/stat/4/idpl/12580541/com/lua-5.1.4-1.i386.rpm.html

If your server complain you have installed already newer version then you can reinstall the installed version by using

-bash-3.2# rpm -qa | grep lua
lua-5.1.4-1.el5.rf
-bash-3.2# rpm -e lua-5.1.4-1.el5.rf
-bash-3.2# rpm -Uvh lua-5.1.4-1.i386.rpm
Preparing… ########################################### [100%]
1:lua ########################################### [100%]

Now type

-bash-3.2# updatedb

-bash-3.2# locate liblua-5.1.so
/usr/lib/liblua-5.1.so

So it shows that your server has the required file for it to install mod_security

Now run

yum install mod_security

It should installed now
mod_security configuration files

  1. /etc/httpd/conf.d/mod_security.conf – main configuration file for the mod_security Apache module.
  2. /etc/httpd/modsecurity.d/ – all other configuration files for the mod_security Apache.
  3. /etc/httpd/modsecurity.d/modsecurity_crs_10_config.conf – Configuration contained in this file should be customized for your specific requirements before deployment.
  4. /var/log/httpd/modsec_debug.log – Use debug messages for debugging mod_security rules and other problems.
  5. /var/log/httpd/modsec_audit.log – All requests that trigger a ModSecurity events (as detected) or a serer error are logged (“RelevantOnly”) are logged into this file.

After installing mod_security , Edit modsecurity_crs_10_config.conf file and make sure

bellow line is enabled.

SecRuleEngine On

Now restart the httpd server by

service httpd restart

Check the /var/log/httpd/error_log for this lines

[Fri Aug 28 10:48:24 2009] [notice] ModSecurity for Apache/2.5.9 (http://www.mod security.org/) configured.

Note : I have tested this on Centos5 (2.6.18-128.1.14.el5xen).

Ref:http://www.cyberciti.biz/faq/rhel-fedora-centos-httpd-mod_security-configuration/

Ref:http://www.modsecurity.org/documentation/

Tools for securing Linux server and its services

Thursday, August 27th, 2009

a) Fail2Ban: Which will ban IP address after few failure attempts

website : http://www.fail2ban.org/wiki/index.php/Main_Page

b)Rootkit Hunter : It will scan your server for any unauthorized scripts.

Website :http://www.chkrootkit.org/

To download : http://sourceforge.net/projects/rkhunter/

c)PortSentry : This tool will block IP who is trying to scan your server for open ports.

Ref : http://www.securityfocus.com/infocus/1580

http://www.securityfocus.com/infocus/1586

How to install : http://www.falkotimme.com/howtos/chkrootkit_portsentry/

d)mod_security

http://www.modsecurity.org/

e) mod_evasive : It will ban IP for certain conditions ,example :

  • Requesting the same page more than a few times per second
  • Making more than 50 concurrent requests on the same child per second
  • Making any requests while temporarily blacklisted (on a blocking list)

Web Site : http://www.zdziarski.com/projects/mod_evasive/

To be.. continue..

Bash script to check for automatic yum updates

Thursday, August 27th, 2009
#!/bin/bash
########################################
# This script will check for available package-
# update for Centos/Redhat 5 sytem
# Written by        : Fosiul Alam
# Version           :1.0
# Created Date      : 27/08/2009
# Last Modification : 27/08/2009
#########################################
 
########################################
# Command use       : yum -e0 -d0 check-update
# Usage             : /yum-script.sh
########################################
 
_GET_HOSTNAME=`hostname`
_TODAY=`date '+%A'`
_YESTERDAY=`date '+%A' --date='1 day ago'`
_CMD_FILE=/tmp/tmp.txt.$_TODAY
_CMD_YESTERDAY_FILE=/tmp/tmp.txt.$_YESTERDAY
_EMAIL_REPORTS=/tmp/yum-reports.txt
 
#Delete Yesterday's tmp.txt file( Housekeeping)
if  [ -e $_CMD_YESTERDAY_FILE ]
then
rm $_CMD_YESTERDAY_FILE
fi
 
#check if yum-reports.txt file exists or not
 
if [ -e $_EMAIL_REPORTS ]
#if the file exists then delete the file
then
rm $_EMAIL_REPORTS
#Create the file again
touch $_EMAIL_REPORTS
#Initialize yum command into the variables
yum -e0 -d0 check-update &gt;$_CMD_FILE
 
#Check if file is exists and not  empty
 
if [ -s $_CMD_FILE ]
 
then
#_EMAIL_REPORTS=/tmp/yum-reports.txt
echo "Daily($_TODAY) Yum Updates Reports for $_GET_HOSTNAME " &gt;&gt; $_EMAIL_REPORTS
echo "There are some updates availabe for your attention" &gt;&gt; $_EMAIL_REPORTS
echo "###########Updates Are###########" &gt;&gt; $_EMAIL_REPORTS
cat $_CMD_FILE &gt;&gt; $_EMAIL_REPORTS
echo "############## Updates Finished#####" &gt;&gt; $_EMAIL_REPORTS
cat $_EMAIL_REPORTS | mail -s "Yum Reports For $_GET_HOSTNAME " fosiul@gmail.com
 
else
echo " NO Updates for $_TODAY" &gt;/dev/null 2&gt;&amp;1
 
fi
 
fi

All about yum command for Redhat/Centos/Fedora

Wednesday, August 26th, 2009

Display List of updated software ( Security fix)

yum list updates
Or
yum check-update

Patch up system by applying all updates

yum update

List all installed packages

rpm -qa

How to find a perticular installed packages( httpd)

rpm -qa | grep httpd

How to look update for specific packages

yum update {package-name-1}

To check for and update httpd package, enter:

yum update httpd

How to install packages by yum

yum install package-name
example : yum install httpd

How to exclude package name from update

yum --exclude=packagename* update

How to check for yum updates automatically by bash script:
http://fosiul.co.uk/index.php/2009/08/bash-script-to-check-for-automatic-yum-updates/
Will be continue….

reverse proxying with apache

Wednesday, August 26th, 2009

Ref: http://httpd.apache.org/docs/2.2/mod/mod_proxy.html

Module : mod_proxy.c

In httpd.conf , under bellow section all reverse proxy rules will go

<IfModule mod_proxy.c>
#ProxyRequests On

ProxyRequests Off
#
#<Proxy *>
# Order deny,allow
# Deny from all
# Allow from .example.com
#</Proxy>

#
# Enable/disable the handling of HTTP/1.1 “Via:” headers.
# (“Full” adds the server version; “Block” removes all outgoing Via: headers)
# Set to one of: Off | On | Full | Block
#
#ProxyVia On

#
# To enable a cache of proxied content, uncomment the following lines.
# See http://httpd.apache.org/docs/2.2/mod/mod_cache.html for more details.
#
#<IfModule mod_disk_cache.c>
# CacheEnable disk /
# CacheRoot “/var/cache/mod_proxy”
#</IfModule>
#
#Add the Reverse Proxy rules

ProxyPass /foo http://foo.example.com/bar
ProxyPassReverse /foo http://foo.example.com/bar

</IfModule>

#End of proxy directives.

Note : Make sure if you use reverse proxy then ProxyRequests is Off.

How to allow perl/cgi script to run from virtualhost

Wednesday, August 26th, 2009

Ref: http://httpd.apache.org/docs/2.0/howto/cgi.html

If you want to run a perl script like this http://www.mydomain.com/test.pl , you will have to to define explicitly use the Options directive, inside your main server configuration file, to specify that CGI execution was permitted in a particular directory:

Example:

<VirtualHost *:80>
ServerAdmin adin@mydomain.co.uk
DocumentRoot /var/www/html/mydomain/
ServerName mydomain.co.uk
ServerName www.mydomain.co.uk
ErrorLog logs/mydomain.co.uk-error_log
CustomLog logs/mydomain.co.uk-access_log common

<Directory “/var/www/html/mydomain/”>
Options FollowSymLinks ExecCGI
AllowOverride None
Order allow,deny
Allow from all

</Directory>
</VirtualHost>

The above directive tells Apache to permit the execution of CGI files.

You will also need to tell the server what files are CGI files. The following AddHandler directive tells the server to treat all files with the cgi or pl extension as CGI programs:

AddHandler cgi-script .cgi .pl

Now Save the configuration file and Restart apache.

vsftpd: Failed to retrieve directory listing

Friday, August 14th, 2009

Problem: If vsftpd failed to retrieves directory listing then do the following:

In vsftpd.conf

pasv_min_port=x
example : [ pasv_min_port=1023]

pasv_max_port=x
example : [pasv_max_port=1050]

Now add port 1023-1050 in iptables

iptables -A INPUT –source xx.xx.xx.xx -p tcp –dport 1023:1050 -j ACCEPT

it will allow filezilla to connect to ftp server via passive mode.

How to Rebuilding failed Linux software RAID

Friday, August 14th, 2009

Ref: http://aplawrence.com/Linux/rebuildraid.html

Recently I had a hard drive fail. It was part of a Linux software RAID 1 (mirrored drives), so we lost no data, and just needed to replace hardware. However, the raid does requires rebuilding. A hardware array would usually automatically rebuild upon drive replacement, but this needed some help.

When you look at a “normal” array, you see something like this:

# cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md2 : active raid1 hda3[1] hdb3[0]
262016 blocks [2/2] [UU]

md1 : active raid1 hda2[1] hdb2[0]
119684160 blocks [2/2] [UU]

md0 : active raid1 hda1[1] hdb1[0]
102208 blocks [2/2] [UU]

unused devices:

That’s the normal state – what you want it to look like. When a drive has failed and been replaced, it looks like this:

Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 hda1[1]
102208 blocks [2/1] [_U]

md2 : active raid1 hda3[1]
262016 blocks [2/1] [_U]

md1 : active raid1 hda2[1]
119684160 blocks [2/1] [_U]
unused devices:

Notice that it doesn’t list the failed drive parts, and that an underscore appears beside each U. This shows that only one drive is active in these arrays – we have no mirror.

Another command that will show us the state of the raid drives is “mdadm”

# mdadm -D /dev/md0
/dev/md0:
Version : 00.90.00
Creation Time : Thu Aug 21 12:22:43 2003
Raid Level : raid1
Array Size : 102208 (99.81 MiB 104.66 MB)
Device Size : 102208 (99.81 MiB 104.66 MB)
Raid Devices : 2
Total Devices : 1
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Fri Oct 15 06:25:45 2004
State : dirty, no-errors
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

Number Major Minor RaidDevice State
0 0 0 0 faulty removed
1 3 1 1 active sync /dev/hda1
UUID : f9401842:995dc86c:b4102b57:f2996278

As this shows, we presently only have one drive in the array.

Although I already knew that /dev/hdb was the other part of the raid array, you can look at /etc/raidtab to see how the raid was defined:

raiddev /dev/md1
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda2
raid-disk 0
device /dev/hdb2
raid-disk 1
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda1
raid-disk 0
device /dev/hdb1
raid-disk 1
raiddev /dev/md2
raid-level 1
nr-raid-disks 2
chunk-size 64k
persistent-superblock 1
nr-spare-disks 0
device /dev/hda3
raid-disk 0
device /dev/hdb3
raid-disk 1

To get the mirrored drives working properly again, we need to run fdisk to see what partitions are on the working drive:

# fdisk /dev/hda

Command (m for help): p

Disk /dev/hda: 255 heads, 63 sectors, 14946 cylinders
Units = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System
/dev/hda1 * 1 13 104391 fd Linux raid autodetect
/dev/hda2 14 14913 119684250 fd Linux raid autodetect
/dev/hda3 14914 14946 265072+ fd Linux raid autodetect

Duplicate that on /dev/hdb. Use “n” to create the parttions, and “t” to change their type to “fd” to match. Once this is done, use “raidhotadd”:

# raidhotadd /dev/md0 /dev/hdb1
# raidhotadd /dev/md1 /dev/hdb2
# raidhotadd /dev/md2 /dev/hdb3

The rebuilding can be seen in /proc/mdstat:

# cat /proc/mdstat
Personalities : [raid1]
read_ahead 1024 sectors
md0 : active raid1 hdb1[0] hda1[1]
102208 blocks [2/2] [UU]

md2 : active raid1 hda3[1]
262016 blocks [2/1] [_U]

md1 : active raid1 hdb2[2] hda2[1]
119684160 blocks [2/1] [_U]
[>………………..] recovery = 0.2% (250108/119684160) finish=198.8min speed=10004K/sec
unused devices:

The md0, a small array, has already completed rebuilding (UU), while md1 has only begun. After it finishes, it will show:

# mdadm -D /dev/md1
/dev/md1:
Version : 00.90.00
Creation Time : Thu Aug 21 12:21:21 2003
Raid Level : raid1
Array Size : 119684160 (114.13 GiB 122.55 GB)
Device Size : 119684160 (114.13 GiB 122.55 GB)
Raid Devices : 2
Total Devices : 2
Preferred Minor : 1
Persistence : Superblock is persistent

Update Time : Fri Oct 15 13:19:11 2004
State : dirty, no-errors
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0

Number Major Minor RaidDevice State
0 3 66 0 active sync /dev/hdb2
1 3 2 1 active sync /dev/hda2
UUID : ede70f08:0fdf752d:b408d85a:ada8922b

I was a little surprised that this process wasn’t entirely automatic. There’s no reason it couldn’t be. This is an older Linux install; I don’t know if more modern versions will just automatically rebuild.

Centos/Redhat/Debain Internet Connection Sharing

Friday, August 14th, 2009

Network Setup :
eth0 = 192.168.2.1 [ Isp router]
eth1 = 10.0.0.2 [ Internal network]

Check if IPv4 forwarding is ON or OFF :
cat /proc/sys/net/ipv4/ip_forward
if result = 0 then will have to On it by this command :

echo “1” > /proc/sys/net/ipv4/ip_forward

Now have to Enable IP masquerading by adding rules in iptables

iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
[ Now all internet request will go via eth0]

If internal computers are unable to ping by domain name, then have to add bellow rule to allow all UDP [53] request to go to router(For Centos and Redhat)
-A RH-Firewall-1-INPUT -m state –state RELATED,ESTABLISHED -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp –dport 53 -j ACCEPT

Or
-A RH-Firewall-1-INPUT -p udp -m state –state NEW -m udp –dport 53 -j ACCEPT

How to sent email to a distribution group by sendmail

Wednesday, August 12th, 2009

goto /etc/mail
vi virtualtable

all@yorudoman.co.uk allusers

Now go to

/etc/mail
vi allusers.txt

user1
user2
user3

[ Here you will have to just write the username (system username)]
[If you have lots of user then you can use script to copy all username from /etc/password to /etc/mail/allusers.txt file]

Now save the file

vi /etc/newaliases

Insert this line

allusers: :include:/etc/mail/allusers.txt

Then make new aliases
That’s it
Now when you will sent email to allusers@yourdomain.co.uk
It will sent that email to every user in that group.

How to make VIM as IDE for Bash and Perl

Wednesday, August 12th, 2009

For Bash IDE:
1. Download bash-support.zip file from this site : http://www.vim.org/scripts/script.php?script_id=365
2. Unzip bash-support.zip file in /etc/vim directory also either copy the bash-support from /etc/vim to the user’s home directory [ cd /home/user, mkdir .vim, cp -r /etc/vim/bash-support /home/user/.vim] Or unzip bash-support.zip in user’s home directory [ /home/user/.vim]

3. Open your script in gvim mood. [ gvim script.sh]
Read More :
http://www.vim.org/scripts/script.php?script_id=365

For Perl :
1.Download the zip file from here : http://www.vim.org/scripts/script.php?script_id=556
Then follow the same way for bash.

Basic Linux User administration Commands

Wednesday, August 12th, 2009
  1. useradd -s /sbin/nologin username : It will prevent user to login to server
  2. userdel -r username: -r delete everything( home directory,mail spool) without -r it will just delete account references from user and groups
  3. usermod -L username : -L to disable user account
  4. usermod -U username : -U enable the user account.
  5. echo ‘mypassword’ | passwd –stdin username : allow to pipe a new plain text password.
  6. groupadd [ [-g gid [-o]] [-r] [-f] groupname : -f to force groupadd to accept an existing group name, -r to create a system group
  7. gpasswd [ -A username] [-M usrname] groupname : -A username to assign username as groupnames’s group administrator. – M username adds username to groupname’s membership roster.
  8. who [-Hil] | [-q] : -H options to add column heading to who’s output, -i to add users idle time, -l force to show fully qualified domian , -q to obtain total number of logged in users.

w [-husf] [username] : by defautl w prints header information. -h disable header information.-s generate the sort output. -f disable the host

informaition.

How to add a new hardrive in linux with LVM

Wednesday, August 12th, 2009

The steps are :

Create Physical volume Or Extend the existing volume:

  1. Create a Physical volume by: pvcreate /dev/hdc ( Here the new partition name is /dev/hdc)
  2. Creating a Volume Group : vgcreate /dev/VolGroup01 /dev/hdc
  3. OR to extended the existing Volume Group :vgextend /devVolgroup01 /dev/hdc

Create Logical Volume :

check how much free PE you got by : vgdisplay
It will show something like this :
Free PE / Size 319 / 9.97 GB
Now to create logical volume
lvcreate -l number_of_PEs /dev/Volgroup01 -n LogicalVolumeName
Or
lvcreate -L 800MB /dev/Volgroup01 -n LogicalVolumeName
Now to check if it works type :lvdisplay
to see how much space you left you can check again by : vgdisplay , it will show something like this
Free PE / Size 294 / 9.19 GB

Now you need to mount the logical volume with a directory:

Format the logical volume : mkfs.ext3 /dev/VolGroup00/LogicalVolumeName
Now mount it : mount /dev/VolGroup00/LogicalVolumeName /mount-point
Now Create a lable for this mount point
e2label /dev/VolGroup00/LogicalVollumeName /mount-point/ and add this reference in /etc/fstab
LABEL=/mount-point /mount-point ext3 defaults 1 2

How to Setup a transparent proxy with Squid

Wednesday, August 12th, 2009

by LinuxTitli [Last updated: December 5, 2007]

Setup :

i) System: HP dual Xeon CPU system with 8 GB RAM (good for squid).
ii) Eth0: IP:192.168.1.1
iii) Eth1: IP: 192.168.2.1 (192.168.2.0/24 network (around 150 windows XP systems))
iv) OS: Red Hat Enterprise Linux 4.0 (Following instruction should work with Debian and all other Linux distros)

Eth0 connected to internet and eth1 connected to local lan i.e. system act as router.

Server Configuration:

  • Step #1 : Squid configuration so that it will act as a transparent proxy
  • Step #2 : Iptables configuration
    • a) Configure system as router
    • b) Forward all http requests to 3128 (DNAT)
  • Step #3: Run scripts and start squid service

First, Squid server installed (use up2date squid) and configured by adding following directives to file:
# vi /etc/squid/squid.conf

Modify or add following squid directives:
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
acl lan src 192.168.1.1 192.168.2.0/24
http_access allow localhost
http_access allow lan

Where,

  • httpd_accel_host virtual: Squid as an httpd accelerator
  • httpd_accel_port 80: 80 is port you want to act as a proxy
  • httpd_accel_with_proxy on: Squid act as both a local httpd accelerator and as a proxy.
  • httpd_accel_uses_host_header on: Header is turned on which is the hostname from the URL.
  • acl lan src 192.168.1.1 192.168.2.0/24: Access control list, only allow LAN computers to use squid
  • http_access allow localhost: Squid access to LAN and localhost ACL only
  • http_access allow lan: — same as above —

Here is the complete listing of squid.conf for your reference (grep will remove all comments and sed will remove all empty lines, thanks to David Klein for quick hint ):
# grep -v "^#" /etc/squid/squid.conf | sed -e '/^$/d'

OR, try out sed (thanks to kotnik for small sed trick)
# cat /etc/squid/squid.conf | sed '/ *#/d; /^ *$/d'

Output:
hierarchy_stoplist cgi-bin ?
acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY
hosts_file /etc/hosts
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern . 0 20% 4320
acl all src 0.0.0.0/0.0.0.0
acl manager proto cache_object
acl localhost src 127.0.0.1/255.255.255.255
acl to_localhost dst 127.0.0.0/8
acl purge method PURGE
acl CONNECT method CONNECT
cache_mem 1024 MB
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
acl lan src 192.168.1.1 192.168.2.0/24
http_access allow localhost
http_access allow lan
http_access deny all
http_reply_access allow all
icp_access allow all
visible_hostname myclient.hostname.com
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
coredump_dir /var/spool/squid

Iptables Configuration:

Next, I had added following rules to forward all http requests (coming to port 80) to the Squid server port 3128 :
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 80 -j DNAT --to 192.168.1.1:3128
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

Here is complete shell script. Script first configure Linux system as router and forwards all http request to port 3128 (Download the fw.proxy shell script):
#!/bin/sh
# squid server IP
SQUID_SERVER="192.168.1.1"
# Interface connected to Internet
INTERNET="eth0"
# Interface connected to LAN
LAN_IN="eth1"
# Squid port
SQUID_PORT="3128"
# DO NOT MODIFY BELOW
# Clean old firewall
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X
# Load IPTABLES modules for NAT and IP conntrack support
modprobe ip_conntrack
modprobe ip_conntrack_ftp
# For win xp ftp client
#modprobe ip_nat_ftp
echo 1 > /proc/sys/net/ipv4/ip_forward
# Setting default filter policy
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
# Unlimited access to loop back
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT
# Allow UDP, DNS and Passive FTP
iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT
# set this system as a router for Rest of LAN
iptables --table nat --append POSTROUTING --out-interface $INTERNET -j MASQUERADE
iptables --append FORWARD --in-interface $LAN_IN -j ACCEPT
# unlimited access to LAN
iptables -A INPUT -i $LAN_IN -j ACCEPT
iptables -A OUTPUT -o $LAN_IN -j ACCEPT
# DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy
iptables -t nat -A PREROUTING -i $LAN_IN -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT
# if it is same system
iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT
# DROP everything and Log it
iptables -A INPUT -j LOG
iptables -A INPUT -j DROP

Save shell script. Execute script so that system will act as a router and forward the ports:
# chmod +x /etc/fw.proxy
# /etc/fw.proxy
# service iptables save
# chkconfig iptables on

Start or Restart the squid:
# /etc/init.d/squid restart
# chkconfig squid on

How to test if squid is working properly ?

See access log file /var/log/squid/access.log:
# tail -f /var/log/squid/access.log

Above command will monitor all incoming request and log them to /var/log/squid/access_log file. Now if somebody accessing a website through browser, squid will log information.

Problem and Solutions:

(a) Windows XP FTP Client

All Desktop client FTP session request ended with an error:
Illegal PORT command.

I had loaded the ip_nat_ftp kernel module. Just type the following command press Enter and voila!
# modprobe ip_nat_ftp

Please note that modprobe command is already added to a shell script (above).

(b) Port 443 redirection

I had block out all connection request from our router settings except for our proxy (192.168.1.1) server. So all ports including 443 (https/ssl) request denied. You cannot redirect port 443, from debian mailing list, “Long answer: SSL is specifically designed to prevent “man in the middle” attacks, and setting up squid in such a way would be the same as such a “man in the middle” attack. You might be able to successfully achive this, but not without breaking the encryption and certification that is the point behind SSL“.

Therefore, I had quickly reopen port 443 (router firewall) for all my LAN computers and problem was solved.

(c) Squid Proxy authentication in a transparent mode

You cannot use Squid authentication with a transparently intercepting proxy.

Basic kernel related commands

Wednesday, August 12th, 2009


depmod -a :
it will add new module automaticaly
modprobe drivername:
it will add that specifiq driver
Preventing “ping of death” :
cat /proc/sys/net/ipv4/tcp_syncookies , output should be 1
lsmod:
to show the installed kernel module
/lib/modules/kernel_virson/Directory :
here all kernel modules are stored
modprobe -r modulename :
will remove that module

Kernel Tuning: Kernel Runtime Parameters
Several kernel features, such as IP forwarding or the maximum number of files, can be
turned on or off without compiling and installing a new kernel or module. These tunable
parameters are controlled by the files in /proc/sys directory. Parameters that you set are made
in the /etc/sysctl.conf file. You use the sysctl command directly. The -p option causes
sysctl to read parameters from the /etc/sysctl.conf file (you can specify a different file). You
can use the -w option to change specific parameters. You reference a parameter with its key.
A key is the parameter name prefixed with its proc system categories (directories), such as
net.ipv4.ip_forward for the ip_forward parameter located in /proc/sys/net/ipv4/. To
display the value of a particular parameter, just use its key. The -a option lists all available
changeable parameters. In the next example, the user changes the domain name parameter,
referencing it with the kernel.domainname key (the domainname command also sets the
kernel.domainname parameter):
# sysctl -w kernel.domainname=”mytrek.com”
The following example turns on IP forwarding:
# sysctl -w net.ipv4.ip_forward=1
If you use just the key, you display the parameter’s current value:
# sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

SSH Dictionary Attack Prevention with iptables

Wednesday, August 12th, 2009

Ref :http://hostingfu.com/article/ssh-dictionary-attack-prevention-with-iptables

Last week (9-15 April). 8,750 failed SSH login attempt, averaging almost one per minute, trying out all kinds of possible user names and left tons of junk in my message log. The recent SSH brute-force attacks (actually it’s not that recent) are rather annoying, and this article at Whitedust.com has useful information on how to prevent this kind of attacks.

For me I have always used AllowUsers directive in /etc/ssh/sshd_config to limit the users that can login. In my setup, I have

AllowUsers root@home-IP my-regular-login

It allows root ssh login, but only from my home ADSL connection with static IP address so I can automate backups. Then it also includes a user ID that I regularly use to log into this VPS. If I need to do some system administration, I’ll use either su or sudo once I am inside.

However I found it is also ideal to slow down the attack when the infested host started to brute force the SSH authentication. There are many scripts/user-land daemons that perform monitoring and blocking. However in a resource limited VPS, I prefer to use something that has less demand in memory/CPU usage. IPTables recent module provides a kernel level solution with little overhead.

This is what I have in my iptables rules:

iptables -N SSH_CHECK
iptables -A INPUT -p tcp --dport 22 -m state --state NEW -j SSH_CHECK
iptables -A SSH_CHECK -m recent --set --name SSH
iptables -A SSH_CHECK -m recent --update --seconds 60 --hitcount 4 --name SSH -j DROP

What it does is:

  1. Create a new chain SSH_CHECK, and all incoming SSH connection (TCP port 22) will go into this chain to test the condition.
  2. Condition is, for any source IP address there cannot be more than 3 SSH connection attempts within a 60 seconds window.
  3. If condition has been met, then all packets from that source IP address will be dropped.
  4. That source IP can only connect again if condition is cleared again, i.e. there has been 60 seconds of quiet time.

I found it quite effectively and dramatically reduce bot attacks on SSH port. Still, it is important to remove shell access from users that no longer require it, and choose sensible random password that is difficult to guess.

Iptables-rules

Wednesday, August 12th, 2009

Allow ssh connection from selected Ip:
iptables -A INPUT –source xx.xx.xx.xx -p tcp –dport 22 -j ACCEPT
iptables -A INPUT –source yy.yyy.yy.yy -p tcp –dport 22 -j ACCEPT
iptables -A INPUT -p tcp –dport 22 -j DROP

Only allow ssh to linux box:

iptables -P INPUT DROP
iptables -P OUTPUT DROP
iptables -P FORWARD DROP

iptables -A INPUT –source xx.xx.xx.xx -p tcp –dport 22 -j ACCEPT
iptables -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -m state –state ESTABLISHED,RELATED -j ACCEPT

How to avoid TCP SYN FLOODING :
echo 1 > /proc/sys/net/ipv4/tcp_syncookies.

Iptables for MASQUERADE

# iptables -t nat -A POSTROUTING -s 192.168.0.0/24 -o eth1 -j MASQUERADE
The following command enables FTP connection tracking through your
firewall:

# modprobe -a ip_conntrack_ftp ip_nat_ftp

How to install openssh-server in knoppix

Tuesday, August 11th, 2009

In knoppix,
cd /etc/apt/
nano sources.list
add any good debain repo such as

deb http://http.us.debian.org/debian stable main contrib non-free

then : Save the file
then : apt-get update

then type: apt-get install openssh-server
also , give a password to root because currently knoppix does not have any root password.

repo ref: http://www.debian.org/doc/manuals/apt-howto/ch-basico.en.html

How to find expensive I/O process for I/O bottol neck

Tuesday, August 11th, 2009

To find the most expensive process which causing the I/O bottol neck :

1. iotop ( http://guichaz.free.fr/iotop/)
Iotop requires Python ≥ 2.5 and a Linux kernel ≥ 2.6.20 with the TASK_DELAY_ACCT and TASK_IO_ACCOUNTING options enabled.
2. idstat from sysstat packages

But Iotop provides more user friendly output then idstat.

How to Change Ip from Dynamic to Static

Tuesday, August 11th, 2009

In Debain :
/etc/networking/interfaces

auto eth0
iface eth0 inet static
address 192.168.1.10
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.1

then /etc/init.d/networking restart

In Centos:
vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=Static
HWADDR=00:0C:29:81:90:33
ONBOOT=yes
IPADDR=192.168.1.1
NETMASK=255.255.255.0
NETWORK=192.168.1.0
GATEWAY=192.168.1.1
BROADCAST=192.168.1.255

then : /etc/network/restart