Boosting Radware Defense Pro's performance ;)

Read more...

Detailed network interface statistics on Linux

Here is a helpful command for getting interface statistics on your linux box. It's called ip. It's bundled with iproute package. You can see information about dropped, collisions,crc,transmit and other info about your network interface.

root@helga:~# ip -s -s link show eth0
2: eth0:  mtu 1500 qdi
WN qlen 1000
  link/ether 00:24:8c:d5:6a:a7 brd ff:ff:ff:ff:ff:ff
  RX: bytes  packets  errors  dropped overrun mcast
  1633058095 16933162 0       0       0       0    
  RX errors: length  crc     frame   fifo    missed
             0        0       0       0       0    
  TX: bytes  packets  errors  dropped carrier collsns
  1994496524 5588574  0       0       0       0    
  TX errors: aborted fifo    window  heartbeat
             0        0       0       0    

Read more...

I/O usage per process on Linux

Linux kernel 2.6.20 and later supports per process I/O accounting. You can access every process/thread's I/O read/write values by using /proc filesystem. You can check if your kernel has built with I/O account by just simply checking /proc/self/io file. If it exists then you have I/O accounting built-in.

$ cat /proc/self/io
rchar: 3809
wchar: 0
syscr: 10
syscw: 0
read_bytes: 0
write_bytes: 0
cancelled_write_bytes: 0

Field Descriptions:

rchar  - bytes read
wchar  - byres written
syscr  - number of read syscalls
syscw  - number of write syscalls
read_bytes  - number of bytes caused by this process to  read
            from underlying storage
write_bytes - number of bytes caused by this process to written from
            underlying storage

As you know, ever process is presented by it's pid number under /proc directory. You can access any process's I/O accounting values by just looking /proc/#pid/io file. There is a utility called iotop which collects these values and shows you in like top utility. You see your processes I/O activity with iotop utility.

Read more...

problem compiling Mysql 5.1 with sphinx engine support

Mysql's full-text search engine is not very powerful and support comes only with Myisam engine. Sphinx is a full-text search engine replacement for Mysql and Postgresql. Also, It's possible to use Sphinx full-text search engine with both Myisam and InnoDB engines. You can read detailed information about mysql compilation with sphinx on Sphinx site.

I have faced with following error message, while compiling Mysql with innodb and sphinx engine support ( ./configure --prefix=/data/mysql3/ --with-plugins=sphinx,innobase). I used the innodb (base) which bundled with Mysql. You can also compile innodb plugin which has additional features. For example, on the fly compression, speed improvements.

configure: error: unknown plugin: sphinx

The trick was to run autorun.sh command in Mysql's source tree before running configure with sphinx engine support. This script will create a new configure script which knows how to compile sphinx engine.

# ./BUILD/autorun.sh

Read more...

nethogs network utility

Nethogs is a small but very useful utility, if you like to see your machine's bandwidth usage per processes. Here is a simple screenshot from my linuxbox.

Read more...

Fast Copy on Unix systems

I needed to copy so many small files and directories (1.2 TB) to another machine with netcat (nc). But I faced with a problem when I tried to run nc program on the background with bash. Bash was suspending standard input when I move nc process to background job, this was causing nc processess to die when standard input was blocked. So, I have found another netcat like utility called socat. Socat is not stop execution while it was working as background job on bash shell. I was able to utilize 100mbit ethernet with few socat processes on server and client side. First of all, I run socat processes on destination server as daemons to listen specific ports. Following, command will run socat in daemon mode to listen on tcp port 4123 and pipe incoming data to tar command on the background.

# (socat -u tcp4-listen:4123 - | tar xzp -C .) &

On source server, I run socat to feed incoming tar data to destination socat daemon where it listens on tcp port 4123. My destination server's ip adddess was 192.168.36.199 .

# (tar -czp - /mydir/1/ | socat -u - tcp4:192.168.3.199:4123)&

So, with a simple bash script I was able to run few socat daemons on destination server with different ports and feed them from source server to achieve higher data transer rate.

Read more...

socket programming with Bash

I have many memcached servers around. So, I needed a tool to check status information on these servers like top utility. I decided to write it in bash scripting language (100% in bash except printf,date and sleep commands :) ). You can simply enhance the script for example getting server names and ports from a config file or display interval parameter from command line. Memcached supports tcp and udp protocols for communication with clients. I used tcp communication in my bash script. Here is information about Memcached's supported server commands. My example script uses "stats" command. Here is sample output for "stats" command from one memcached server.

I have used the information provided by "stats" command. Script uses pure bash builtin commands to access a tcp server like memcached.

You can download the script from here.


Here is sample output from the script.



If you're using Ubuntu, probably you will not be able to run this script, because bash is not compiled with "--enable-net-redirections" parameter on Ubuntu systems. RedHat based distributions has no problem with bash net-directions like Fedora,Centos and RedHat ES. Read more...