Detailed network interface statistics on Linux

Here is a helpful command for getting interface statistics on your linux box. It's called ip. It's bundled with iproute package. You can see information about dropped, collisions,crc,transmit and other info about your network interface.

root@helga:~# ip -s -s link show eth0
2: eth0:  mtu 1500 qdi
WN qlen 1000
  link/ether 00:24:8c:d5:6a:a7 brd ff:ff:ff:ff:ff:ff
  RX: bytes  packets  errors  dropped overrun mcast
  1633058095 16933162 0       0       0       0    
  RX errors: length  crc     frame   fifo    missed
             0        0       0       0       0    
  TX: bytes  packets  errors  dropped carrier collsns
  1994496524 5588574  0       0       0       0    
  TX errors: aborted fifo    window  heartbeat
             0        0       0       0    

I/O usage per process on Linux

Linux kernel 2.6.20 and later supports per process I/O accounting. You can access every process/thread's I/O read/write values by using /proc filesystem. You can check if your kernel has built with I/O account by just simply checking /proc/self/io file. If it exists then you have I/O accounting built-in.

$ cat /proc/self/io
rchar: 3809
wchar: 0
syscr: 10
syscw: 0
read_bytes: 0
write_bytes: 0
cancelled_write_bytes: 0

Field Descriptions:

rchar  - bytes read
wchar  - byres written
syscr  - number of read syscalls
syscw  - number of write syscalls
read_bytes  - number of bytes caused by this process to  read
            from underlying storage
write_bytes - number of bytes caused by this process to written from
            underlying storage

As you know, ever process is presented by it's pid number under /proc directory. You can access any process's I/O accounting values by just looking /proc/#pid/io file. There is a utility called iotop which collects these values and shows you in like top utility. You see your processes I/O activity with iotop utility.

problem compiling Mysql 5.1 with sphinx engine support

Mysql's full-text search engine is not very powerful and support comes only with Myisam engine. Sphinx is a full-text search engine replacement for Mysql and Postgresql. Also, It's possible to use Sphinx full-text search engine with both Myisam and InnoDB engines. You can read detailed information about mysql compilation with sphinx on Sphinx site.

I have faced with following error message, while compiling Mysql with innodb and sphinx engine support ( ./configure --prefix=/data/mysql3/ --with-plugins=sphinx,innobase). I used the innodb (base) which bundled with Mysql. You can also compile innodb plugin which has additional features. For example, on the fly compression, speed improvements.

configure: error: unknown plugin: sphinx

The trick was to run autorun.sh command in Mysql's source tree before running configure with sphinx engine support. This script will create a new configure script which knows how to compile sphinx engine.

# ./BUILD/autorun.sh

nethogs network utility

Nethogs is a small but very useful utility, if you like to see your machine's bandwidth usage per processes. Here is a simple screenshot from my linuxbox.

Fast Copy on Unix systems

I needed to copy so many small files and directories (1.2 TB) to another machine with netcat (nc). But I faced with a problem when I tried to run nc program on the background with bash. Bash was suspending standard input when I move nc process to background job, this was causing nc processess to die when standard input was blocked. So, I have found another netcat like utility called socat. Socat is not stop execution while it was working as background job on bash shell. I was able to utilize 100mbit ethernet with few socat processes on server and client side. First of all, I run socat processes on destination server as daemons to listen specific ports. Following, command will run socat in daemon mode to listen on tcp port 4123 and pipe incoming data to tar command on the background.

# (socat -u tcp4-listen:4123 - | tar xzp -C .) &

On source server, I run socat to feed incoming tar data to destination socat daemon where it listens on tcp port 4123. My destination server's ip adddess was 192.168.36.199 .

# (tar -czp - /mydir/1/ | socat -u - tcp4:192.168.3.199:4123)&

So, with a simple bash script I was able to run few socat daemons on destination server with different ports and feed them from source server to achieve higher data transer rate.

socket programming with Bash

I have many memcached servers around. So, I needed a tool to check status information on these servers like top utility. I decided to write it in bash scripting language (100% in bash except printf,date and sleep commands :) ). You can simply enhance the script for example getting server names and ports from a config file or display interval parameter from command line. Memcached supports tcp and udp protocols for communication with clients. I used tcp communication in my bash script. Here is information about Memcached's supported server commands. My example script uses "stats" command. Here is sample output for "stats" command from one memcached server.

I have used the information provided by "stats" command. Script uses pure bash builtin commands to access a tcp server like memcached.

You can download the script from here.


Here is sample output from the script.



If you're using Ubuntu, probably you will not be able to run this script, because bash is not compiled with "--enable-net-redirections" parameter on Ubuntu systems. RedHat based distributions has no problem with bash net-directions like Fedora,Centos and RedHat ES.

Mirroring your OpenSolaris root disk with ZFS

Solaris/OpenSolaris supports booting with ZFS filesystem. For example, if you look into your opensolaris insallation you'll notice the zfs rpool (root pool). ZFS is default filesystem on opensolaris (optional with Solaris). If you look into detailts of that pool with zpool utility , you'll see that it's a single disk or a partition (depends how you partitioned your disk on installation process of opensolaris). If that single disk crashes, you may lost your data and operating system.That's one point of failure. The simple solution is to convert our zpool into RAID-1 with a secondary disk (with same capacity or bigger one).

As you can see, our zpool contains a single disk called c4t0d0. Let's add a new disk to our rpool to convert it into mirror. zpool will do it automatically for us. One important point here is that Solaris doesn't support boot devices to be in EFI label format. So, if your new disk label is in EFI format, you need to convert it into SMI label. So, I'll use fdisk command to create a Solaris2 partition and remove any EFI labels if present. My new disk name is c4t1d0.

After creating the partition which covers whole disk area, I copied partition map of original rpool member c4t0d0 to our new disk c4t1d0.

Now, it's time to attach our brand new disk into rpool. After attaching the disk, our rpool pool will be converted into mirror (raid-1). Except one thing, the grub boot manager. We need to install it into new disk by hand using installgrub command.

Now, we have created our mirror on rpool. It will take time to sync original disk to new disk. You can watch this process by using zpool's status command.