Hammer Filesystem

On Feb 17 2009, new release 2.2 of DragonFly BSD released which includes a new filesystem called hammer. Hammer is a new filesystem which intented to replace ffs on DragonFly systems. Hammer filesystem has many advanced futures compared to ffs filesystem like

• instant crash recovery
• large file systems and multivolume support
• data integrity checking
• history and snapshots
• reblocking (on the fly defragmentation)
• mirroring
  

Let's start by creating a new hammer filesystem on a new disk device by using hammer's newfs_hammer command. As you can see below, we used the '-L' parameter. Every hammer filesystem requires a label name and it's mandatory.




Let's mount in under /mnt directory by using "noatime" option. mount_hammer command must be used for this operation.


Looking to df output you can see our new hammer filesystem labeled as "mydata". Now, let's see where hammer filesystem differs from ffs.

History feature

History metadata of a hammer filesystem is written on every 30 seconds. You can use hammer utility to access your files history data. Usage is simple as, calling hammer utility with history command and given your file name which you want to see whole history.

# hammer history /mnt/test.txt



You may be noticed that I have used "sync" command, because I didn't want to wait the the kernel sync operation occur (30 seconds) and forced it by using sync comand to get history result fast.


Snapshot feature

Now let's see hammer's snapshot feature. I have created a directory called "/snapshots" where I want to keep all of my hammer filesystem snaphosts. We'll use hammer utility for taking snapshots.

# hammer snapshot /mnt /snapshots/mnt_snapshot1

I have created a file on /mnt filesystem and took a snapshot. After a sucsessful snapshot, I deleted the original file from /mnt and accessed it by using the snapshot I took before deleting the file (/snapshots/mnt_snapshot1/test.txt).


Undo Feature

You can see and rollback every change between the taken snapshots and original files by using hammer's undo command. You can use "-d" option to get diff output to see every changes on the given file line by line between every snapshot and original one. Also, "-a" option gives you whole change history of given file. Here, I created a test file called "myfile.txt" under my new hammer filesystem (/mnt) and then took a snapshot. After taking snapshot, I added one more line to my test file and used "undo -d /mnt/myfile.txt" to get diff output and see what changes made the file between snapshots.





Mutivolume Feature

As I mentioned above, you can use multi volumes and create a big disk volume with hammer filesystem. Now, we'll create a single hammer volume by using two disks (ad1 and ad3).
You just call newfs_hammer and mount_hammer commands by only giving two or more disk device names instead of one.Commands are straight forward. First, we need to create new hammer filesystem on disks.

# newfs_hammer -L big /dev/ad1 /dev/ad3



I have used to disks and each size is 8GB. After creating new hammer filesystems on disks , we can mount them as one big volume. (I know 8GB is not big :) ).

# mount_hammer /dev/ad1 /dev/ad3 /mnt

You can see on /mnt mount point we have a disk in size 16GB, total of two 8GB disks.

Mirror Feature

Hammer uses Pseudo File Systems (PFS) to duplicate inode numbers to slaves. Therefore, mount_null command required for this operation. This duplication is triggered by using hammer utlity with mirror-copy parameter. First we need to create two pfs, called for master and slave. I created them as /mymaster and /myslave. Master volume must be created with pfs-master and slave with pfs-slave parameter by using hammer utility. One thing to take in accounting here is that slave pfs must use master's shared-uuid number.




After creating mymaster and myslave pfs we have to use mirror-copy to start initial mirroring operation by using these pfs links , because mount_null can't access them without this copy operation.



After, running first mirror copy operation on our pfs links, we can mount our original hamme volumes where we want and associate them with the pfs links we created and sync (mirror-copy) our mounted hammer volumes.


You can use mount_null mount points to do your filesystems operations. You can see that slave volume (myslave) is read-only and you don't have write access to it. You can only use master volume (mymaster) to make your write operations.If you're looking an alternative filesystem to ffs on BSD systems, I suggest you to evaluate both ZFS on FreeBSD and Hammer on DragonFly BSD.

xfs filesystem fragmentation

Many filesystems use many techniques to overcome filesystem fragmentation and XFS filesystem is one of them. But in long usage, you may face with fragmentation problems.Here, I will show you how to find fragmentation ratio in your xfs filesystem and how to re-organize your xfs filesystem with xfs_fsr utility to reduce fragmentation on a live xfs filesystem. First, let's check our xfs filesystem's fragmenation ratio using xfs_db (xfs debugger) utility. As shown below, we used two parameters. First one is "-r", this means that we use debugger in read-only mode. We do not want to accidently do anything bad to our filesystem. Also, this option is useful if we are debugging a mounted filesystem. Second option is "-c" command, which simply takes it's parameter as xfs_db command and run it. Without "-c" parameter, xfs_db will run it's interactive shell where you can run xfs_db commands.



We see that our xfs filesystem on /dev/sdb device has 0.14% fragmentation. This is very low fragmentation. We don't need to defrag it. But here how to do it with xfs_fsr. This utility re-organizes a xfs filesystem while it's mounted file by file. As you can see in sample below, it runs with "-t" parameter, this tells xfs_fsr utility to run maximum that given seconds and quit.


There is one more option you can use while using xfs filesystem to avoid future fragmentations. It's called "allocsize", where it preallocates disk space before writing to a file. You have to set this preallocation size while mounting your xfs filesystem. This will prevent fragmentation on your xfs filesystem if you give a reasonable size depending on your average file sizes.

mount -t xfs -o noatime,allocsize=8M /dev/sdb /mydata

you can also set this option in your /etc/fstab.

stopping and resuming a process run in Linux

You can stop (pause) a running process by sending "-STOP" signal via kill command in Linux. You can re-run same stopped process again by sending "-CONT" signal to it. Assume that we have a process to stop and it's pid is 8674. Following, command will stop the running process id 8674.

# kill -STOP 8674

Now the process is stopped, to resume it's running state, you have to send "-CONT" signal.

# kill -CONT 8674

In Solaris, you can do the same task by using pstop and prun utilities or you can use kill command and use signals "SIGSTOP" and "SIGCONT".

measuring your network throughput on Solaris

You can use nicstat utility to measure your network device utilization and throughput. The utility itself uses solaris kstat for network device statistics information. Statistics include network read and write in kilobytes, packets and device utilization.You can compile nicstat with gcc as shown below.

# gcc nicstat.c -o nicstat -lkstat -lgen -lsocket

Following image shows a sample output from nicstat utility.

getting network interface statistics on Solaris

There are many ways to get network interface status and statistics on Solaris. I'll show some of them. First one is well known command netstat. netstat's "-i" parameter will give you statistics of devices used for ip traffic. Statistics include input/output total packets,errors and collisions for each network device.


Next method is by using solaris ndd command. First, we have to find our device name. This can be done by ifconfig command.
My network device name is "e1000g0" as seen in sample output above. Now, it's time to ask ndd which parameters are supported by this device. We do this by calling ndd (ndd /dev/e1000g0 \?) with our device name and "\?" parameter to display every supported parameter.


ndd let's you change some interface settings. They're marked as "read and write". The parameter names clearly shows what they are for. As you can see in sample output below, ndd can give us link speed, link status (cable plugged/unplugged),duplex mode and autoneg status. Following sample output shows us that my network interface link status is 1, which means the network cable is plugged,Network speed is 1000mbit and auto negotiation is on.
The other command is called dladm. dladm can give us above results with one command and in easily readable format. Also, it can give network interface statistics like netstat with "-s" parameter except collisions.

iptables and connection limit

Linux iptables uses a module called ip_conntrack. The name of the module exactly says what it does. It tracks every connection attempt (every connection state) in a hashed table.Every hash entry contains a linked list. Each linked listed entry is called a bucket presenting a connection. You can see your machine's current connections by using ip_conntrack module's proc entry.


Of course, this table has a default value. It is likely that you will reach this limit on a machine getting high connection rates. First clue will be the following log message on your system logs (/var/log/messages).

ip_conntrack: table full, dropping packet

In this case, your machine (iptables) will drop every incoming connection because there is no free hash entry to store any incoming connection (udp or tcp). In this case, there is a configuration option to bump this connection limit to higher values. It's called ip_conntrack_max. It's in /proc filesystem (/proc/sys/net/ipv4/ip_conntrack_max).

You can easily increase this number by simply echo 'ing new number in to this file.

This conntrack tables is divided in linked list entries by hash number as I mention above. The calculation is CONNTRACK_MAX/HASHSIZE. Every hash contains linked lists for connections.This parameter is locted in /sys/module/ip_conntrack/parameters/hashsize. In our case it's default value is 4096. You remember that before changing our default value of conntrack_max, it was 32768. The default hashsize 4096 is produced by 32768/8 calculation. You can tweak this number on your system to increase performance of iptables. Don't forget to put your settings in /etc/sysctl.conf. Because overriding this /proc/ and /sys/ values is only temporary change. There will be gone when you reboot your machine. The hashsize parameter in /sys/ is called buckets in sysctl conf configuration. Also, check other conntrack parameters via "sysctl net.ipv4.netfilter". By setting some other options like wait time can reduce your entries in conntrack table.

Solaris Interrupts

In this post, I'll be shown you how to access irq table on a Solaris system and how to get statistics about the number of interrupts produced on Solaris. We'll use two commands intrstat and mdb.
intrstat command shows how many interrupts produced by each device and how many of them handled by each cpu on the system. "%tim" column shows the time where spend by that driver's interrupt handler on processing cpu. You can pass a parameter to intrstat and tell it to gather information in given intervals (in seconds). Following is a sample output of intrstat where it gets statistics in 5 seconds interval.

Now, after getting interrupt statistics we can now examine the irq table on Solaris system. For this job, we'll use solaris mdb (Solaris modular debugger). We can access irq table through the kernel. So, we'll pass the "-k" parameter to mdb. This will tell it to get in kernel debugging mode. mdb will use /dev/ksyms and /dev/kmem to access Solaris kernel. After running "mdb -k", type "::interrupts" command to acess irq table in Solaris system. Here is a sample output:

This table shows us which driver(s) (ISR) using which irq. Share column shows how many devices sharing the same irq. IPL gives irq priority. Type column show irq type used Fixed (legacy), MSI, IPI interprocessor interrupt (xcalls - cross call).