Thursday, January 1, 2009

Caching filesystems on linux with fs-cache (cachefilesd)

fs-cache can be use to cache nfs,local or other network filesystems. Here is an example howto cache a nfs mount point on Centos linux distribution. First of make sure that cachefilesd package is installed. Then make sure that cachefilesd (/etc/init.d/cachefilesd) runs on every boot. You can set it to run on startup by chkconfig utility if you like.
Cachefilesd has a config file named /etc/cachefilesd.conf, where you can set where the cache content will be written. The option is called "dir".

eg. dir /cache

Note: caching directory should be an ext3 filesystem.

Let's assume that we have a nfs mount point called mydisk on server 192.168.0.2. Let's create an entry in our fstab for this mount point

192.168.0.2:/mydisk /mydisk nfs tcp,fsc 0 0

the only option here is "fsc" for letting the caching start. After using check your cache directory in this case our directory is /cache to see it's working. There will be 2 directories there called graveyard and cache. Here is some cache filenames created on my cache directory.

./cache/@4a/I03nfs/@4f/Jk0M00wg00000000000000Y--knmon200/@08
./cache/@4a/I03nfs/@4f/Jk0M00wg00000000000000Y--knmon200/@08/Eo0g000M00E60180000gA5Ac0gmgO00FY0100
./cache/@4a/I03nfs/@4f/Jk0M00wg00000000000000Y--knmon200/@10
./cache/@4a/I03nfs/@4f/Jk0M00wg00000000000000Y--knmon200/@10/Eo0g000M00E601800000D5Ac0cmgO04FY0100
./cache/@4a/I03nfs/@4f/Jk0M00wg00000000000000Y--knmon200/@77
./cache/@4a/I03nfs/@4f/Jk0M00wg00000000000000Y--knmon200/@77/Eo0g000M00E60180000wcAv20fak00Ad12000

Monday, December 15, 2008

logging errors with erlang and mochiweb

If you would like to run mochiweb or any erlang application and log messages to disk to review later, all you have to do is create a simple config file and wrote something in it like as follows:

[{sasl, [
%% minimise shell error logging
{sasl_error_logger, false},
%% only report errors
{errlog_type, error},
%% define the parameters of the rotating log
%% the log file directory
{error_logger_mf_dir,"/var/logs/error_logs"},
%% # bytes per logfile
{error_logger_mf_maxbytes,10485760}, % 10 MB
%% maximum number of
{error_logger_mf_maxfiles, 10}
]}].

let's say we call this filename erlang_log.config. The trick here is that we have to create the log directory for erlang and make sure it's writable by the user which runs erlang process.
# mkdir /var/logs/error_logs/
you can also make it world writable if you don't care about who reads it on the server.
# chmod 777 /var/log/error_logs/

now pass the config file as parameter to your erl command (on mochiweb part, you can edit start.sh or start-dev.sh file).

erl -config erlang_log

when you first run your erlang process or mochiweb, first of all run on your erlang command prompt
rb:start().
to create index files in log directory and load rb module.Here is some useful commands to see log messages created by erlang. The log files in the directory are binary so you cannot simple read them.You have to read them through erlang (rb module).
rb:list().
%% shows log messages index
rb:show(number).
%% shows given indexed numbered message details
rb:rescan().
%% re-reads the error log directory index.

running mochiweb or any erlang application in background

Erlang is Ericsson's concurrent programming language.See Erlang Wikipedia for more information. Mochiweb is a web framework to create lightweight http servers written in Erlang language.
After creating a skeleton project with mochiweb (./scripts/new_mochiweb.erl). You can write your own code and compile it. Here is an article shows this in detail.
After all, running your new http server is done by star.sh or start-dev.sh in your application directory where you created by new_mochiweb.erl script. The script runs erl (The Erlang Emulator) command with some parameters. But it runs in foreground, if you want to run the erlang application in background simply add "-detached" parameter to your erl command. But, there is a drawback here. The erlang application runs in background and we cannot see the error and any information log messages now. I'll will explain how to log theses messages in log file and read them in another post.

Tuesday, November 25, 2008

profiling mysql queries

Mysql database server has a new query profiling option. It's a very useful feature to profile queries on server and find bottlenecks.
There is an article on mysql site which describes this feature in detail.
Using the New MySQL Query Profiler

Thursday, November 6, 2008

finding backup superblocks on ext2 and ext3 filesystems

If you have a situation where your ext2 or ext3 filesystem's primary superblock is corrupted, you need to pass one of the backup superblocks to fsck program. In this case you can find the backup superblocks by using mkfs program. All you have to do is, run mkfs with "-n" option, where mkfs is run in dry mode. It will not destroy your filesystem, it only prints out what it will do on real mkfs procedure.By the way, you have to add the parameters if you have used when your filesystem originally created to get correct values.

eg. mke2fs -n /dev/sda1

and now you can pass your backup superblock number to fsck (fsck.ext2,fsck.ext3) with "-b" parameter.

freeing cache on linux with drop_caches

Linux kernel 2.6.16 and later has a option which let's you free inode/dentry and pagecache caches on your system. The file is located on /proc filesystem.
It has 3 options. For example:

# echo 1 > /proc/sys/vm/drop_caches

frees pagecache on your system.

# echo 2 > /proc/sys/vm/drop_caches

frees inodes and dentry caches

# echo 3 > /proc/sys/vm/drop_caches

frees pagecache and inode/dentry caches.

tip: some dirty caches are not freeable so it's better to use "sync" command before using drop_caches for freeing much more memory on your linux system.

Wednesday, October 29, 2008

disabling bell beep sound on linux console

Sometimes its annoying to hear a beep sound everytime you hit on tab button for completion on linux console. Here is the quickway how to disable that beep sound.
All you have to do is add the following line in /etc/inputrc file or uncomment it if it presents.

set bell-style none

the following image is the default inputrc file on centos system. You can uncomment the line which says "set bell-style none" for disabling tab completion beep sound.