1. Which File System for a Large Storage Array Under Linux?

    Mon 30 November 2009

    There are many file systems available under Linux, however only a few of them can be used for a large storage array.

    I am assuming that you want to create a single file system. I don't care if you use LVM or other layers beneath, this is about which file system to use.

    I will discuss each file system in short.

    • EXT3

    Old and trusted. but slow, wastes tremendous amounts of free space and has an 16 TB limit. Not recommended. At 16 TB, you will lose significant amounts of free space.

    • JFS

    Developed by IBM, it seems an ok file system and it supports 16+ TB file systems. However it is abandoned by IBM and does not seem to have a future. The reason I did not choose JFS is because it does not seem to be in wide spread use and therefore, and I do not want to take any risk in this regard. I might encounter some rare bug because I would be one of the few Linux jfs users that creates 16+ TB file systems.

    • XFS

    A file systems that works. Supports 16+ TB file systems and is widely used. It has some drawbacks, such as data loss (files are zeroed) when something goes wrong (powerloss) however, nobody that is interested in creating a large filesystem is going to run the box without a UPS.

    • Reiserfs

    The guy who designed and wrote it is in jail for killing his wife. Not much future here, i suppose. Also, it does not support file systems bigger than 16 TB.

    • Reiser4

    Some new version of Reiserfs which is still not available as part of a stable Linux kernel.

    • EXT4

    Ext3 upgraded to current days need, although with a draw back. On paper, it supports 16 TB+

    file system, but currently, in practice, it does not. I found out the hard way and had to reformat my array with another file system.

    My choice:

    Honestly, there is not much of an option currently under Linux other than XFS. In the past, I had some quirks with file names that contained strange characters causing trouble. However, I never lost any data whatsoever.

    Currently, what I'm really hoping for is ZFS for Linux or a stable version of btrfs. These are truly modern day file systems with support for snapshots, etc. But this is still a dream. Until then, I will stick with XFS.

    Tagged as : Uncategorized
  2. 'Tip of the Day: Scrolling in GNU Screen'

    Mon 30 November 2009

    Just the answer:

    By default, on Debian, the scroll back buffer is about 1K lines. This can be changed in the .screenrc file in your home directory. The following example sets the scroll back buffer to 5K lines.

    defscrollback 5000

    Set the scroll back buffer on the fly with:

    First enter the command line mode:

    ctrl + a, :

    Then enter:

    scrollback 2000

    To set the scroll back buffer to 2000 lines.

    To actually scroll back, the actuall stuff why you may be reading this post:

    First enter copy mode:

    ctrl + a, [

    Use standard VI controls to navigate through the lines (h j k l).

    Use ctrl+b to scroll a full page up

    Use ctrl+f to scroll a full page down.

    For more details, please visit this link:

    scrollback-buffer/

    Tagged as : Uncategorized
  3. 'Gzip With Parallel Compression Support: Pigz'

    Sun 22 November 2009

    The speed at which files are compressed with gzip is currently almost always determined by the speed of the CPU. However, standard unix gzip is single- threaded and only uses a single CPU (core).

    However, the maintainer of the zlib library has released 'pigz' or 'pig-zee' whichs adds just that: support for parallel compression. This dramatically improves the speed at which a file can be gzipped.

    In this example, a 3 GB compressable file is gzipped:

    Gzip:

    root@Core7i:~# time gzip pigz.bin

    real 1m58.994s

    user 1m56.480s

    sys 0m1.820s

    Pigz:

    root@Core7i:~# time pigz pigz.bin

    real 0m31.524s

    user 2m54.890s

    sys 0m2.900s

    This simple and a bit unscientific example shows a 400% speed improvement. Since the Core i7 has four real cores, this shows that pigz scales nicely.

    Tagged as : Uncategorized

Page 11 / 23