Louwrentius

To get a list of all disk drives of a Linux system, such as this:

Disk /dev/md0: 58.0 GB
Disk /dev/md1: 2015 MB
Disk /dev/md5: 18002.2 GB
Disk /dev/sda: 60.0 GB
Disk /dev/sdb: 60.0 GB
Disk /dev/sdc: 1000.1 GB
Disk /dev/sdd: 1000.1 GB
Disk /dev/sde: 1000.1 GB
Disk /dev/sdf: 1000.1 GB
Disk /dev/sdg: 1000.1 GB
Disk /dev/sdh: 1000.1 GB
Disk /dev/sdi: 1000.1 GB
Disk /dev/sdj: 1000.1 GB
Disk /dev/sdk: 1000.1 GB
Disk /dev/sdl: 1000.1 GB
Disk /dev/sdm: 1000.1 GB
Disk /dev/sdn: 1000.1 GB
Disk /dev/sdo: 1000.1 GB
Disk /dev/sdp: 1000.1 GB
Disk /dev/sdq: 1000.1 GB
Disk /dev/sdr: 1000.1 GB
Disk /dev/sds: 1000.2 GB
Disk /dev/sdt: 1000.2 GB
Disk /dev/sdu: 1000.2 GB
Disk /dev/sdv: 1000.2 GB

You can use the following command:

#!/bin/bash
for x in `cat /proc/diskstats | grep -o 'sd.\|hd.\|md.' | sort -u`
do 
    fdisk -l /dev/$x 2>/dev/nul| grep 'Disk /' | cut -d "," -f 1 
done

I have released LIFS, the Linux Iptables Firewall Script. This script allows you to setup a firewall within minutes. It is easy to use, yet very powerful. It uses Iptables and even improves upon some limitations of Iptables.

Every person who has to maintain some kind of Iptables-based firewall should really look into LIFS. It will make managing your firewall much more convenient.

For more advanced purposes. LFS allows you to create object groups. These are groups of individual hosts, networks or services (tcp/udp).

Look at this example of object groups in action. Read and understand.

HTTP_SERVICES="
    80/tcp
   443/tcp
"

WEB_SERVER_1=192.168.0.10
WEB_SERVER_2=192.168.0.11

WEB_SERVERS="
    $WEB_SERVER_1
    $WEB_SERVER_2
"

allow_in any "$WEB_SRVERS" any "$HTTP_SERVICES"

As you can see, a single firewall rule in fact creates 4 rules, one for each host and port. This functionality can be found in commercial based firewalls but it is not build-in into Iptables. LIFS fixes this.

LIFS is a continuation of LFS, which has been discontinued.

When a disk fails or gets kicked out of your RAID array, it often takes a lot of time to recover the array. It takes 5 hours for my own array of 20 disks to recover a single drive.

Wouldn't it be nice if that time can be reduced? Even to 5 seconds?

Although not enabled by default, you can enable so called 'bitmaps'. As I understand it, a bitmap is basically a map of your RAID array and it charts which areas need to be resynced if a drive fails.

This is great, because I have the issues that of every 30 reboots, sometimes a disk won't get recognized and the array is degraded. Adding the disk back into the array will mean that the system will be recovering for 5+ hours.

I enabled Bitmaps and after adding a missing disk back into the array, the array was recovered instantly.

Isn't that cool?

So there are two types of bitmapsL

1. internal: part of the array itself
2. external: a file residing on an external drive outside the array

The internal bitmap is integrated in the array itself. Keeping the bitmap up to date will probably affect performance of the array. However I didn't notice any performance degradation.

The external bitmap is a file that must reside on a EXT2 or EXT3 based file system that is not on top of the RAID array. So this means that you need an extra drive for this or need to use your boot drive for this. I can imagine that this solution will have less impact on the performance of the array but it is a bit more hassle to maintain.

I enabled an internal bitmap on my RAID arrays like this:

mdadm --grow /dev/md5 --bitmap=internal

This is all there is to it. You can configure an external bitmap like this:

mdadm --grow /dev/md5 --bitmap=/some/directory/somefilename

There probably will be some performance penalty involved, but it does not seem to affect sequential throughput, which is the only thing that is important for my particular case.

For most people, I would recommend configuring an internal bitmap, unless you really know why you would have to use an external bitmap.

When a disk has no free space left, all kinds of trouble can occur.

Therefore, I've created a script that monitors the used space of a volume and deletes the oldest file if a certain threshold is reached.

The script will keep on deleting the oldest file present on disk until used capacity is below the threshold.

So you can tell the script to monitor volume /storage and delete old files if the used capacity is bigger than 95 percent.

The script works like this:

./deleteoldfiles.sh <mount point> <percentage>

The mount point represents a volume or physical disk. The percentage represents the maxium used capacity threshold.

The script reads the output of the 'df -h' command to determine 'disk' usage.

Example:

bash-3.2$ ./deleteoldfiles.sh /Volumes/usb 92

DELETE OLD FILES 1.00

Usage of 90% is within limit of 92 percent.

How let's see what happens when the threshold is exceeded.

bash-3.2$ sudo ./deleteoldfiles.sh /Volumes/usb 92

DELETE OLD FILES 1.00

Usage of 97% exceeded limit of 92 percent.
Deleting oldest file /Volumes/usb/a/file02.bin
Usage of 91% is within limit of 92 percent.

Here you notice that an old file is deleted and that the script checks again if there is now enough free space. If not, another file would have been deleted.

If you have a need for it, have fun. It was a fun little scripting exercise.

The script works under Linux and Mac OS X.

The current state of Linux is amazing. If we take a look at, for example, Ubuntu Linux, we have to admit that the Linux desktop is really becoming a nice, user-friendly environment. I'm truly starting to like what I see. I considered whiping Mac OS X from my macbook, but there are some reasons why I won't. 

• I like to mess around in Adobe Photoshop now and then. There is no serious Linux alternative.

• I've got an iPhone, so now i'm 'stuck' with iTunes. I really like my iPhone btw.

• I really like iPhoto, it is very easy to use and archive my photo's.

• My entire music collection is in iTunes -> especially the song ratings are important.

Maybe those reasons can be overcome. But for day-to-day usage, I still prefer Mac OS X above Linux. Especially software like iPhoto makes life so much easier. The hardware comes at a premium but I think it is worth it.