Louwrentius

When a disk has no free space left, all kinds of trouble can occur.

Therefore, I've created a script that monitors the used space of a volume and deletes the oldest file if a certain threshold is reached.

The script will keep on deleting the oldest file present on disk until used capacity is below the threshold.

So you can tell the script to monitor volume /storage and delete old files if the used capacity is bigger than 95 percent.

The script works like this:

./deleteoldfiles.sh <mount point> <percentage>

The mount point represents a volume or physical disk. The percentage represents the maxium used capacity threshold.

The script reads the output of the 'df -h' command to determine 'disk' usage.

Example:

bash-3.2$ ./deleteoldfiles.sh /Volumes/usb 92

DELETE OLD FILES 1.00

Usage of 90% is within limit of 92 percent.

How let's see what happens when the threshold is exceeded.

bash-3.2$ sudo ./deleteoldfiles.sh /Volumes/usb 92

DELETE OLD FILES 1.00

Usage of 97% exceeded limit of 92 percent.
Deleting oldest file /Volumes/usb/a/file02.bin
Usage of 91% is within limit of 92 percent.

Here you notice that an old file is deleted and that the script checks again if there is now enough free space. If not, another file would have been deleted.

If you have a need for it, have fun. It was a fun little scripting exercise.

The script works under Linux and Mac OS X.

Over the last couple of years, Debian Linux has released new stable versions about every two years. This pace is great for progress, but there is a serious problem. This problem is related to their support for older Debian stable versions.

If you read the quote below from the Debian Security FAQ it will dawn upon you:

    Q:  How long will security updates be provided?
    A:  The security team tries to support a stable distribution 
        for about one year after the next stable distribution has 
        been released, except when another stable distribution is 
        released within this year. It is not possible to support 
        three distributions; supporting two simultaneously is 
        already difficult enough.

Translation: after 3 years, you must apt-get dist-upgrade or be screwed, you will no longer receive security updates!.

Apt-get dist-upgrade or die, so to speak.

The problem is that the whole apt-get dist-upgrade thing is cool and all, but in my experience, it doesn't work. Even a simple web server gets screwed up badly. You need to diff all config files and spend quite some time reviewing all changes and fixing the broken stuff.

I'd rather spend the time creating a new fresh Debian installation based on the new stable release than on tinkering with the aftermath of an apt-get dist-upgrade. But that also takes a lot of effort.

I want an operating system that will be supported for the next five years so I don't have to spend time on this upgrade process every 3 years for a system that is otherwise fully functional and rock solid.

To tease you a little bit: Microsoft Windows operating systems are supported for ages. But that's not an option for me, I stick with Linux, but Debian does not provide this kind of extended support.

But Ubuntu does.

Ubuntu releases LTS versions: Long Term Support versions that will receive security updates for at least 5 years.

All the goodness of Debian but with longer support. That is the reason my shop will switch to Ubuntu Server LTS.

I'd like to introduce the utility 'dstat'. Dstat provides detailed statistics about what is currently happening on your Linux box.

Dstat allows you to monitor the system load, disk troughput, disk io, network bandwith, and many more items.

Dstat is so valuable because it provides you with all required information on a single row that updates every so often. It is a great tool for debugging system performance. An example of the output of dstat:

Click on this image to view the dstat output at the original size. It will tell you a small story on what happened on the system.

As you can see, a copy action is going on between two disks. Then suddenly, some other process is writing data to the source disk. Both read and write performance drops. As soon as the additional writing process stops, the read and write performance of the still running copy process returns back to normal.

The real benefit of this utility is that it clearly provides almost all information you might want to know in a single line.

One of the most used options are the disk throughput and network throughput columns. By default, dstat displays the aggregated throughput for all disks and network cards. You can bypass this behavior by specifying individual disks or network devices with the -D or -N option, like:

dstat -D sda,sdb -N eth0,eth1 20

Please note that the '20' argument at the end specifies how often the screen gets updated. Thus every single row is the average of that 20 second time frame.

For a full overview of all options that are available, issue the 'dstat --list' command.

You may find it very useful.