Louwrentius

Inotify is a mechanism in the Linux kernel that reports when a file system event occurs.

The inotifywait comand line utility can be used in shell scripts to monitor directories for new files. It can also be used to monitor files for changes. Inotifywait must be installed and is often not part of the base installation of your Linux distro. However, if you need to monitor a directory or some task like that, it is worth the effort. (apt-get install inotify-tools).

Here is how you use it:

inotifywait -m -r -e close_write /tmp/ | while read LINE; do echo $LINE | awk '{ print $1 $3 }'; done

Let's dissect this example one part at a time. The most interesting part is this:

inotifywait -m -r -e close_write /tmp/

What happens here? First, inotifywait monitors the /tmp directory. The monitoring mode is specified with the -m option, otherwise inotifywait would exit after the first event. The -r option specifies recursion, beware of large directory trees. The -e option is the most important part. You only want to be notified of new files if they are complete. So only after a close_write event should your script be notified of an event. A 'create' event for example, should not cause your script to perform any action, because the file would not be ready yet.

The remaining part of the example is just to get output like this:

/tmp/test1234/blablaf

/tmp/test123

/tmp/random.bin

This output can be used to use as an argument to other scripts or functions, in order to perform some kind of action on this file.

This mechanism is specific to Linux. So it is not a OS independent solution.

If you background a bash or shell process, how do you determine if it has finished? Since inter process communication is not possible using shell scripts, people often refer to while loops or other polling mechanisms to determine if some process has stopped.

However, the one player that knows best if a process has finished is the process itself. So if only this process could tell the parent or other process about this...

The solution is using a FIFO or 'pipe'. A listener process reads the pipe and executes a command for every message received through this pipe. This was already build-in into PPSS. However, PPSS had this dirty while loop that polls every x seconds to determine if there are still running workers. If not, PPSS finishes itself.

However, while loops and polling mechanisms are evil, dirty and bad. The nicest solution is to make PPSS fully asynchronous. To achieve this, every job must tell PPSS that is has finished. PPSS already has this listening process that listens to the pipe for commands. If this channel is used by workers to communicate that a worker is finished, PPSS will know when all workers are finished.

This makes PPSS a lot faster and responsive.