Louwrentius

Introduction

Fio is a widely-used tool for performing storage benchmarks. Fio offers a lot of options to create a storage benchmark that would best reflect your needs. Fio allows you to assess if your storage solution is up to its task and how much headroom it has.

Fio outputs .json and .log files that need further processing if you would like to make nice charts. Charts may help better communicate your test results to other people.

To make graphs of Fio benchmark data, I've created fio-plot. With fio-plot you can generate charts like:

It's very common that you want to run multiple benchmarks with different parameters to compare results. To generate the data of the charts, many benchmarks need to be run. This process needs to be automated.

Automating Fio benchmarks

I've chosen to build my own tool to automate Fio benchmarking. This tool is called bench_fio and is part of fio-plot. I'm aware that - as part of fio - a tool called genfio is provided, to generate fio job files with multiple benchmarks. It's up to you what you want to use. Bench-fio is tailored to output data in a way that aligns with fio-plot.

Bench-fio allows you to benchmark loads with different iodepths, simultaneous jobs, block sizes and other parameters. A benchmark run can consist of hundreds of tests and take many hours.

When you run bench_fio, you can expect output like this:

████████████████████████████████████████████████████
    +++ Fio Benchmark Script +++

Job template:                  fio-job-template.fio
I/O Engine:                    libaio
Number of benchmarks:          98
Estimated duration:            1:38:00
Devices to be tested:          /dev/md0
Test mode (read/write):        randrw
IOdepth to be tested:          1 2 4 8 16 32 64
NumJobs to be tested:          1 2 4 8 16 32 64
Blocksize(s) to be tested:     4k
Time per test (s):             60
Mixed workload (% Read):       75 90

████████████████████████████████████████████████████
4% |█                        | - [0:04:02, 1:35:00]-]

Bench-fio runs real-time and shows the expected remaining time. It also shows all relevant parameters that have been configured for this benchmark run. This makes it easier to spot any mis-configurations.

Notice that this benchmark consists of 98 individual tests: iodepth x NumJobs x Mixed Workload parameters (7 x 7 x 2). With a standard of 60 seconds per benchmark

This is an example of the command-line syntax: :::text ./bench_fio --target /dev/md0 -t device --mode randrw -o RAID_ARRAY --readmix 75 90

More examples can be found here.

My home networking setup

I live in a two story apartment, with on the top floor my utilities closet and my living room. The bottom floor contains a bedroom with all my servers and networking gear.

So this is my setup (click for a bigger version):

I like to run my own router but my utilities closed is not the safest place in terms of security and climate.

By default, most people who run their own home router will use a box with two network interfaces, one connected to the (cable) modem and the other one connected to the home network.

I could have done the same thing, by running a cable from the modem to my router, and a second cable back up towards my closet (and livingroom).

However, I didn't want to run multiple cables from my utilities closed to my bedroom downstairs, I saw no need for that: because I can use VLANs.

The small 8-port switch in the closed is connected with a single (long) cable to the 24-port switch I have in my bedroom downstairs. This switch connects to my router and multiple servers.

I've setup a trunk between these two switches where my internet traffic flows over 'VLAN 100' and my home network uses 'VLAN 200'.

The router, an HP N40L, has only a single network interface. I just expose the two VLANS as 'tagged' and let the router route traffic between the two VLANS. No need for a second interface (as many home setups do).

So in my setup there are two trunks, one between the two switches and the other one between the bedroom switch and my router. All other devices are connected to untagged network ports, in their appropriate VLAN.

The small switch in the closet is responsible for carrying my home network to the switch in my living room.

The raspberry pi connects to my smart meter to collect information about my power and gas usage.