Articles in the Networking category

  1. Eztables: Simple Yet Powerful Firewall Configuration for Linux

    November 16, 2013

    I've created and released Eztables on Github. Anyone who ever has a need to setup a firewall on Linux may be interested in this project.

    It doesn't matter if you need to protect a laptop, server or want to setup a network firewall. Eztables supports it all.

    If you're not afraid to touch the command line and edit a text file, you may be quite pleased with Eztables.

    Go check it out!

  2. Linux: Script That Creates Table of Network Interface Properties

    August 15, 2013

    My server has 5 network interfaces and I wanted a quick overview of some properties. There may be an existing linux command for this but I couldn't find it so I quickly wrote my own script (download).

    This is the output:

    showinterfacesimage

    The only requirement for this script is that you have 'ethtool' installed.

    Update 2013-08-17

    I recreated the script in python (download) so I can just dynamically format the table and not use ugly hacks I used in the bash script.

    Tagged as : Linux Networking
  3. How to Compile HAProxy From Source and Setup a Basic Configuration

    August 14, 2013

    To learn more about HAProxy I decided to compile it from source and use it to load-balance traffic to louwrentius.com across two different web servers.

    I run HAProxy on a VPS based on Ubuntu 12.04 LTS. Let's dive right in.

    First, we need to download the source. Don't copy/pased the exact code, you should download the latest version of HAProxy.

    cd /usr/src
    wget "http://haproxy.1wt.eu/download/1.4/src/haproxy-1.4.24.tar.gz"
    tar xzf haproxy-1.4.24.tar.gz
    cd haproxy-1.4.24
    

    Before you can compile software, you must make sure you have a working build-environment. With Ubuntu or Debian, you should run:

    apt-get install build-essential
    

    If you open the README file in the root directory, you will find some detailed instructions on how to compile HAProxy, which is really straight-forward.

    Compiling HAProxy

    Best CPU performance

    The manual states that by default, it will compile HAProxy with no CPU-specific optimisations. To enable CPU-specific optimisations, you need to use the 'native' option.

    The extra argument we are supplying to 'make' wil be:

    CPU=native
    

    Libpcre support

    It recommends to compile HAproxy with libpcre as it provides way better performance than other libc PCRE implementations. You need to install libpcre like this:

    apt-get install libpcre3-dev
    

    The extra argument we are supplying to 'make' wil be:

    USE_PCRE=1
    

    Splicing support

    A Linux-specific feature is support for the splice() system call. This system call allows data to be moved between file descriptors within kernel space, not touching user space. It entirely depends on your setup if this feature will be of any use to you. As splicing can be disabled within the configuration file of HAProxy, I would recommend compiling HAProxy with support for splicing.

    The extra argument we are supplying to 'make' wil be:

    USE_LINUX_SPLICE=1
    

    Transparent mode support

    I learned that HAProxy also supports a transparent mode where it seems to 'spoof' the client IP-address to the backend servers. This way, the backend servers see the actual client IP-address, not the IP-address of the HAProxy load-balancer(s).

    For this setup to work, you need additional firewall rules and meet some routing requirements. I'm not sure why this would be important and the linked article also mentions a work-around where an additional HTTP-header is used: x-forwarded-for.

    I found this article about how to configure lighttpd to log the x-forwarded-for header. Here are some instructions for Ngnix.

    The extra argument we are supplying to 'make' wil be:

    USE_LINUX_TPROXY=1
    

    Encrypted password support

    It's possible to limit access to HAProxy features (like statistics) to specific users and their passwords. These passwords can be stored in plain-text or as a (more secure) hash of the password, using crypt.

    The extra argument we are supplying to 'make' wil be:

    USE_LIBCRYPT=1
    

    Compiling HAproxy

    If we would use all discussed options, our Make command would look like this:

    make TARGET=custom CPU=native USE_PCRE=1 USE_LIBCRYPT=1 USE_LINUX_SPLICE=1 USE_LINUX_TPROXY=1
    

    Installing HAproxy

    By default, HAProxy is installed in /usr/local/haproxy with the following command:

    make install
    

    If you want to start HAProxy at boot time, you need a startup script. HAProxy does provide a startup script for Redhat-based distro's, but not for Debian-based distros.

    HAProxy is also available pre-compiled as an Ubuntu or Debian package. These packages also contain a startup script. I used such a script and modified it to work with the HAProxy version I compiled from source. Basically, I only altered some paths, but you can find it here

    Configuration

    HAProxy is very versatile and the actual configuration will entirely depend on your specific needs. I will document some basic scenario's with some examples.

    HAProxy has many configuration options, but don't worry, those are often well-documented.

    Scenario 1: Load-balancing

    In this scenario, we have one load balancer based on HAProxy and it's goal is to load-balance traffic across two backend HTTP-servers.

    global
        daemon
        user haproxy
        group haproxy
        chroot /home/haproxy
        maxconn 256
    
    defaults
        mode http
        timeout connect 5000ms
        timeout client 50000ms
        timeout server 50000ms
    
    frontend http-in
        bind *:80
        default_backend servers
    
    backend servers
        balance roundrobin  
        server ws01 1.1.1.1:80 
        server ws02 1.1.1.2:80
    

    Reading the global section, we learn that HAProxy should run as a daemon, that it should run as a specific system user and thus drop all privileges after startup. It also should chroot to /home/haproxy, a directory which should be empty and not writable by the HAProxy user or group. HAProxy will permit at most 256 simultaneous connections.

    The defaults section learns us that we are running in HTTP mode. HAProxy can load-balance any TCP-traffic. In HTTP mode, it can understand and read HTTP header information and apply different actions, allowing for more control.

    Now we encounter the interesting part. The default_backend keyword shows that all traffic entering on TCP-port 80 should be directed to the backend 'servers'. The 'backend' section contains the actual backend servers that will be able to handle traffic. The load-balancing algorithm used is round-robin: every web server is used in turn. Visitor 1 hits webserver 1. Visitor 2 hits webserver 2. Visitor 3 hits webserver 1, and so on.

    Scenario 2: Fail-over

    In scenario 1, we only discussed load-balancing. However, if one of the servers becomes unavailable, users will be facing error-messages generated by HAProxy. This is often undesired, we want HAProxy to check the status of the backend servers and direct traffic only to servers that are available. HAProxy should not forward clients to backend servers that are not responsive.

    This desired behaviour requires a few extra options within the 'backend' section.

    backend servers
        balance roundrobin
        option httpchk
        server ws01 1.1.1.1:80 check inter 4000
        server ws02 1.1.1.2:80 check inter 4000
    

    This configration makes HAProxy check both backend webservers for every 4000ms (4 seconds) for availability. By default, HAProxy only tests if it's possible to make a TCP-connection with the webserver. Ofcourse, this will not always tell you if a webserver is properly operational. This is why 'option httpchk' is added to the configuration. HAProxy will then connect to the backend webserver and issue an HTTP OPTIONS-request, which will be a better gauge to determine if the web server service is active. With additional options you can make HAProxy request specific URIs.

    Additional configuration options

    Logging

    HAProxy supports logging to Syslog. You can configure it to log to the local syslog daemon, or to a centralised log server.

    global
        log 127.0.0.1 local0 debug
        log-tag haproxy
    

    All log messages are prefixed with 'haproxy'. They are sent to localhost and the verbosity is 'debug'.

    defaults
        log global
    
    frontend http-in
        log global
        option httplog clf
    

    Option httplog clf makes HAProxy log in a similar log format as Apache. A tool like AWstats can then easily parse the log and generate some statistics.

    backend servers
        log global
    

    The 'backend' section will only log messages related to the availability of backend servers. Actual request-logging is performed through the 'frontent' section.

    Prioritising backend servers

    Some backend servers may have more performance and bandwidth available then others. Using the 'weight' parameter, you can make sure that certain services get more traffic then others.

    backend servers
        balance roundrobin
        option httpchk
        server ws01 1.1.1.1:80 check inter 4000 weight 10
        server ws02 1.1.1.2:80 check inter 4000 weight 20
    

    In this example, webserver ws02 will receive twice as many request as webserver ws01. But the load will still be balanced across both webservers.

    Enabling statistics

    HAProxy has a build-in webpage that shows performance metrics and the status of backend hosts. This webpage is not enabled by default.

    defaults
        stats enable
        stats auth username:password
        stats uri /mystatspage
        stats refresh 5s
    

    Please note that with this configuration, the statistics page may be accessible from the internet. As the page may provide some information about your environment that could be of benefit to attackers, it's wise to configure strong passwords and to configure a uri that is not easy to predict/guess. Beware that the password is transmitted in clear-text!

    For security reasons I would recommend to have the statistics page only accessible from within your own network and not accessible directly from the internet in any way.

    In this next scenario I assume that the load balancer has two network interfaces and is connected to both the internet and an internal 'backend' network that uses IP-addresses in the 10.x.x.x range.

    For security reasons, I would bind the statistics web page to the 'backend' interface, so it will never be accessible through the internet.

    listen HAProxy-stats 10.0.10.10:81 stats enable stats auth user:pass stats uri /stats stats refresh 5s stats show-legends

    Final words

    This basic tutorial should leave you with an up-and-running HAProxy. There are some topics I did not discuss, like handling of SSL-traffic. HAProxy 1.4 does not support SSL but version 1.5 will have native SSL-support. In the mean time, you will need to use Ngnix or 'stud' for SSL-offloading.

  4. Overview of Open-Source Load Balancers

    August 07, 2013

    I was looking at open-source load balancing software and it seems that there isn't a nice overview except from this website, although many of the listed projects seem dead.

    I've made a selection of products that seem to be relevant. The biggest problem with open-source software is that projects are abandoned or unmaintained. So I created this table and added a column 'last product update' which gives you a feel for how active the project is.

    Product Last product update
    ngnix 2013 July
    Lighttpd November 2012
    HAproxy 2013 June
    Pound 2011 December
    Varnish 2013 June
    Zen Load Balancer 2013 February
    Apache 2013 July
    Linux Virtual Server Unmaintained?
    XLB HTTP Load Balancer 2009 February
    Octopus Load Balancer 2011 November
    Squid 2013 July
    Date of measurement: August 2013

    I currently don't have hands-on experience with these products. Some of those products are briefly discussed at this blog - worth a visit.

    There are many more products but most seem to be abandoned years ago. If you feel there are more products that are noteworthy but not in this list, feel free to contact me or comment about it.

    It seems that the top-3 web servers like ngnix, Apache and Lighttpd all have support for load balancing. It depends on your needs, time and knowledge if you want to invest in other products or stick with the web server software you know.

    At this location some people are talking about the pro's and con's of commercial off-the-shelve products vs. home-grown open-source solutions.

    Tagged as : load-balancing
  5. Example of a Home Networking Setup With VLANs

    February 05, 2012

    Updated October 24, 2012, see below.

    This post is a description of my home network setup based on gigabit ethernet. I did a non-standard trick with VLANs that may also be of interest to other people. I'm going to start with a diagram of the network. Just take a look (click to enlarge).

    home network

    I now have replaced my Mac Mini with a HP N40L router based on Ubuntu 12.04 LTS. This server is now placed in the basement. The managed netgear switch is swapped with the Airport extreme.

    home network

    Design

    I have a Mac mini running Linux that acts as my internet router. The closet that houses the cable modem is not a friendly environment for such a device and there is not a good location for it. The closet is also outside of my house, behind a door not too well protected. So this is why I keep my router inside my house.

    From this closet, one UTP cable terminates in the living room, the other in the basement. This configuration has a very big problem. How do I run two different networks over one wire?

    I have to connect my iMac to my 'internal' home network. However, the Mac mini must be connected to both the internet network segment (connected to the cable modem) and the home network. All through a single UTP cable.

    Therefore I use VLANs. I transport both the internet network and the local home network though one cable. VLAN 10 is for internet, VLAN 20 for my local home network. For this all to work you need managed switches that support 802.1q.

    How traffic flows

    So let's say that the server is accessing the internet to obtain the latest Linux security updates. How does this network traffic flow through the infrastructure (click to enlarge)?

    network flow

    All internet traffic must flow through the router. Thus, even if the traffic from the basement travels through the switch next to the cable modem, it must first travel to the router in the living room. There the router decides if the traffic is permitted to go out to the internet and thus enter the internet VLAN.

    Pros and cons

    Pros:

    • Just a single cable to the living room
    • no extra USB-based ethernet adapters required for the Mac mini
    • Mac mini resides in save and computer friendly environment

    Cons:

    • Managed switches supporting VLANs are relatively expensive

Page 2 / 4