HAProxy: Load Balancing and High Availability

Prerequisites

• Operating system: Debian 12+, Ubuntu 22.04+ or CentOS/RHEL 8+
• Privileges: root or sudo access
• Network: At least 2 backend servers to test load balancing
• Resources: 1 vCPU, 512 MB RAM minimum (HAProxy is very lightweight)
• Ports: 80, 443 available on the HAProxy server

Installation

Installation via the repositories (Debian/Ubuntu)

The simplest method to install HAProxy on Debian or Ubuntu:

sudo apt update
sudo apt install -y haproxy

Check the installed version:

haproxy -v

Installation from the official PPA (recent version)

To get a more recent version with the latest features:

sudo apt install -y software-properties-common
sudo add-apt-repository ppa:vbernat/haproxy-2.8 -y
sudo apt update
sudo apt install -y haproxy=2.8.*

Compiling from source

For full control over the compilation options:

# Build dependencies
sudo apt install -y build-essential libssl-dev libpcre3-dev zlib1g-dev

# Download and compile
wget https://www.haproxy.org/download/2.8/src/haproxy-2.8.5.tar.gz
tar xzf haproxy-2.8.5.tar.gz
cd haproxy-2.8.5

make TARGET=linux-glibc USE_OPENSSL=1 USE_PCRE=1 USE_ZLIB=1
sudo make install

Enable and start the service:

sudo systemctl enable haproxy
sudo systemctl start haproxy
sudo systemctl status haproxy

HAProxy Architecture

HAProxy relies on three main components that define the traffic flow:

• Frontend: Entry point for traffic. It defines the listening address and port, the protocol (HTTP/TCP) and the routing rules to the backends.
• Backend: Group of servers that receive the traffic. It defines the distribution algorithm, the health checks and the list of target servers.
• Listen: Combination of a frontend and a backend in a single block. Convenient for simple configurations.

                   +-----------+
  Client --------->| Frontend  |
                   | (port 80) |
                   +-----+-----+
                         |
                    ACL / Routing
                    /
           +-------+--+    +--+-------+
           | Backend  |    | Backend  |
           |   web    |    |   api    |
           +--+----+--+    +--+----+--+
              |    |          |    |
           srv1  srv2      srv3  srv4

Basic Configuration

The main configuration file is located at /etc/haproxy/haproxy.cfg. It is structured into sections:

global section

System parameters that apply to the entire HAProxy process:

global
    log /dev/log local0
    log /dev/log local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

    # SSL parameters
    ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384
    ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets

defaults section

Default values inherited by the frontends and backends:

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

HTTP Load Balancing

The most common configuration consists of distributing HTTP traffic across several web servers.

Frontend and backend configuration

frontend http_front
    bind *:80
    default_backend http_back

backend http_back
    balance roundrobin
    option httpchk GET /health
    http-check expect status 200
    server web1 192.168.1.10:80 check inter 3s fall 3 rise 2
    server web2 192.168.1.11:80 check inter 3s fall 3 rise 2
    server web3 192.168.1.12:80 check inter 3s fall 3 rise 2 backup

Distribution algorithms

HAProxy offers several algorithms suited to different use cases:

# Roundrobin: cyclic distribution (default)
balance roundrobin

# Leastconn: server with the fewest active connections
# Ideal for requests of varying durations
balance leastconn

# Source: session affinity by source IP
# The same client always goes to the same server
balance source

# URI: hash on the URI to optimize caching
balance uri

# First: fills one server before moving to the next
# Useful for reducing the number of active servers
balance first

HTTP health checks

Health checks verify that the backend servers are operational:

backend http_back
    balance roundrobin
    option httpchk GET /health
    http-check expect status 200

    # inter: interval between checks (default 2s)
    # fall: number of failures before declaring DOWN
    # rise: number of successes before declaring UP
    server web1 192.168.1.10:80 check inter 3s fall 3 rise 2 weight 100
    server web2 192.168.1.11:80 check inter 3s fall 3 rise 2 weight 100

TCP Load Balancing

HAProxy can also distribute raw TCP traffic for services such as MySQL, PostgreSQL or Redis.

MySQL load balancing (reads)

listen mysql_cluster
    bind *:3306
    mode tcp
    balance leastconn
    option mysql-check user haproxy_check

    server mysql1 192.168.1.20:3306 check inter 5s fall 3 rise 2
    server mysql2 192.168.1.21:3306 check inter 5s fall 3 rise 2
    server mysql3 192.168.1.22:3306 check inter 5s fall 3 rise 2 backup

Redis load balancing

listen redis_cluster
    bind *:6379
    mode tcp
    balance first
    option tcp-check

    tcp-check send PING

    tcp-check expect string +PONG

    server redis1 192.168.1.30:6379 check inter 3s fall 3 rise 2
    server redis2 192.168.1.31:6379 check inter 3s fall 3 rise 2

SSL/TLS Termination

HAProxy can handle SSL termination to offload cryptographic processing from your backend servers.

Preparing the certificate

HAProxy requires a PEM file combining the certificate and the private key:

# Combine certificate + private key + intermediate chain
cat /etc/letsencrypt/live/example.com/fullchain.pem \
    /etc/letsencrypt/live/example.com/privkey.pem \
    > /etc/haproxy/certs/example.com.pem

# Secure the file
chmod 600 /etc/haproxy/certs/example.com.pem

HTTPS configuration with redirection

frontend https_front
    bind *:80
    bind *:443 ssl crt /etc/haproxy/certs/example.com.pem

    # HTTP to HTTPS redirection
    http-request redirect scheme https unless { ssl_fc }

    # Security headers
    http-response set-header Strict-Transport-Security "max-age=31536000; includeSubDomains"

    default_backend http_back

backend http_back
    balance roundrobin
    # Forward the client's real IP
    option forwardfor
    http-request set-header X-Forwarded-Proto https if { ssl_fc }

    server web1 192.168.1.10:80 check
    server web2 192.168.1.11:80 check

ACLs and Routing

ACLs (Access Control Lists) let you route traffic conditionally based on various criteria.

Routing by domain name

frontend http_front
    bind *:80

    # ACLs based on the Host header
    acl is_api hdr(host) -i api.example.com
    acl is_app hdr(host) -i app.example.com
    acl is_static hdr(host) -i static.example.com

    # Routing to the matching backends
    use_backend api_back if is_api
    use_backend app_back if is_app
    use_backend static_back if is_static
    default_backend app_back

Routing by URL path

frontend http_front
    bind *:80

    # ACLs based on the path
    acl is_api path_beg /api/
    acl is_admin path_beg /admin/
    acl is_websocket hdr(Upgrade) -i websocket

    use_backend api_back if is_api
    use_backend admin_back if is_admin
    use_backend ws_back if is_websocket
    default_backend web_back

Routing by header and HTTP method

frontend http_front
    bind *:80

    # Routing by HTTP method
    acl is_post method POST
    acl is_get method GET

    # Routing by custom header
    acl is_mobile hdr_sub(User-Agent) -i mobile
    acl is_json hdr(Accept) -i application/json

    use_backend api_write if is_post is_api
    use_backend api_read if is_get is_api
    use_backend mobile_back if is_mobile

Advanced Health Checks

Health checks allow HAProxy to detect and automatically remove failing servers.

HTTP health checks

backend web_back
    option httpchk
    http-check send meth GET uri /health ver HTTP/1.1 hdr Host www.example.com
    http-check expect status 200

    # Check with expected content
    http-check expect string "status":"ok"

    server web1 192.168.1.10:80 check inter 5s fall 3 rise 2
    server web2 192.168.1.11:80 check inter 5s fall 3 rise 2

Custom TCP health checks

backend smtp_back
    mode tcp
    option tcp-check

    tcp-check connect
    tcp-check expect string 220
    tcp-check send EHLO haproxy

    tcp-check expect string 250
    tcp-check send QUIT


    server smtp1 192.168.1.40:25 check

Health check options

backend app_back
    # Check every 5 seconds
    # DOWN after 3 consecutive failures
    # UP after 2 consecutive successes
    server app1 192.168.1.10:8080 check inter 5s fall 3 rise 2

    # Server in drain mode (finishes current sessions)
    server app2 192.168.1.11:8080 check drain

    # Backup server (enabled only if all others are DOWN)
    server app3 192.168.1.12:8080 check backup

    # Slowstart: gradual ramp-up of the load (60 seconds)
    server app4 192.168.1.13:8080 check slowstart 60s

Statistics and Monitoring

HAProxy includes a very comprehensive statistics page to monitor the state of your infrastructure in real time.

Enabling the stats page

listen stats
    bind *:8404
    stats enable
    stats uri /stats
    stats refresh 10s
    stats admin if LOCALHOST
    stats auth admin:YourPassword

Then go to http://your-server:8404/stats to view the metrics.

Available metrics

• Sessions: Active and total connections, rate per second
• Bytes: Inbound and outbound traffic per server
• Status: State of each server (UP, DOWN, DRAIN, MAINT)
• Response time: Average, max, percentiles
• Errors: Connection errors, timeouts, HTTP 4xx/5xx codes

Prometheus export

HAProxy 2.x natively supports exporting metrics in Prometheus format:

frontend prometheus
    bind *:8405
    http-request use-service prometheus-exporter if { path /metrics }
    no log

High Availability

To prevent HAProxy itself from becoming a single point of failure (SPOF), deploy two instances with Keepalived.

Active/passive architecture

              Virtual IP (VIP)
              192.168.1.100
                    |
            +-------+-------+
            |               |
    +-------+---+   +---+-------+
    | HAProxy 1 |   | HAProxy 2 |
    |  (MASTER) |   |  (BACKUP) |
    +-----+-----+   +-----+-----+
          |               |
    +-----+-----+---------+-----+
    |           |               |
  srv1        srv2            srv3

Installing and configuring Keepalived

sudo apt install -y keepalived

MASTER node configuration (/etc/keepalived/keepalived.conf):

vrrp_script check_haproxy {
    script "/usr/bin/killall -0 haproxy"
    interval 2
    weight 2
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 101
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass YourPassword
    }

    virtual_ipaddress {
        192.168.1.100/24
    }

    track_script {
        check_haproxy
    }
}

BACKUP node configuration (identical except):

vrrp_instance VI_1 {
    state BACKUP
    priority 100
    # The rest is identical to the MASTER
}

# Start Keepalived on both nodes
sudo systemctl enable keepalived
sudo systemctl start keepalived

# Check that the VIP is active on the MASTER
ip addr show eth0 | grep 192.168.1.100

Securing

Rate limiting

Protect your backends against abuse by limiting the number of requests per IP:

frontend http_front
    bind *:80

    # Tracking table by IP
    stick-table type ip size 100k expire 30s store http_req_rate(10s)

    # Track requests by IP
    http-request track-sc0 src

    # Block if more than 100 requests in 10 seconds
    http-request deny deny_status 429 if { sc_http_req_rate(0) gt 100 }

Blacklist and whitelist

frontend http_front
    bind *:80

    # Blacklist from a file
    acl is_blacklisted src -f /etc/haproxy/blacklist.txt
    http-request deny if is_blacklisted

    # Whitelist for administration
    acl is_admin_ip src 10.0.0.0/8
    acl is_admin_path path_beg /admin
    http-request deny if is_admin_path !is_admin_ip

Security headers

frontend http_front
    bind *:443 ssl crt /etc/haproxy/certs/example.com.pem

    # Security headers
    http-response set-header X-Frame-Options DENY
    http-response set-header X-Content-Type-Options nosniff
    http-response set-header X-XSS-Protection "1; mode=block"
    http-response set-header Referrer-Policy strict-origin-when-cross-origin
    http-response set-header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"

    # Hide the backend server version
    http-response del-header Server

Troubleshooting

Checking the configuration

# Validate the configuration syntax
sudo haproxy -c -f /etc/haproxy/haproxy.cfg

# Reload without service interruption
sudo systemctl reload haproxy

Diagnostic commands

# View the HAProxy logs
sudo journalctl -u haproxy -f

# Check listening ports
sudo ss -tlnp | grep haproxy

# Interact with the admin socket
echo "show stat" | sudo socat stdio /run/haproxy/admin.sock

# Show the state of the backends
echo "show servers state" | sudo socat stdio /run/haproxy/admin.sock

# Disable a server for maintenance
echo "disable server http_back/web1" | sudo socat stdio /run/haproxy/admin.sock

# Re-enable a server
echo "enable server http_back/web1" | sudo socat stdio /run/haproxy/admin.sock

Common errors

• 503 Service Unavailable: All backend servers are DOWN. Check the health checks and network connectivity.
• 502 Bad Gateway: The backend responds but with an error. Check the application server logs.
• 408 Request Timeout: The client or backend is too slow. Increase the timeout client or timeout server values.
• Connection refused: Check that the backend service is actually listening on the configured port.

Conclusion

HAProxy is an essential tool for any infrastructure that requires load distribution and high availability. With this tutorial, you have the knowledge to:

• Install and configure HAProxy on a Linux server
• Distribute HTTP and TCP traffic across several servers
• Set up SSL/TLS termination
• Route traffic with conditional ACLs
• Monitor your infrastructure via the built-in statistics
• Ensure high availability with Keepalived
• Secure your load balancer with rate limiting and headers

To go further, explore HAProxy's advanced features such as PROXY protocol support, dynamic maps, Lua scripts for custom logic, and multithreading to fully leverage multi-core servers.