How to load balance 4 Modem Cable internet access

by Damia Soler
v1.0, Oct 2005

Introduction

Every year I assist to Benaguasil party and colabotate with the organization on technical issues, mainly Internet access, each year we have a different scenario, the sponsors one year give us an ADSL, other year a Cable modem, even a wifi access. We always have a Linux server that deals with the connection and give access to the LAN-party doing NAT,or proxy.

This year (2005) we had four Cable Modems each one with 4MB download rate. The exact equipment were four cable modems with one Ethernet port with DHCP assignement (with no configuration options and no NAT option only single mode). We have a Linux server with five ethernet adapters.

And we want to give this download thougput (4Mb x 4) to the LAN party.

Aproaches

We made a brainstorming thinking about how to use all four connections, some ideas, only one of them developed (Are all other posibles?).

-We thought to use a comercial solution, like linkproof from radware, probably we could obtain one borrowed, but we have no clear experience about that product with four ethernets with dhcp on each port. But more important we want to make it work with free software :-) only with the Linux server.

-SQUID: A network of squid caches, each one connected to a different Cable modem. Is posible to one squid on a multihomed host to balance connections? All questions unknow.

-To distribute the hosts on LAN, giving each group of hosts a different default gateway. The problem was that we need four host to NAT, due that Modem Cable don't allow NAT, single IP for each one. We had only one PC, to do everything, with for Pcs, or NAT capable routers this aproach could be achieved, but we didn't try due to lack of PCs.

-To statically NAT and route routing each group of client host for a specific gateway. This is very similar to the case before, but done with only on Pc, with source routing and NAT. I thought it is posible to do, buy we didn't make it works.

-To statically NAT and route routing each group of protocols for a specific gateway. This is basically the same, but making policiy routing works giving each protocol a routing gateway thought different cable modem, this solutions have an special characteristic, you could assign web trafic a Cable modem, P2P other Cable modem, having some QOS a service control.

-To dynamicaly balance route and NAT. Better that do assignments of clients or protocols, we try this aproach that balance dinamically each line. This is what we are going to explain how we did it :-)

Choosing distribution

I am a Debian fan, and then don't try to install any other distribution. Then we install Debian sarge, however this tutorial have no very specific distribution issues, and I think it could work on any distribution with iptables toools, etc..

The setup

We have five ethernet interfaces, one of them is the internal LAN interface with a static IP, all others have dhcp ip assigned by the cable modem Operator Ono, our /etc/network/interfaces was:

# The loopback interface
# automatically added when upgrading
auto lo
iface lo inet loopback
# automatically added when upgrading
auto eth0
iface eth0 inet static
            address 192.168.1.1
            netmask 255.255.0.0
##
auto eth1
iface eth1 inet dhcp
##
auto eth2
iface eth2 inet dhcp
##
auto eth3
iface eth3 inet dhcp
##
auto eth4
iface eth4 inet dhcp

Restating networking we had the ip assignement:

To see the default route of each interface we use netstat -nrv and we saw that we have two diffentet default gateways.

Then we create two scripts, that makes the line balancer works…

route del default
route del default
route del default
route del default
ip route add default equalize scope global \
                        nexthop via 81.203.144.1 dev eth1 weight 1 \
                        nexthop via 81.203.144.1 dev eth2 weight 1 \
                        nexthop via 81.203.128.1 dev eth3 weight 1 \
                        nexthop via 81.203.144.1 dev eth4 weight 1

To check everything OK, netstat -nrv is not useful, you have to use ip route list.

IPTABLES=/sbin/iptables

$IPTABLES -t nat -A POSTROUTING -o $EXTIF1 -j SNAT --to-source 81.203.149.126
$IPTABLES -t nat -A POSTROUTING -o $EXTIF2 -j SNAT --to-source 81.203.144.81
$IPTABLES -t nat -A POSTROUTING -o $EXTIF3 -j SNAT --to-source 81.203.128.69
$IPTABLES -t nat -A POSTROUTING -o $EXTIF4 -j SNAT --to-source 81.203.144.88

To see thing woking, one useful tool is iptraf :-)

Problems and TODO

How can we see the default gateway assignement on each interface, how can we use it autoamtically on our scripts.

How the balance is performed? Could be tuned or customized?

Links

At the moment:

• Benaguasil party

• Damia

• Some coments form linuxpakistan