Transparent web-caching using netfilter, iproute2, ipchains and squid

Linux Advanced Routing & Traffic Control HOWTO
Prev | Chapter 15. Cookbook | Next

# 15.5. Transparent web-caching using netfilter, iproute2, ipchains and squid

This section was sent in by reader Ram Narula from Internet for Education (Thailand).

The regular technique in accomplishing this in Linux is probably with use of ipchains AFTER making sure that the "outgoing" port 80(web) traffic gets routed through the server running squid.

There are 3 common methods to make sure "outgoing" port 80 traffic gets routed to the server running squid and 4th one is being introduced here.

Making the gateway router do it.

  If you can tell your gateway router to match packets that has outgoing destination port of 80 to be sent to the IP address of squid server.

  BUT

  This would put additional load on the router and some commercial routers might not even support this.

Using a Layer 4 switch.

  Layer 4 switches can handle this without any problem.

  BUT

  The cost for this equipment is usually very high. Typical layer 4 switch would normally cost more than a typical router+good linux server.

Using cache server as network's gateway.

  You can force ALL traffic through cache server.

  BUT

  This is quite risky because Squid does utilize lots of CPU power which might result in slower over-all network performance or the server itself might crash and no one on the network will be able to access the Internet if that occurs.

Linux+NetFilter router.

  By using NetFilter another technique can be implemented which is using NetFilter for "mark"ing the packets with destination port 80 and using iproute2 to route the "mark"ed packets to the Squid server.

    |----------------|
    | Implementation |
    |----------------|

     Addresses used
     10.0.0.1 naret (NetFilter server)
     10.0.0.2 silom (Squid server)
     10.0.0.3 donmuang (Router connected to the Internet)
     10.0.0.4 kaosarn (other server on network)
     10.0.0.5 RAS
     10.0.0.0/24 main network
     10.0.0.0/19 total network

    |---------------|
    |Network diagram|
    |---------------|

    Internet
    |
    donmuang
    |
    ------------hub/switch----------
    |        |             |       |
    naret   silom        kaosarn  RAS etc.

First, make all traffic pass through naret by making sure it is the default gateway except for silom. Silom's default gateway has to be donmuang (10.0.0.3) or this would create web traffic loop.

(all servers on my network had 10.0.0.1 as the default gateway which was the former IP address of donmuang router so what I did was changed the IP address of donmuang to 10.0.0.3 and gave naret ip address of 10.0.0.1)

    Silom
    -----
    -setup squid and ipchains

Setup Squid server on silom, make sure it does support transparent caching/proxying, the default port is usually 3128, so all traffic for port 80 has to be redirected to port 3128 locally. This can be done by using ipchains with the following:

    silom# ipchains -N allow1
    silom# ipchains -A allow1 -p TCP -s 10.0.0.0/19 -d 0/0 80 -j REDIRECT 3128
    silom# ipchains -I input -j allow1

Or, in netfilter lingo:

    silom# iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128

(note: you might have other entries as well)

For more information on setting Squid server please refer to Squid FAQ page on http://squid.nlanr.net).

Make sure ip forwarding is enabled on this server and the default gateway for this server is donmuang router (NOT naret).

    Naret
    -----
    -setup iptables and iproute2
    -disable icmp REDIRECT messages (if needed)

1. "Mark" packets of destination port 80 with value 2

       naret# iptables -A PREROUTING -i eth0 -t mangle -p tcp --dport 80 \
        -j MARK --set-mark 2

2. Setup iproute2 so it will route packets with "mark" 2 to silom

       naret# echo 202 www.out >> /etc/iproute2/rt_tables
       naret# ip rule add fwmark 2 table www.out
       naret# ip route add default via 10.0.0.2 dev eth0 table www.out
       naret# ip route flush cache

   If donmuang and naret is on the same subnet then naret should not send out icmp REDIRECT messages. In this case it is, so icmp REDIRECTs has to be disabled by:

       naret# echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
       naret# echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
       naret# echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects

The setup is complete, check the configuration

    On naret:

    naret# iptables -t mangle -L
    Chain PREROUTING (policy ACCEPT)
    target     prot opt source               destination
    MARK       tcp  --  anywhere             anywhere           tcp dpt:www MARK set 0x2

    Chain OUTPUT (policy ACCEPT)
    target     prot opt source               destination

    naret# ip rule ls
    0:      from all lookup local
    32765:  from all fwmark        2 lookup www.out
    32766:  from all lookup main
    32767:  from all lookup default

    naret# ip route list table www.out
    default via 203.114.224.8 dev eth0

    naret# ip route
    10.0.0.1 dev eth0  scope link
    10.0.0.0/24 dev eth0  proto kernel  scope link  src 10.0.0.1
    127.0.0.0/8 dev lo  scope link
    default via 10.0.0.3 dev eth0

    (make sure silom belongs to one of the above lines, in this case
    it's the line with 10.0.0.0/24)

    |------|
    |-DONE-|
    |------|

## 15.5.1. Traffic flow diagram after implementation

    |-----------------------------------------|
    |Traffic flow diagram after implementation|
    |-----------------------------------------|

    INTERNET
    /\
    ||
    \/
    -----------------donmuang router---------------------
    /\                                      /\         ||
    ||                                      ||         ||
    ||                                      \/         ||
    naret                                  silom       ||
    *destination port 80 traffic=========>(cache)      ||
    /\                                      ||         ||
    ||                                      \/         \/
    \\===================================kaosarn, RAS, etc.

Note that the network is asymmetric as there is one extra hop on general outgoing path.

    Here is run down for packet traversing the network from kaosarn
    to and from the Internet.

    For web/http traffic:
    kaosarn http request->naret->silom->donmuang->internet
    http replies from Internet->donmuang->silom->kaosarn

    For non-web/http requests(eg. telnet):
    kaosarn outgoing data->naret->donmuang->internet
    incoming data from Internet->donmuang->kaosarn

Prev | Home | Next
Prioritizing interactive traffic | Up | Circumventing Path MTU Discovery issues with per route MTU settings