Linux 2.6 IPsec VPNs

Update: Howtoforge has a good tutorial about setting up “roadwarrior” VPNs using IPsec & KAME.

Since I was unable to find a clear description of setting up an IPSec-based VPN using the native IPSec stack from USAGI and KAME userland tools that are part of Linux 2.6.x, I thought I would post a short explanation.


Network Topology

I am going to explain a very straightforward topology often found in the field. You have two networks, network A ( and network B ( Each network has router/gateway/firewall system, gateway A ( and gateway B ( Each of these gateway systems as an external, public, IP address: &

In this topology, our end goal is to allow a hosts on network A, say, to securely contact a host on network B, say, via an IPSec tunnel.

Note: In many cases, I will provide only a single example. You must ensure the other gateway is configured similarly, often by reversing the configuration. This is left as an exercise to the reader.


Setting up IPSec involves two steps: keying and policy. Normally, you use racoon to provide keying and setkey to establish policy. I will instead use racoon-tool as convenient shortcut. This tool was written by the Debian Maintainer of the racoon package in order to emulate some of the nice configuration syntaxt of the FreeS/WAN, an older IPSec implementation for Linux. It dynamically generates a racoon.conf (/var/lib/racoon/racoon.con) and also sets up the policy based on a single configuration file.

First, we simply state that racoon should use the notify priority when sending messages to syslogd:

        log: notify

It is helpful to create a %default peer and connection in order to avoid duplication of configuration directives:

        verify_identifier: on
        hash_algorithm[0]: sha1
        encryption_algorithm[0]: aes

The above src_ip directive simply says that our connections will be using our public IP.

Next, we define our peer, gatewayB, identifying it by address:

        peers_identifier: address

Finally, we define a policy so that packets from networkA to networkB are encrypted via gatewayB:

        admin_status: enabled

Unfortunately, I have no idea what the admin_status: enabled directive does, or even whether it is required.

You must also add a key to /etc/racoon/psk.txt for the remote gateway:

# Entry for gatewayB 0x2eba016ffc2314869ae9f9a3b8901a173242f0c8

A randomly generated key is best, and can be created with the following command:

$ dd if=/dev/random count=20 bs=1 | xxd -ps

(The xxd command is part of the vim package.)

Make sure that you can the racoon package configured to use racoon-tool by either editing /etc/defaults/racoon or reconfiguring the package. Also, you should probably reload the tool by executing /etc/init.d/racoon reload. You can then check that your policy is in effect by running setkey -DP. You should get something like the following, followed by a bunch of default policies:[any][any] any
        in ipsec
        created: Nov 18 23:01:24 2004  lastused:
        lifetime: 0(s) validtime: 0(s)
        spid=1512 seq=9 pid=9800
        refcnt=1[any][any] any
        out ipsec
        created: Nov 18 23:01:24 2004  lastused: Nov 18 23:05:31 2004
        lifetime: 0(s) validtime: 0(s)
        spid=1505 seq=8 pid=9800

Now that the policy is in effect, the kernel will ask the racoon daemon for a security association (SA), when it needs to deal with a packet matching the policy. Racoon will negotiate an SA with the remote gateway on the fly. (This will not happen until after we’ve setup the routing, below, but you can see it by using the setkey -D command.)


In order for a packet to match the policy we have setup, and also be forwarded to the remove gateway, we must adjust the routing table using the ip command (Provided by the Debian iproute2 package.).

The policy we have setup, says that packets coming from and going to must be encrypted and authenticated. This is achieved by adding a slightly weird entry to the routing table on gatewayA:

$ ip route add via src

And on gatewayB:

$ ip route add via src

Once you have added the new entries to the routing tables on both gateways, we test the tunnel. If you ping from gatewayA, and then execute setkey -D, you will see the new SA that has been automatically created.

This is the routing table you should have on gateway “A”:

root@gatewayA:/tmp# ip route show dev eth1  proto kernel  scope link  src via dev eth1  src dev eth0  proto kernel  scope link  src
default via dev eth1

Followed by the routing table present on gateway “B”:

root@gatewayB:/tmp# ip route show dev eth1  proto kernel  scope link  src dev eth0  proto kernel  scope link  src via dev eth1  src
default via dev eth1

In the above examples, & are the default gateways that connect your public networks. (The routers usually provided by your ISP.)

Note: if you have any firewall rules enabled, you are going to have hassles. Notice that the IPSec tunnel doesn’t have it’s own device: the encrypted packets are going through a public interface. This will confuse any firewall rules that expect to be able to categorize packets by interface. I intend to figure out a good solution and post it at a later date.

Update: I now have a fully working IPsec gateway/router/firewall. See this other post.

Update: It seems that when the external addresses of the gateways on are different subnets, which was not the case during my testing, you must add a route like ip route add OTHER_NETWORK via LOCAL_DEFAULT_GW src INTERNAL_IP. Otherwise, you will get an error: RTNETLINK answers: Network is unreachable. Please leave a comment if you can clarify this.