Configuring AH on Solaris

The first installment of this series covered the underlying IPsec protocols as well as how the Internet Key Exchange (IKE) works. This article will discuss configuring IPsec to protect the traffic between two Solaris hosts. In this case, data confidentiality between the two hosts is not of concern however the integrity of the data is. The reference design for this configuration is shown in Figure 1 below. Two Solaris hosts, both running Solaris 8 10/01, are connected via a switch. This is not a very typical scenario but one that can be used between two hosts that are protecting the confidentiality of the transmitted traffic through other means such as Kerberos or Secure Shell.

Figure 1 -- Solaris Authentication Header Tunnel Setup

Configuration Files and Utilities

Solaris 8 and 9 provide several utilities to control the configuration of the IPsec policies on a host. The two primary utilities are ipsecconf and ipseckey. ipsecconf configures the IPsec policy for the host. Inbound and outbound traffic is subject to the policy checks set forth through the ipsecconf utility. What action is to be taken with regards to the traffic is determined by whether the traffic matches a policy entry. ipsecconf can be run manually or at boot time. If the /etc/inet/ipsecinit.conf file exists, then ipsecconf is automatically invoked from the /etc/init.d/inetinit file.

The ipseckey utility is used to manage the security associations used to implement IPsec between hosts. It allows the root user to manually manipulate the security association databases. It can be used to set up SAs between hosts when automated key management is not possible. The default installation of Solaris 8 does not provide automated key management capabilities. These were introduced in Solaris 9 through the ikeadm and the ikecert utilties as well as the in.iked daemon.

/usr/sbin/ipsecconf

The ipsecconf utility configures the system wide IPsec policies on a host. Each entry input through the ipsecconf utility protects traffic in one direction so two entries are needed to fully protect traffic between two hosts. ipsecconf can take several arguments one of with is the -f flag (to read the IPsec policy information from a file) and another is the -l flag to provide a long listing of current IPsec policies in use. The structure of the ipsecinit.conf file is described in the section below.

/etc/inet/ipsecinit.conf

The /etc/inet/ipsecinit.conf file contains the policy information to be implemented by the ipsecconf utility. Each policy is defined on a separate line and follows a pre-defined format:

{pattern} action {properties} 

The syntax of the file is as shown above. Both the pattern and the properties fields are delineated by enclosure in {}. {pattern} is defined as the traffic pattern which will be applied to inbound and outbound traffic. If the inbound/outbound traffic matches the defined pattern then action will be applied to the traffic. The action is determined by the properties of the policy entry as defined in {properties}. The following are the defined name-value pairs that can be used to define {pattern}:

• Source Address/Prefix Length (saddr/plen)
• Destination Address/Prefix Length (daddr/plen)
• Source Mask (smask) -- not used if prefix length is given with the source address
• Destination Mask (dmask) -- not used if prefix length is given with the destination address
• Source Port (sport)
• Destination Port (dport)
• Upper Layer Protocol (ulp) -- TCP, UDP, or other layer 4 protocol

There are three defined actions possible:

• apply -- apply IPsec to the traffic matching the {pattern} as defined in the {properties}.
• permit -- permit the traffic if it matches the {pattern} and the {properties}. Otherwise discard the traffic.
• bypass -- If the traffic matches the {pattern} then bypass any policy checks. Bypass actions are evaluated before all others.

The policy {properties} are defined by the following name-value pairs:

• Authentication algorithms (auth_algs) -- indicates that AH is in use and defines which algorithms are being used, MD5 or SHA1
• Encryption algorithms (encr_algs) -- indicates that ESP is being used and defines which algorithms are in use for encrypting the traffic.
• Encryption/Authentication algorithms (encr_auth_algs) -- ESP is in use but is being used both for encrypting the data as well as authenticating the traffic
• Direction (dir) -- The direction which the entry is valid for. This can be one of two values: in or out.
• Security Association (sa) -- This property determines which security association will be applied to the traffic. This property can be one of two values: unique or shared. A value of unique will cause a new, unused security association to be established for any packets that match the policy {pattern} whereas a value of shared will use a pre-existing security association.

Examples help in understanding the format and the nature of the ipsecinit.conf file. The following examples utilize the reference network shown in Figure 1 above.

Example 1 -- Protecting Traffic between two hosts

In this example, all traffic between two hosts is to be protecting using just AH. This may be because the confidentiality of the traffic is protected by other means or that the traffic is not considered of a "sensitive" nature. The authentication algorithm used is MD5.

On host vpn1:

{saddr vpn1 daddr vpn2} apply {auth_algs MD5 sa shared}
{saddr vpn2 daddr vpn1} permit {auth_algs MD5}

The first line specifies that traffic originating from vpn1 destined for vpn2 should have the AH header added and that the authentication algorithm used is MD5. It also specifies that the security association (sa) is a pre-shared sa (discussed below). The second line specifies that traffic from host vpn2 destined for vpn1 should be permitted through it is encapsulated using AH and can be authenticated using the MD5 authentication algorithm. A similar configuration would be needed On host vpn2 in order to allow traffic to pass between the two hosts and be protected using AH:

{saddr vpn2 daddr vpn1} apply {auth_algs MD5 sa shared}
{saddr vpn1 daddr vpn2} permit {auth_algs MD5}

Example 2 -- Authenticating inbound traffic to telnet port

In this example all traffic bound for the telnet port must be authenticated using AH with MD5 (see Figure 2). Any telnet traffic from a host which is not using AH with MD5 is rejected (see Figure 3):

On vpn1:

{dport telnet} permit {auth_algs MD5 dir in}

Figure 2 -- Authenticated Telnet Success

Figure 3 -- Unauthenticated Telnet

In this example the IPsec policy can be used as a traffic filter to prevent unwanted access to certain services. This is very similar to the way IPsec can be used in Windows 2000 as a traffic filter to a host. While the ipsecinit.conf can be very complicated in nature it allows for a significant amount of flexibility. The following command loads the policies defined in the /etc/inet/ipsecinit.conf file:

/usr/sbin/ipsecconf -a /etc/inet/ipsecinit.conf

Calling the ipsecconf utility without any arguments will list the IPsec policies in effect on the host as shown in Figure 4.

Figure 4 -- IPsec Policy Listing

To flush the current IPsec policies use the -f flag to ipsecconf:

/usr/sbin/ipsecconf -f
/usr/sbin/ipseckey

Once the IPsec policies are defind a security association (SA) must be setup. This is done through the /usr/sbin/ipseckey utility. The SA can be setup in a file and then loaded into the system using the -f option to ipseckey or it can be done interactively through ipseckey as shown below. ipseckey's command language is more extensive than ipsecconf's. ipseckey allows for the addition, update, deletion, and monitoring of security associations. The add, delete, get, and update commands require additional parameters to be supplied to ipseckey . The typical format of a command to ipseckey is:

[command] [ah|esp] spi [spi #] dst [dst addr] [additional options]

Command can be one of the following values:

• add - add a security association to the database
• update - update a security association in the database. This command must be run from the interactive mode of ipseckey because of the fact that keying material would be visible through the ps command.
• delete - delete a specific sa from the database
• get - get information about a specific sa from the database
• flush - flush all current sa's in the database
• monitor - Permits the monitoring of keying information in the sa database.
• passive_monitor/pmonitor - This allows the monitoring of keying information in the sa database as well but leaves out certain messages that would be displayed using the monitor command
• dump - Display all security associations
• save - Save security associations to a file. The filename to write to must be provided as well.
• help - Provides a listing of possible commands.

As mentioned above SA information can be input either through the command line interface of ipseckey or by loading the SA information from a file using the -f [filename] option. To ensure that the SA database survives across system reboots it is best to save the database to a file and then load the database using the -f option. An example ipseckey SA configuration is shown below:

[root@vpn1 /]
16 # ipseckey
ipseckey> add ah spi 5346 src 10.89.144.244 dst 10.89.144.243 authalg HMAC-MD5 
  authkey AF123BCDE89C53482AA4221CF89E343D
ipseckey> add ah spi 6435 src 10.89.144.243 dst 10.89.144.244 authalg HMAC-MD5 
  authkey F123BCE6583132CF68DACB9FC8339D4B
ipseckey> dump
Base message (version 2) type DUMP, SA type AH.
Message length 136 bytes, seq=3, pid=27148.
SA: SADB_ASSOC spi=0x1923, replay=0, state=MATURE
SA: Authentication algorithm = HMAC-MD5
SA: flags=0x0 < >
SRC: Source address (proto=0/)
SRC: AF_INET:  port = 0, 10.89.144.243 (vpn1).
DST: Destination address (proto=0/)
DST: AF_INET:  port = 0, 10.89.144.244 (vpn2).
AKY: Authentication key.
AKY: f123bce6583132cf68dacb9fc8339d4b/128
 LT: Lifetime information
CLT: 0 bytes protected, 0 allocations used.
CLT: SA added at time Tue Aug 27 17:08:04 2002
CLT: Time now is Tue Aug 27 17:08:08 2002

Base message (version 2) type DUMP, SA type AH.
Message length 136 bytes, seq=3, pid=27148.
SA: SADB_ASSOC spi=0x14e2, replay=0, state=MATURE
SA: Authentication algorithm = HMAC-MD5
SA: flags=0x0 < >
SRC: Source address (proto=0/)
SRC: AF_INET:  port = 0, 10.89.144.244 (vpn2).
DST: Destination address (proto=0/)
DST: AF_INET:  port = 0, 10.89.144.243 (vpn1).
AKY: Authentication key.
AKY: af123bcde89c53482aa4221cf89e343d/128
 LT: Lifetime information
CLT: 0 bytes protected, 0 allocations used.
CLT: SA added at time Tue Aug 27 17:07:29 2002
CLT: Time now is Tue Aug 27 17:08:08 2002

Dump succeeded for SA type 0.
ipseckey> save ah /etc/inet/keyfile
ipseckey> ^D
[root@vpn1 inet]
71 # more /etc/inet/keyfile
# This key file was generated by the ipseckey(1m) command's 'save' feature.

# begin assoc
add ah      spi 0x1923 encr_alg 0 auth_alg hmac-md5 replay 0      
  src vpn1     dst vpn2         authkey f123bce6583132cf68dacb9fc8339d4b/128 
# end assoc

# begin assoc
add ah      spi 0x14e2 encr_alg 0 auth_alg hmac-md5 replay 0      
  src vpn2     dst vpn1         authkey af123bcde89c53482aa4221cf89e343d/128 
# end assoc

[root@vpn1 inet]
73 # ipseckey flush
[root@vpn1 inet]
74 # ipseckey dump
Dump succeeded for SA type 0.
[root@vpn1 inet]
75 # ipseckey -f /etc/inet/keyfile
[root@vpn1 inet]
76 # ipseckey dump
Base message (version 2) type DUMP, SA type AH.
Message length 136 bytes, seq=1, pid=27338.
SA: SADB_ASSOC spi=0x1923, replay=0, state=MATURE
SA: Authentication algorithm = HMAC-MD5
SA: flags=0x0 < >
SRC: Source address (proto=0/)
SRC: AF_INET:  port = 0, 10.89.144.243 (vpn1).
DST: Destination address (proto=0/)
DST: AF_INET:  port = 0, 10.89.144.244 (vpn2).
AKY: Authentication key.
AKY: f123bce6583132cf68dacb9fc8339d4b/128
 LT: Lifetime information
CLT: 0 bytes protected, 0 allocations used.
CLT: SA added at time Wed Aug 28 16:14:15 2002
CLT: Time now is Wed Aug 28 16:14:16 2002

Base message (version 2) type DUMP, SA type AH.
Message length 136 bytes, seq=1, pid=27338.
SA: SADB_ASSOC spi=0x14e2, replay=0, state=MATURE
SA: Authentication algorithm = HMAC-MD5
SA: flags=0x0 < >
SRC: Source address (proto=0/)
SRC: AF_INET:  port = 0, 10.89.144.244 (vpn2).
DST: Destination address (proto=0/)
DST: AF_INET:  port = 0, 10.89.144.243 (vpn1).
AKY: Authentication key.
AKY: af123bcde89c53482aa4221cf89e343d/128
 LT: Lifetime information
CLT: 0 bytes protected, 0 allocations used.
CLT: SA added at time Wed Aug 28 16:14:15 2002
CLT: Time now is Wed Aug 28 16:14:16 2002

Dump succeeded for SA type 0.

Returning to the reference network used in this article, the above example was performed on the host vpn1. This sets up two SAs: one from 10.89.144.243 -> 10.89.244.244 and a second SA from 10.89.144.244 -> 10.89.144.243. The same information must be entered on the second host, vpn2 but the source and destination IP addresses must be swapped. A simple startup script can be used to ensure that the IPsec SA database survives across a reboot:

[root@vpn1 inet]
78 # cat /etc/init.d/ipseckey
#
# Simple boot script to ensure that the IPsec security association (sa)
# database survives across reboots.
#
if [ -f /etc/inet/keyfile -a -f /etc/inet/ipsecinit.conf ]; then
     /usr/sbin/ipseckey -f /etc/inet/keyfile
fi
[root@vpn1 rc2.d]
79 # ln ../init.d/ipseckey S99ipseckey
[root@vpn1 rc2.d]
80 # ls -l S99ipseckey
-rw-r--r--   2 root     other        225 Aug 28 16:18 S99ipseckey

With the IPsec policies in place and the sa databases populated with the proper security associations the traffic between the hosts is protected using IPsec. Figure 5 below shows the capture of traffic on host vpn1 (10.89.144.243) without any IPsec policies in place. Note the lack of the AH header in the TCP packet.

Figure 5 - Traffic Capture without AH

Figure 6 shows the capture of traffic between the two hosts vpn1 and vpn2 This traffic was capture on host vpn1 (10.89.144.243) telneting to host vpn2 (10.89.144.244). The two systems have IPsec policies in place to permit traffic between the two systems as well as a shared security association. In this capture, note the presence of the AH header between the IP header and the TCP header.

Figure 6 - Traffic Capture with AH

The next article in this series will cover the configuration of two Solaris hosts as VPN gateways between two networks. Two configurations will be considered: using ESP alone for authentication and data confidentiality and AH in combination with ESP where AH will provide authentication and ESP will be used for data confidentiality.

To read the next installment in this series, click here.