Put IPsec to work in YOUR application

Hello coders!

Most people know that you can use ipsecconf(1m) to apply IPsec policy enforcement to an existing application. For example, if you wish to only allow inbound telnet traffic that's under IPsec protection, you'd put something like this into /etc/inet/ipsecinit.conf or other ipsecconf(1m) input:

# Inbound telnet traffic should be IPsec protected
{ lport 23 } ipsec { encr_algs any(128..) encr_auth_algs md5 sa shared}
             or ipsec { encr_algs any(128..) encr_auth_algs sha1 sa shared}

Combine that with appropriate IKE configuration or manual IPsec keys, and you can secure your telnet traffic against eavesdropping, connection hijacking, etc.

For existing services, using ipsecconf(1m) is the most expedient way to bring IPsec protection to bear on packets.

For new services, or services that are being modified anyway, consider using per-socket policy as an alternative. Some advantages to per-socket policy are:

• Per-socket policy is stored internally in network session state (the conn_t structure in OpenSolaris). Entries from ipsecconf(1m) are stored in the global Security Policy Database (SPD). No global SPD entries means lower latency for fresh flow creation, and less lock acquisition.


• Per-socket bypass means fewer bypass entries in global SPD. If I bypass remote-port 80 using ipsecconf(1m), I can, in theory, enter the system with a remote TCP packet with port=80. There's an RFE (6219908) to work around this, but per-socket is still quicker. I'd love a web proxy with the ability to set per-socket bypass.

The newly SMF-ized inetd(1m) would be a prime candidate for per-socket policy. See RFE 6226853, and this might be something someone in the OpenSolaris community would like to tackle!

Let's look at the ipsec_req_t structure that's been around since Solaris 8 in /usr/include/netinet/in.h:

/*
 * Different preferences that can be requested from IPSEC protocols.
 */
#define IP_SEC_OPT              0x22    /* Used to set IPSEC options */
#define IPSEC_PREF_NEVER        0x01
#define IPSEC_PREF_REQUIRED     0x02
#define IPSEC_PREF_UNIQUE       0x04
/*
 * This can be used with the setsockopt() call to set per socket security
 * options. When the application uses per-socket API, we will reflect
 * the request on both outbound and inbound packets.
 */

typedef struct ipsec_req {
        uint_t          ipsr_ah_req;            /* AH request */
        uint_t          ipsr_esp_req;           /* ESP request */
        uint_t          ipsr_self_encap_req;    /* Self-Encap request */
        uint8_t         ipsr_auth_alg;          /* Auth algs for AH */
        uint8_t         ipsr_esp_alg;           /* Encr algs for ESP */
        uint8_t         ipsr_esp_auth_alg;      /* Auth algs for ESP */
} ipsec_req_t;

The ipsec_req_t is a subset of what one can specify with ipsecconf(1m) in Solaris 9 or later, but it matched what one could do with Solaris 8's version. Algorithm values are derived from PF_KEY (see /usr/include/net/pfkeyv2.h for values), as below. One could also use getipsecalgbyname(3nsl). If I wish to set a socket to use ESP with AES and and MD5, I'd set it up as follows:

        int s;  /* Socket file descriptor... */
        ipsec_req_t ipsr;

        .....

        /* NOTE: Do this BEFORE calling connect() or accept() for TCP sockets. */
        ipsr.ipsr_ah_req = 0;
        ipsr.ipsr_esp_req = IPSEC_PREF_REQUIRED;
        ipsr.ipsr_self_encap_req = 0;
        ipsr.ipsr_auth_alg = 0;
        ipsr.ipsr_esp_alg = SADB_EALG_AES;
        ipsr.ipsr_esp_auth_alg = SADB_AALG_MD5HMAC;
        if (setsockopt(s, IPPROTO_IP, IP_SEC_OPT, &ipsr, sizeof (ipsr)) == -1) {
                perror("setsockopt");
                bail();  /* Ugggh, we failed. */
        }
        /* You now have per-socket policy set. */

Notice I mentioned setting the socket option BEFORE calling connect() or accept? This is because of a phenomenon we implement called connection latching. Basically, connection latching means that once an endpoint is connect()-ed, the IPsec policy (whether set per-socket or inherited from the state of the global SPD at the time) latches in place. We made this decision to avoid keeping policy-per-datagram state for things like TCP retransmits.

One thing per-socket policy does not address is the case of unconnected datagram services. In a perfect world, we could have IPsec policy information percolate all the way to the socket layer, where an application can make fully-informed per-datagram decisions on whether or not a particular packet was secured or not. It's a hard problem, requiring XNET sockets (to use sendmsg() and recvmsg() with ancillary data).

BTW, if you want to bypass whatever global entries are in the SPD, you can zero out the structure, and set all three (ah, esp, self_encap) action indicators to IPSEC_PREF_NEVER. You need to be privileged (root or "sys_net_config") to use per-socket bypass, however.

So modulo the keying problem (setting up IKE or having both ends agree on IPsec manual keys), you can put IPsec to work right in your application. In fact, if you use IKE, you can let IKE sort out permissions and access control (by using PKI-issued certificates, self-signed certificates, or preshared keys) and have policy merely determine the details of the protection required.

---

EDITED: This entry brought to you by the Technorati tags IPsec, Solaris, and OpenSolaris.