Unnumbered interfaces confuse Quagga

The whole reason I was reading e-mail on a Sunday was not to look for telnetd exploits.

I was logged in because Team IPsec runs its punchin IPsec remote-access server (sometimes called a VPN server, but I hate that term because it's pushed by too many middlebox vendors) which was having routing problems.

As stated before, Solaris implements tunnels as point-to-point interfaces. For a remote-access server like we have in punchin, this means every external IP address gets a tunnel interface. (Until we had Tunnel Reform, this meant only one client per external IP address, which messed up NATs for multiple clients.) A tunnel interface has two addresses - a local one and a remote one. The local one can be shared with other tunnels or even with a different local interface (like the local ethernet). Such interfaces are called unnumbered interfaces.

A remote access server does forward packets, and is therefore by definition a router. One of our servers just swapped out Zebra (from older OpenSolaris/Nevada build) to Quagga. We use Quagga's OSPF to learn the topology of the Sun internal network (the SWAN).

As clients "punch out", their tunnel gets destroyed. Now each of these tunnels shares the same local IP address with our ethernet to the SWAN. Unfortunately, these "interface down" events confuse Quagga, and suddenly all of my punchin clients can't move bits to the internal network anymore.

There is a workaround, and that's to assign a different local IP address than the one that is directly connected to the SWAN for use with all of the client tunnels. It's not that painful, as I only lose one out of 256 possible client addresses (our engineering ones only have a /24 from which to allocate client addresses). Still, as an esteemed colleague said, "I hope that's not the *whole* solution."

It isn't, and I would like to ask the Quagga community (as I've already asked our local routing folks, Paul Jakma and Alan Maguire) to make sure that Quagga and its routing protocols play nicely with unnumbered interfaces. It'll allow me to plumb tunnels until I'm all out of address space! :)

This entry brought to you by the Technorati tags routing, Solaris, and OpenSolaris.

How OpenSolaris did its job during this telnet mess

I don't have a tag for general Security because dammit, I'm still a networking person who works on security!

Anyway, you've seen elsewhere about how Alan H. turned around the S10 fix as quickly as he could. I'm going to tell you how Alan already found this:

D 1.67  07/02/11 19:46:41 danmcd        90 89   00009/00010/04896
6523815 LARGE vulnerability in telnetd

when he went to file a bug that'd already been putback into Nevada/OpenSolaris.

The best place to see what happened is to visit the OpenSolaris discussions, especially this thread.

I was reading e-mail on a Sunday because of an operations problem I was having with one of our punchin IPsec remote access servers. (I'll discuss the problem, a routing one, in a followup entry later today.) I found the initial note and read the PDF file to which "skunsul" so graciously provided a link. MAN I was embarassed. After trying it on some lab machines and my laptop, I brought up the in.telnetd source (at the line number provided by Kingcope). My first approach was to verify the content of the $USER environment variable fed to in.telnetd. I compiled-and-ran the fix, which seemed to work. Great! Time to find some code reviewers.

My only regret about this was not putting the review on security-discuss@opensolaris.org or networking-discuss@opensolaris.org. I'll try better next time, especially for something that was announced on an opensolaris list initially. Anyway, two reviewers (OpenSolaris board member and well-known Sun Good Guy Casper Dik, and crypto framework expert Krishna Yenduri) suggested that login(1) is already getopt-compliant, and that I should just pass "--" between the rest of the arguments and the contents of $USER, no matter how *&^$-ed up it is. Because it was a Sunday, I didn't get rapid turnaround on e-mail replies. This is why the putback didn't happen until six hours after I'd read the note from skunsul. Krishna also recommended (in the spirit of open development) that I place the diffs on the very thread, and I did just that.

Anyone I know here who happened to have seen the initial note would've jumped on this in the same way - please don't think I did something others wouldn't do. My point is - this is the first security exploit reported to us via OpenSolaris, and I think the "Open" part of OpenSolaris helped out the code, as well as Sun's customers.

This entry brought to you by the Technorati tags Solaris and OpenSolaris.

Tunnel Reform Code Review starts now.

Hey everyone!

The IPsec Tunnel Reform project's code review is now underway. Take a look and see what it took to bring up IPsec Tunnel-Mode processing in a world where tunnels are not actions from a policy, but rather a first-class network interface (or at least after Clearview it will be).

Highlights for administrators include:

• Augmentiations to ipsecconf(1m) to specify a tunnel interface's policy, whether it's S9-style IP-in-IP transport mode, or RFC 2401-compliant Tunnel Mode.


• No changes to IKE configuration.


• You can configure tunnel security without ifconfig(1m) using just ipsecconf(1m). We put all IPsec policy in ipsecconf(1m) and let ifconfig manage interfaces (and route(1m) manage routing).


• Additions to ipseckey(1m) for manual tunnel-mode SA configuration, or monitoring of kernel interactions with Key Management.


• Better interoperability with everyone else's Tunnel Mode IPsec.

Highlights for OpenSolaris-hackers include:

• New per-tunnel policy structure: ipsec_tun_pol_t, which instantiates the existing policy-head per tunnel.


• Getting rid of IRE_DB_REQ messages for SA addition/updates. This improves SA-adding performance and reduces the complexity of the ESP and AH modules.


• New PF_KEY and PF_POLICY messages to reflect Tunnel Mode.


• Shifting of tunnel IPsec policy enforcment from the lower-instance of IP to "tun" itself. (NOTE: This will change again when we merge with Clearview.)

Share your comments on tref-discuss, and let us know what you think!

This entry brought to you by the Technorati tags IPsec, Solaris, and OpenSolaris.

Tunnel Reform now open for your perusal

IPsec in Solaris has one missing piece, and we're about to put it in place.

The IPsec Tunnel Reform project aims to give Solaris and OpenSolaris an RFC 2401-compliant tunnel-mode implementation.

There's a lot of changes in the source base, some of which aren't open sourced (IKE), but most of which are in existing OpenSolaris code. The project page has a webrev showing the changes thus far. We're trying to be more open in our development processes here in the Solaris group, and showing you Tunnel Reform before we've finished it, AND before we've started major test efforts, is Team IPsec's own way of contributing to this openness.

Think of the source snapshot as a "Code Preview" instead of a "Code Review". There's a newly-rewhacked design document there too, and we'd like you to look at it and discuss it on the OpenSolaris communities or the tref-discuss@opensolaris.org mailing list.

And once we're done with this, we can think about RFC 4301 (2401's replacement) and friends, more zones support, SMF-izing things, giving TX labelled SA support... :)

---

This entry brought to you by the Technorati tags IPsec, Solaris, and OpenSolaris.

ESP without authentication considered harmful

Hopefully you will read this and go "That's obvious". I'm writing this entry, however, for those who don't.

When IPsec was being specified over 10 years ago, attacks against cipher-block-chaining (CBC) encryption were understood. ESP has an authentication algorithm because AH had a vocal-enough opposition to merit having packet integrity in ESP also (there are also performance arguments for ESP-auth).

Now there actual attacks with actual results. Kenny Paterson and Arnold Yau have published a paper with attacks against no-authentication ESP Tunnel Mode. I believe some of the techniques can also be employed against Transport Mode as well, but again, only with no authentication present.

The simple solution, of course, is to employ your choice of ESP Authentication (encr_auth_algs in ipsecconf(1m) or ifconfig(1m)) or AH (auth_algs in ipsecconf(1m) or ifconfig(1m)) with your IPsec deployment. We warn users about such configurations with ifconfig(1m) today. There is an RFE to eliminate or make very difficult encryption-only configurations in Solaris. Maybe someone in the OpenSolaris community would like to take a stab at it?

Also in Solaris 10 01/06 (aka. Update 1)

Solaris 10 Update 1 has shipped. There are nifty things like new-boot in there, but here's a small, subtle, but perhaps blog-worthy entry.

RFC 3947 NAT-Traversal is now part of Solaris. The RFCs were published literally days after the Solaris 10 development gate closed its doors for anything but critical bug fixes. We had used the draft-09 version of NAT-T for Solaris 10, but now we have the RFC-compliant one, which should insure maximum interoperability.

Not a big deal, but IPsec-savvy folks out there ought to know.

It's time to enjoy my Christmas/New-Year's break. Enjoy your own end-of-year activities (whatever they are) and catch you in 2006.

Put IPsec to work in YOUR application

Hello coders!

Most people know that you can use ipsecconf(1m) to apply IPsec policy enforcement to an existing application. For example, if you wish to only allow inbound telnet traffic that's under IPsec protection, you'd put something like this into /etc/inet/ipsecinit.conf or other ipsecconf(1m) input:

# Inbound telnet traffic should be IPsec protected
{ lport 23 } ipsec { encr_algs any(128..) encr_auth_algs md5 sa shared}
             or ipsec { encr_algs any(128..) encr_auth_algs sha1 sa shared}

Combine that with appropriate IKE configuration or manual IPsec keys, and you can secure your telnet traffic against eavesdropping, connection hijacking, etc.

For existing services, using ipsecconf(1m) is the most expedient way to bring IPsec protection to bear on packets.

For new services, or services that are being modified anyway, consider using per-socket policy as an alternative. Some advantages to per-socket policy are:

• Per-socket policy is stored internally in network session state (the conn_t structure in OpenSolaris). Entries from ipsecconf(1m) are stored in the global Security Policy Database (SPD). No global SPD entries means lower latency for fresh flow creation, and less lock acquisition.


• Per-socket bypass means fewer bypass entries in global SPD. If I bypass remote-port 80 using ipsecconf(1m), I can, in theory, enter the system with a remote TCP packet with port=80. There's an RFE (6219908) to work around this, but per-socket is still quicker. I'd love a web proxy with the ability to set per-socket bypass.

The newly SMF-ized inetd(1m) would be a prime candidate for per-socket policy. See RFE 6226853, and this might be something someone in the OpenSolaris community would like to tackle!

Let's look at the ipsec_req_t structure that's been around since Solaris 8 in /usr/include/netinet/in.h:

/*
 * Different preferences that can be requested from IPSEC protocols.
 */
#define IP_SEC_OPT              0x22    /* Used to set IPSEC options */
#define IPSEC_PREF_NEVER        0x01
#define IPSEC_PREF_REQUIRED     0x02
#define IPSEC_PREF_UNIQUE       0x04
/*
 * This can be used with the setsockopt() call to set per socket security
 * options. When the application uses per-socket API, we will reflect
 * the request on both outbound and inbound packets.
 */

typedef struct ipsec_req {
        uint_t          ipsr_ah_req;            /* AH request */
        uint_t          ipsr_esp_req;           /* ESP request */
        uint_t          ipsr_self_encap_req;    /* Self-Encap request */
        uint8_t         ipsr_auth_alg;          /* Auth algs for AH */
        uint8_t         ipsr_esp_alg;           /* Encr algs for ESP */
        uint8_t         ipsr_esp_auth_alg;      /* Auth algs for ESP */
} ipsec_req_t;

The ipsec_req_t is a subset of what one can specify with ipsecconf(1m) in Solaris 9 or later, but it matched what one could do with Solaris 8's version. Algorithm values are derived from PF_KEY (see /usr/include/net/pfkeyv2.h for values), as below. One could also use getipsecalgbyname(3nsl). If I wish to set a socket to use ESP with AES and and MD5, I'd set it up as follows:

        int s;  /* Socket file descriptor... */
        ipsec_req_t ipsr;

        .....

        /* NOTE: Do this BEFORE calling connect() or accept() for TCP sockets. */
        ipsr.ipsr_ah_req = 0;
        ipsr.ipsr_esp_req = IPSEC_PREF_REQUIRED;
        ipsr.ipsr_self_encap_req = 0;
        ipsr.ipsr_auth_alg = 0;
        ipsr.ipsr_esp_alg = SADB_EALG_AES;
        ipsr.ipsr_esp_auth_alg = SADB_AALG_MD5HMAC;
        if (setsockopt(s, IPPROTO_IP, IP_SEC_OPT, &ipsr, sizeof (ipsr)) == -1) {
                perror("setsockopt");
                bail();  /* Ugggh, we failed. */
        }
        /* You now have per-socket policy set. */

Notice I mentioned setting the socket option BEFORE calling connect() or accept? This is because of a phenomenon we implement called connection latching. Basically, connection latching means that once an endpoint is connect()-ed, the IPsec policy (whether set per-socket or inherited from the state of the global SPD at the time) latches in place. We made this decision to avoid keeping policy-per-datagram state for things like TCP retransmits.

One thing per-socket policy does not address is the case of unconnected datagram services. In a perfect world, we could have IPsec policy information percolate all the way to the socket layer, where an application can make fully-informed per-datagram decisions on whether or not a particular packet was secured or not. It's a hard problem, requiring XNET sockets (to use sendmsg() and recvmsg() with ancillary data).

BTW, if you want to bypass whatever global entries are in the SPD, you can zero out the structure, and set all three (ah, esp, self_encap) action indicators to IPSEC_PREF_NEVER. You need to be privileged (root or "sys_net_config") to use per-socket bypass, however.

So modulo the keying problem (setting up IKE or having both ends agree on IPsec manual keys), you can put IPsec to work right in your application. In fact, if you use IKE, you can let IKE sort out permissions and access control (by using PKI-issued certificates, self-signed certificates, or preshared keys) and have policy merely determine the details of the protection required.

---

EDITED: This entry brought to you by the Technorati tags IPsec, Solaris, and OpenSolaris.