Before this build, IPsec NAT-Traversal was performed by pushing a STREAMS module on top of an open UDP socket. This module (nattymod) would either strip UDP headers out of ESP-in-UDP packets, or strip the "0-SPI" marker (four bytes of zeroes) before passing the datagram up to the application.
This method worked, but it had some flaws, including the implicit setting of certain socket options (UDP_INCLHDR) that would then potentially be blocked from applications that actually required them. Also, nattymod did not perform the insertion of the 0-SPI automatically, the application was stuck doing that on its own. And while FireEngine merged TCP in to IP for S10, we needed to wait for one of the earlier builds of OpenSolaris to get the UDP equivalent.
With the new NAT-Traversal scheme, a key management application (like our closed-source in.iked(1M)) that wishes to aid in NAT-Traversal simply sets a new socket option: UDP_NAT_T_ENDPOINT. If this option is set, the following things happen:
- On inbound packets, the first four bytes after the UDP header are inspected.
- If there are less than four byte, the packet is dropped, and assumed to be a NAT-T keepalive.
- If the four-byte are all zeros (i.e. the 0-SPI), they are stripped and regular UDP processing occurs.
- Otherwise, the UDP header is stripped and the packet is shuffled off the IPsec's ESP for processing.
- On outbound packets
- The 0-SPI is inserted between the UDP header and the user-generated data.
- ESP will send ESP-in-UDP by itself depending on the Security Association's properties.
This will help anyone who wants to port their open-source IKE or other key management application to Solaris deal with the possibility of NAT boxes.
And on a related note, this will be mentioned during Nicolas Droux's OpenSolaris Networking for Developers talk next week at Sun Tech Days in Boston. I'll be there too, talking about S10 and OpenSolaris security features, as well as being in the audience for Nicolas's talk.