Confidentiality is provided by an encryption format known as encapsulating security payload,

IPsec provides two security protocols for protecting data:

• Authentication Header [AH]

• Encapsulating Security Payload [ESP]

AH provides data integrity by using an authentication algorithm. It does not encrypt the packet.

ESP typically protects the packet with an encryption algorithm and provides data integrity with an authentication algorithm. Some encryption algorithms provide both encryption and authentication, such as AES GCM.

The AH protocol cannot be used with network address translation [NAT].

Authentication Header

The authentication header provides data authentication, strong integrity, and replay protection to IP packets. AH protects the greater part of the IP packet. As the following illustration shows, AH is inserted between the IP header and the transport header.

The transport header can be TCP, UDP, SCTP, or ICMP. If a tunnel is being used, the transport header can be another IP header.

Encapsulating Security Payload

The encapsulating security payload [ESP] protocol provides confidentiality over what the ESP encapsulates. ESP also provides the services that AH provides. However, ESP does not protect the outer IP header. ESP provides authentication services to ensure the integrity of the protected packet. Because ESP uses encryption-enabling technology, a system that provides ESP can be subject to import and export control laws.

The ESP header and trailer encapsulate the IP payload. When encryption is used with ESP, it is applied only over the IP payload data, as shown in the following illustration.

In a TCP packet, the ESP header is authenticated and it encapsulates the TCP header and its data. If the packet is an IP-in-IP packet, ESP protects the inner IP packet. Per-socket policy allows self-encapsulation, so ESP can encapsulate IP options when necessary.

Self-encapsulation can be used by writing a program that uses the setsockopt[] system call. If self-encapsulation is set, a copy of the IP header is made to construct an IP-in-IP packet. For example, when self-encapsulation is not set on a TCP socket, the packet is sent in the following format:

[ IP[a -> b] options + TCP + data ]

When self-encapsulation is set on that TCP socket, the packet is sent in the following format:

[ IP[a -> b] + ESP [ IP[a -> b] options + TCP + data ] ]

For further discussion, see Transport and Tunnel Modes in IPsec.

Security Considerations When Using AH and ESP

The following table compares the protections that are provided by AH and ESP.

Table 9  Protections Provided by AH and ESP in IPsec

Protocol Packet Coverage Protection Against Attacks AH Protects packet from the IP header to the end of the transport data Provides strong integrity, data authentication: Ensures that the receiver receives exactly what the sender sent Is susceptible to replay attacks when an AH does not enable replay protection Replay, cut-and-paste ESP Protects packet from the ESP header to the end of the transport data With encryption option, encrypts the IP payload. Ensures confidentiality Eavesdropping With authentication option, provides the same payload protection as AH Replay, cut-and-paste With both options, provides strong integrity, data authentication, and confidentiality Replay, cut-and-paste, eavesdropping

Authentication and Encryption Algorithms in IPsec

IPsec security uses two types of algorithms, authentication and encryption. The AH protocol uses authentication algorithms. The ESP protocol can use encryption as well as authentication algorithms. You can obtain a list of the algorithms on your system and their properties by using the ipsecalgs command. For more information, see the ipsecalgs[1M] man page. You can also use the functions that are described in the getipsecalgbyname[3NSL] man page to retrieve the properties of algorithms.

IPsec uses the Cryptographic Framework to perform encryption and authentication. The Cryptographic Framework enables IPsec to take advantage of hardware acceleration where the hardware supports it.

For more information, see the following:

• Chapter 1, Cryptography in Oracle Solaris in Managing Encryption and Certificates in Oracle Solaris 11.3

• Chapter 8, Introduction to the Oracle Solaris Cryptographic Framework in Developer’s Guide to Oracle Solaris 11.3 Security

How confidentiality is provided by an ESP?

What are the uses of encapsulating security payload?

Which of the following services are provided by encapsulating security payload?

Which protocol security IPSec protocol provides confidentiality by encrypting data?