AH achieves authenticity by applying a keyed one-way hash function to the packet to create a hash or message digest. The hash is combined with the text and is transmitted in plaintext. The receiver detects changes in any part of the packet that occur during transit by performing the same one-way hash function on the received packet and comparing the result to the value of the message digest that the sender supplied. Authenticity is assured because the one-way hash also employs a shared secret key between the two systems.

The AH function is applied to the entire packet, except for any IP header fields that normally change in transit. Fields that normally change during transit are called mutable fields. For example, the Time to Live (TTL) field is considered mutable because routers modify this field.

The AH process occurs in this order:

1. The IP header and data payload are hashed using the shared secret key.

2. The hash builds a new AH header, which is inserted into the original packet.

3. The new packet is transmitted to the IPsec peer router.

4. The peer router hashes the IP header and data payload using the shared secret key, extracts the transmitted hash from the AH header, and compares the two hashes.

The hashes must match exactly. If one bit is changed in the transmitted packet, the hash output on the received packet changes and the AH header will not match.

AH supports MD5 and SHA algorithms.

Advantage:

• AH provide the IP header integrity check (ESP does not).

Disadvantage:

• AH does not provide encryption, Anti-replay machanism.
• AH may not work if the environment uses NAT,because of integrity check.

ESP

ESP provides confidentiality by encrypting the payload. It supports a variety of symmetric encryption algorithms. If ESP is selected as the IPsec protocol, an encryption algorithm must also be selected. It does not have a port number, but has a set of parameters.

The default algorithm for IPsec is 56-bit DES. Cisco products also support the use of 3DES, AES, and SEAL for stronger encryption.

ESP can also provide integrity and authentication. First, the payload is encrypted. Next, the encrypted payload is sent through a hash algorithm, such as MD5 or SHA. The hash provides authentication and data integrity for the data payload.

Optionally, ESP can also enforce anti-replay protection. Anti-replay protection verifies that each packet is unique and is not duplicated. This protection ensures that a hacker cannot intercept packets and insert changed packets into the data stream. Anti-replay works by keeping track of packet sequence numbers and using a sliding window on the destination end.

When a connection is established between a source and destination, their counters are initialized at zero. Each time a packet is sent, a sequence number is appended to the packet by the source. The destination uses the sliding window to determine which sequence numbers are expected. The destination verifies that the sequence number of the packet is not duplicated and is received in the correct order.

For example, if the sliding window on the destination is set to one, the destination is expecting to receive the packet with the sequence number one. After it is received, the sliding window moves to two. When detection of a replayed packet occurs, such as the destination receiving a second packet with the sequence number of one, an error message is sent, the replayed packet is discarded, and the event is logged.

Anti-replay is typically used in ESP, but it is also supported in AH.

IKE uses ISAKMP for phase 1 and phase 2 of key negotiation. In phase 1 and phase 2, two sessions are created, phase 1 is the management session, phase 2 is the security association.

Phase 1 (Bidirectional)

Phase 1 negotiates a security association (a key) between two IKE peers. The key negotiated in phase 1 enables IKE peers to communicate securely in phase 2. During phase 2 negotiation, IKE establishes keys (security associations) for other applications, such as IPsec.

In Phase 1, two IPsec peers perform the initial negotiation of SAs. The basic purpose of Phase 1 is to negotiate ISAKMP policy, authenticate the peers, and set up a secure tunnel between the peers. This tunnel will then be used in Phase 2 to negotiate the IPsec policy.

Note: The phrases IKE policy and ISAKMP policy are equivalent. The phrase ISAKMP policy is used in this course to better match the commands (crypto isakmp policy, show isakmp policy, etc.) as well as to clarify that the ISAKMP policy applies to the IKE Phase 1 tunnel.

Phase 1 can be implemented in main mode or aggressive mode. When main mode is used, the identities of the two IKE peers are hidden. Aggressive mode takes less time than main mode to negotiate keys between peers. However, since the authentication hash is sent unencrypted before the tunnel is established, aggressive mode is vulnerable to brute-force attacks.

Note: In Cisco IOS software, the default action for IKE authentication is to initiate main mode. However, Cisco IOS software will respond in aggressive mode to an IKE peer that initiates aggressive mode.

Phase 2(unidirection)