HOW IPSEC WORKS Original Main

IPsec (Internet Protocol Security)

IPSec Tunnel mode:

IPSec Tunnel mode: In IPSec Tunnel mode, the original IP packet (IP header and the Data payload) is encapsulated within another packet. In IPSec tunnel mode the original IP Datagram from is encapsulated with an AH (provides no confidentiality by encryption) or ESP (provides encryption) header and an additional IP header. The IP addresses of the newly added outer IP header are that of the VPN Gateways. The traffic between the two VPN Gateways appears to be from the two gateways (in a new IP datagram), with the original IP datagram is encrypted (in case of ESP) inside IPSec packet.

IPSec Tunnel mode is most widely used to create site-to-site IPSec VPN.

IPSec Transport mode: In IPSec Transport mode, only the Data Payload of the IP datagram is secured by IPSec. IP Header is the original IP Header and IPSec inserts its header between the IP header and the upper level headers.

IPSec Transport mode can be used when encrypting traffic between two hosts or between a host and a VPN gateway.

Note: The data is Encrypted and Authenticated if Encapsulating Security Payload (ESP) is used, or just authenticated if Authentication Header (AH) used.

ON IPSEC Tunnel Mode it uses two mechanism

·         AH (Authentication Header)

·         ESP (Encapsulating Security Payload)

Authentication Header (AH)—A security protocol for authenticating the source of an IP packet and verifying the integrity of its content

  Encapsulating Security Payload (ESP)—A security protocol for encrypting the entire IP packet (and authenticating its content)

The AH protocol provides a mechanism for authentication only. AH provides data integrity, data origin authentication, and an optional replay protection service. Data integrity is ensured by using a message digest that is generated by an algorithm such as HMAC-MD5 or HMAC-SHA. Data origin authentication is ensured by using a shared secret key to create the message digest. Replay protection is provided by using a sequence number field with the AH header. AH authenticates IP headers and their payloads, with the exception of certain header fields that can be legitimately changed in transit, such as the Time To Live (TTL) field.

The ESP protocol provides data confidentiality (encryption) and authentication (data integrity, data origin authentication, and replay protection). ESP can be used with confidentiality only, authentication only, or both confidentiality and authentication. When ESP provides authentication functions, it uses the same algorithms as AH, but the coverage is different. AH-style authentication authenticates the entire IP packet, including the outer IP header, while the ESP authentication mechanism authenticates only the IP datagram portion of the IP packet.

·         Internet Key Exchange (IKE)
IKE establishes the keys used by AH and ESP to authenticate the communication end points, verify integrity and by ESP to encrypt and decrypt data.

The entire process of IPsec consists of five steps:

·         Initiation: something has to trigger the creation of our tunnels. For example when you configure IPsec on a router, you use an access-list to tell the router what data to protect. When the router receives something that matches the access-list, it will start the IKE process. It’s also possible to manually initiate the tunnel.

·         IKE phase 1: we negotiate a security association to build the IKE phase 1 tunnel (ISAKMP tunnel).

·         IKE phase 2: within the IKE phase 1 tunnel, we build the IKE phase 2 tunnel (IPsec tunnel).

·         Data transfer: we protect user data by sending it through the IKE phase 2 tunnel.

·         Termination: when there is no user data to protect then the IPsec tunnel will be terminated after awhile.

Now you have an idea of the basics of IPsec, let’s take a closer look at each of the different components.

IPsec (Internet Protocol Security) is a framework that helps us to protect IP traffic on the network layer. Why? because the IP protocol itself doesn’t have any security features at all. IPsec can protect our traffic with the following features:

·         Confidentiality: by encrypting our data, nobody except the sender and receiver will be able to read our data.

·         Integrity: we want to make sure that nobody changes the data in our packets. By calculating a hash value, the sender and receiver will be able to check if changes have been made to the packet.

·         Authentication: the sender and receiver will authenticate each other to make sure that we are really talking with the device we intend to.

·         Anti-replay: even if a packet is encrypted and authenticated, an attacker could try to capture these packets and send them again. By using sequence numbers, IPsec will not transmit any duplicate packets.

Before we can protect any IP packets, we need two IPsec peers that build the IPsec tunnel.

 IKE Phase One

The basic purpose of IKE phase one is to authenticate the IPSec peers and to set up a secure channel between the peers to enable IKE exchanges. IKE phase one performs the following functions:

The IKE negotiation parameters determine:

·         Which encryption algorithms may be applied for converting messages into a form that only the intended recipient can read

·         Which authentication hash may be used for creating a keyed hash from a preshared or private key

·         Which Diffie-Hellman group (DH Group) will be used to generate a secret session key

This phase requires the following:

·         An encryption algorithm- This determines the depth and type of encryption.  (Keep in mind the deeper the encryption the slower the connection)

·         AES (advanced encryption Standard).  The key size used for an AES cipher specifies the number of repetitions of transformation rounds that convert the input, called the plaintext, into the final output, called the ciphertext. The number of cycles of repetition are as follows:

·         10 cycles of repetition for 128-bit keys.

·         12 cycles of repetition for 192-bit keys.

·         14 cycles of repetition for 256-bit keys.

·         3DES – (Triple DES) they tripled the depth of the encryption of DES

·         DES – A 56 bit algorithm that is susceptible to brute force attacks

·         AES256 – Considered the top encryption cipher has the most combinations of possible keys

·         Blowfish – A symmetric-key block cipher. This a good encryption cipher that encrypts in blocks

·         Cast – another symmetric-key block cipher

·         A Authentication hashing algorithm

·         SHA– (secure Hash Algorithm) this is a 160-bit hash

·         MD5 – (message-digest algorithm) a 128-bit hash

·         Diffie-Hellman key (DH key ) also known as MODP Group on other Site 2 Site VPNs

·         Group1 – 768-bit

·         Group2 – 1024-bit

·         Group5 – 1536-bit

·         Lifetime – Rekeying time in milliseconds.

IKE phase one occurs in two modes:

·         Main mode

·         Aggressive mode

Main Mode

Main Mode:

IKE Phase 1 operating in main mode works with both parties exchanging a total of 6 packets, that’s right 6 packets is all it takes to complete phase 1.

1.    The first packet is sent from the initiator of the IPSec tunnel to its remote endpoint, this packet contains the ISAKMP policy

2.    The second packet is sent from the remote endpoint back to the initiator, this packet will be the exact same information matching the ISAKMP policy sent by the initiator.

3.    The third packet is sent from the initiator to the remote endpoint, this packet contains the Key Exchange payload and the Nonce payload, the purpose of this packet is generate the information for the DH secret key

4.    This fourth packet as you would expect comes from the remote endpoint back to initiator and contains the remote endpoints Key Exchange and Nonce payload.

5.    The fifth packet is from the initiator back to the remote endpoint with identity and hash payloads, the identity payload has the device’s IP Address in, and the hash payload is a combination of keys (including a PSK, if PSK authentication is used)

6.    The sixth packet from the remote endpoint to the initiator contains the corresponding hash payloads to verify the exchange.

Aggressive Mode:

IKE Phase 1 operating in aggressive mode only exchanges 3 packets compared to the 6 packets used in main mode. One downside in aggressive is the fact it not as secure as main mode.

• The first packet from the initiator contains enough information for the remote endpoint to generate its DH secret, so this one packet is equivalent to the first four packets in main mode.
• The second packet from the remote endpoint back to the initiator contains its DH secret
• The third packet from the initiator includes identity and hash payloads. After the remote endpoint receives this packet it simply calculates its hash payload and verifies it matches, if it matches then phase one is established.

IKE Phase 2

The IKE phase 2 tunnel (IPsec tunnel) will be actually used to protect user data. There is only one mode to build the IKE phase 2 tunnel which is called quick mode.

Just like in IKE phase 1, our peers will negotiate about a number of items:

·         IPsec Protocol: do we use AH or ESP?

·         Encapsulation Mode: transport or tunnel mode?

·         Encryption: what encryption algorithm do we use? DES, 3DES or AES?

NULL — Do not use an encryption algorithm.
DES — Digital Encryption Standard, a 64-bit block algorithm that uses a 56-bit key.
3DES — Triple-DES; plain text is encrypted three times by three keys.
AES128 — A 128-bit block algorithm that uses a 128-bit key.
AES192 — A 128-bit block algorithm that uses a 192-bit key.
AES256 — A 128-bit block algorithm that uses a 256-bit key.

·         Authentication: what authentication algorithm do we use? MD5 or SHA?

NULL — Do not use a message digest.
MD5 — Message Digest 5.
SHA1 — Secure Hash Algorithm 1 - a 160-bit message digest.

·         Lifetime: how long is the IKE phase 2 tunnel valid? When the tunnel is about to expire, we will refresh the keying material.

·         (Optional) DH exchange: used for PFS (Perfect Forward Secrecy).

·         Enable replay detection     Optionally enable or disable replay detection. Replay attacks occur when an unauthorized party intercepts a series of IPsec packets and replays them back into the tunnel.

This negotiation happens within the protection of our IKE phase 1 tunnel 

Once IKE phase 2 has completed, we are finally ready to protect some user data. Let’s see how this is done…

Step 4: IPSec Encrypted Tunnel

After IKE phase two is complete and quick mode has established IPSec SAs, information is exchanged by an IPSec tunnel. Packets are encrypted and decrypted using the encryption specified in the IPSec SA. This IPSec encrypted tunnel can be seen in Figure 1-18.
Figure 1-18 IPSec Encrypted Tunnel

Step 5: Tunnel Termination

IPSec SAs terminate through deletion or by timing out. An SA can time out when a specified number of seconds have elapsed or when a specified number of bytes have passed through the tunnel. When the SAs terminate, the keys are also discarded. When subsequent IPSec SAs are needed for a flow, IKE performs a new phase two and, if necessary, a new phase one negotiation. A successful negotiation results in new SAs and new keys. New SAs can be established before the existing SAs expire so that a given flow can continue uninterrupted. 

Data transfer: Incoming and outgoing network traffic is encapsulated according to the bundle of algorithms and parameters provided by their respective negotiated IPsec SA to provide confidentiality and authenticity (ESP protocol) or authenticity only (AH protocol).

Tunnel termination: A tunnel is closed when its IPsec SAs terminate through deletion or by timing out. An IPsec SA can time out when a specified number of seconds have elapsed or when a specified number of bytes have passed through the tunnel.