Test VPN Connectivity
Test VPN Connectivity
|
Initiate IKE phase 1 by either pinging a host across the tunnel or using the following CLI command:test vpn ike-sa gateway <gateway_name>
|
enter the following command to test if IKE phase 1 is set up:show vpn ike-sa gateway <gateway_name>In the output, check if the Security Association displays. If it does not, review the system log messages to interpret the reason for failure.
|
Initiate IKE phase 2 by either pinging a host from across the tunnel or using the following CLI command:test vpn ipsec-sa tunnel <tunnel_name>
|
enter the following command to test if IKE phase 1 is set up:show vpn ipsec-sa tunnel <tunnel_name>In the output, check if the Security Association displays. If it does not, review the system log messages to interpret the reason for failure.
|
To view the VPN traffic flow information, use the following command:show vpn flowtotal tunnels configured: 1filter - type IPSec, state anytotal IPSec tunnel configured: 1total IPSec tunnel shown: 1name id state local-ip peer-ip tunnel-i/f-------------------------------------------------------------------------vpn-to-siteB 5 active 100.1.1.1 200.1.1.1 tunnel.41
Starting from PAN-OS 8.0, we can enable IPSec VPN specific debugs per peer:
Pre PAN-OS 8.0
admin@PA-VM-7.1> debug ike
> global global > pcap pcap > socket socket > stat show IKE daemon statistics
Post PAN-OS 8.0
admin@PA-VM-8.0> debug ike
> gateway debug IKE gateway > global global > pcap pcap > socket socket > stat show IKE daemon statistics > tunnel debug IPSec tunnel
Using the " gateway " or " tunnel " keyword you can enable the logs per VPN gateway or IPSEC tunnel
Example:
admin@PA-VM-8.0> debug ike gateway IKE-GW-HQ
> clear clear IPSec tunnel statistics > off Turn off IPSec tunnel debug logging > on Turn on IPSec tunnel debug logging > stats show IPSec tunnel statistics admin@PA-VM-8.0> debug ike gateway IPSEC-HQ > clear clear IPSec tunnel statistics > off Turn off IPSec tunnel debug logging > on Turn on IPSec tunnel debug logging > stats show IPSec tunnel statistics
Note:
- debug filters can be enabled for up to 5 IKE Gateways and/or IPSEC tunnels
How to Troubleshoot IPSec VPN connectivity issues
Labels:
· ,
· VPN
This document is intended to help troubleshoot IPSec VPN connectivity issues. It is divided into two parts, one for each Phase of an IPSec VPN.
Phase 1
· To rule out ISP-related issues, try pinging the peer IP from the PA external interface. Ensure that pings are enabled on the peer's external interface.
· If pings have been blocked per security requirements, see if the other peer is responding to the main/aggressive mode messages, or the DPDs. Check for the responses of the "Are you there?" messages from the peer in the system logs under the Monitor tab or under ikemgr logs.
· Check that the IKE identity is configured correctly.
· Check that the policy is in place to permit IKE and IPSec applications. Usually this policy is not required if there is no clean-up rule configured on the box. If a clean-up rule is configured, the policy is configured usually from the external zone to the external zone.
· Check that proposals are correct. If incorrect, logs about the mismatch can be found under the system logs, or by using the following CLI command:
> less mp-log ikemgr.log
· Check that preshared key is correct. If incorrect, logs about the mismatch can be found under the system logs, or by using the following CLI command:
> less mp-log ikemgr.log
· Take packet captures to analyze the traffic. Use filters to narrow the scope of the captured traffic.
· Useful CLI commands:
> show vpn ike-sa gateway <name>
> test vpn ike-sa gateway <name> > debug ike stat
Advanced CLI commands:
· For detailed logging, turn on the logging level to debug:
> debug ike global on debug
> less mp-log ikemgr.log
· To view the main/aggressive and quick mode negotiations, it is possible to turn on pcaps for capturing these negotiations. Messages 5 and 6 onwards in the main mode and all the packets in the quick mode have their data payload encrypted:
> debug ike pcap on
> view-pcap no-dns-lookup yes no-port-lookup yes debug-pcap ikemgr.pcap
· Turn off debugs
> debug ike pcap off
Configuring packet filter and captures restricts pcaps only to the one worked on, debug IKE pcap on shows pcaps for all VPN traffic.
To check if NAT-T is enabled, packets will be on port 4500 instead of 500 from the 5th and 6th messages of main mode.
Check if vendor id of the peer is supported on the Palo Alto Networks device and vice-versa.
Phase 2
· Check if the firewalls are negotiating the tunnels, and ensure that 2 unidirectional SPIs exist:
> show vpn ipsec-sa
> show vpn ipsec-sa tunnel <tunnel.name>
· Check if proposals are correct. If incorrect, logs about the mismatch can be found under the system logs under the monitor tab, or by using the following command:
> less mp-log ikemgr.log
· Check if pfs is enabled on both ends. If incorrect, logs about the mismatch can be found under the system logs under the monitor tab, or by using the command:
> less mp-log ikemgr.log
· Check the proxy-id configuration. This is usually not required when the tunnel is between two Palo Alto Networks firewalls, but when the peer is from another vendor, IDs usually need to be configured.
A mismatch would be indicated under the system logs, or by using the command:
> less mp-log ikemgr.log
· Useful CLI commands:
> show vpn flow name <tunnel.id/tunnel.name>
> show vpn flow name <tunnel.id/tunnel.name> | match bytes
· Check if encapsulation and decapsulation bytes are increasing. If the firewall is passing traffic, then both values should be increasing.
> show vpn flow name <tunnel.id/tunnel.name> | match bytes
If encapsulation bytes are increasing and decapsulation is constant, then the firewall is sending but not receiving packets.
· Check to see if a policy is dropping the traffic, or if a port translating device in front of PAN that might be dropping the ESP packets.
> show vpn flow name <tunnel.id/tunnel.name> | match bytes
If decapsulation bytes are increasing and encapsulation is constant, then the firewall is receiving but not transmitting packets.
· Check to see if a policy is dropping the traffic:
> test routing fib-lookup virtual-router default ip <destination IP>
-------------------------------------------------- runtime route lookup -------------------------------------------------- virtual-router: default destination: 10.5.1.1 result: interface tunnel.1 > show routing route > test vpn ipsec-sa tunnel <name>
Advanced CLI commands:
> debug ike global on debug > less mp-log ikemgr.log > debug ike pcap on > view-pcap no-dns-lookup yes no-port-lookup yes debug-pcap ikemgr.pcap > debug ike pcap off
If tunnels are up but traffic is not passing through the tunnel:
· Check security policy and routing.
· Check for any devices upstream that perform port-and-address-translations. Because ESP is a layer 3 protocol, ESP packets do not have port numbers. When such devices receive ESP packets, there is a high possibility they may silently drop them, because they do not see the port numbers to translate.
· Apply debug packet filters, captures or logs, if necessary, to isolate the issue where the traffic is getting dropped.
|
The options to configure policy-based IPsec VPN are unavailable.
Go to System > Feature Select. Select Show More and turn on Policy-based IPsec VPN.
If your VPN fails to connect, check the following:
- Ensure that the pre-shared keys match exactly (see The pre-shared key does not match (PSK mismatch error) below).
- Ensure that both ends use the same P1 and P2 proposal settings (see The SA proposals do not match (SA proposal mismatch) below).
- Ensure that you have allowed inbound and outbound traffic for all necessary network services, especially if services such as DNS or DHCP are having problems.
- Check that a static route has been configured properly to allow routing of VPN traffic.
- Ensure that your FortiGate unit is in NAT/Route mode, rather than Transparent.
- Check your NAT settings, enabling NAT traversal in the Phase 1 configuration while disabling NAT in the security policy. You might need to pin the PAT/NAT sessiontable, or use some of kind of NAT-T keepalive to avoid the expiration of your PAT/NAT translation.
- Ensure that both ends of the VPN tunnel are using Main mode, unless multiple dial-up tunnels are being used.
- If you have multiple dial-up IPsec VPNs, ensure that the Peer ID is configured properly on the
- FortiGate and that clients have specified the correct Local ID.
- If you are using FortiClient, ensure that your version is compatible with the FortiGate firmware by reading the FortiOS Release Notes.
- If you are using Perfect Forward Secrecy (PFS), ensure that it is used on both peers. You can use the
diagnose vpn tunnel listcommand to troubleshoot this. - Ensure that the Quick Mode selectors are correctly configured. If part of the setup currently uses firewall addresses or address groups, try changing it to either specify the IP addresses or use an expanded address range. This is especially useful if the remote endpoint is not a FortiGate device.
- If XAUTH is enabled, ensure that the settings are the same for both ends, and that the FortiGate unit is set to Enable as Server.
- Check IPsec VPN Maximum Transmission Unit (MTU) size. A 1500 byte MTU is going to exceed the overhead of the ESP-header, including the additional ip_header,etc. You can use the
diagnose vpn tunnel listcommand to troubleshoot this. - If your FortiGate unit is behind a NAT device, such as a router, configure port forwarding for UDP ports 500 and 4500.
- Remove any Phase 1 or Phase 2 configurations that are not in use. If a duplicate instance of the VPN tunnel appears on the IPsec Monitor, reboot your FortiGate unit to try and clear the entry.
We use Juniper VPN hardware at our side here at Nexcess and have successfully created tunnels to just about everything including Cisco ASA and PIX, Checkpoint, Sonicwall, Netgear, and Zyxel to name a few. From a troubleshooting standpoint, it doesn’t really matter what device you are using at each end of the tunnel as long as there are no known interoperability issues between the two. While setting up these tunnels, issues have come up and as a general guideline there are basically three things that you should look for when a tunnel fails to work as expected:
- Phase 1 negotiations fail
- Phase 2 negotiations fail
- The tunnel is established, but not passing traffic.
The first step to take when you have a tunnel that is not working based on any three of the above conditions is to verify that both ends of the tunnel have the exact same information. Often the remote tunnel may be configured by a different individual, so accurate communication of all necessary information between both parties is key:
- Make sure your encryption settings, hashes, lifetimes, etc are the exact same for both ends of the tunnel for both phase1 and phase2 negotiations. Verify your gateways are the same. Make sure your internal subnets are different and have the correct information including masks.
- Verify the pre-shared key on each end is the same. Watch out for additional whitespace at the beginning or the end of the key that was inadvertently pasted in.
- Also verify that each end is using the same type of tunnel. If one side is configuring a route-based tunnel while the other is a policy-based, you will run into issues.
Enabling debug logging for the tunnel you are attempting to establish is the next step to take once you have confirmed that the information at both sides of the tunnel is presumed correct. The steps to perform this will vary from device to device, but make sure your logging is verbose enough to display the actual handshaking as the connection attempts to establish.
If phase1 negotiations are failing for you, check that the encryption algorithm, authentication method, hash algorithm, and lifetime are the exact same on both sides for the phase 1 proposal. Once verified, then look at the gateway configuration. The initiator mode should be the same on both sides along with the remote gateway ip. If you are seeing nothing in the logs, there is a good chance the remote gateway ip is incorrect or a firewall is blocking connectivity somewhere between the two.
If phase 1 negotiations have established and you are failing on phase 2, there are a few different items to check. Just as with the phase 1 settings, verify the phase 2 proposal encryption algorithm, authentication algorithm or hash, and lifetime are the same on both sides. If using perfect forward secrecy, now would be a good time to check it as well. The tunnel type should also be checked, verify a policy or route based tunnel is the same on each end. In the event of a route based tunnel, the proxy-id settings should be correct at each side. Most policy based tunnels will auto generate the proxy-id settings automatically, so the proxy-id is not needed in this case. Also check interface binding. Make sure the tunnel is bound to the public facing interface (or whichever interface the tunnel should be established over).
Now If you have both phase 1 and phase 2 successful negotiations and your tunnel is reported as up but you cannot pass traffic, you need to focus on firewall policies and routing. Routing is the first and easiest thing to check. If you can console you VPN appliance, simply send a ping to the remote private gateway and verify connectivity over the tunnel. If the ping fails, check your policies in both a route or policy based tunnel configuration and make sure that ICMP is set to pass and both source and destination networks are set correctly. If the ping succeeds, next check your layer 3 routing table on your primary gateway ( if it is not the VPN appliance itself). The destination network should be using the VPN appliance private ip for the gateway. Alternatively, you could add local static routes on each machine and device that would be traversing the tunnel. This of course would become tedious if operating more than a handful of machines. Finally verify all policies are in place for both directions of the tunnel for proper network security. Allow only the protocols that are needed to pass each way over the tunnel. Taking this a step further, you can lock down the devices by source and destination ip address policies if necessary.
All of the above steps should solve any tunnel issues you are experiencing. If you are still unable to establish the tunnel, try a different set of encryption settings. There may be some strange incompatibilities with one or more of the devices. Also check the release notes for the latest firmware version of your VPN appliance (since you have already upgraded any firmware to the latest version). It may offer some hints of what your continuing issue may be. Finally, look up the knowledgebase for your specific appliance as it may offer further suggestions for interoperability if each device is different.
Issue
A site-to-site IPSec VPN between a Palo Alto Networks firewall and a firewall from a different vendor is configured.
Phase 1 succeeds, but Phase 2 negotiation fails.
A look at the ikemgr.log with the CLI command:
> tail follow yes mp-log ikemgr.log
shows the following errors:
( description contains 'IKE protocol notification message received: INVALID-ID-INFORMATION (18).' )
and
IKE phase-2 negotiation is failed as initiator, quick mode. Failed SA: 216.204.241.93[500]-216.203.80.108[500] message id:0x43D098BB. Due to negotiation timeout
Cause
The most common phase-2 failure is due to Proxy ID mismatch.
Resolution
To resolve Proxy ID mismatch, please try the following:
- Check the Proxy ID settings on the Palo Alto Networks firewall and the firewall on the other side.
Note: Proxy ID for other firewall vendors may be referred to as the Access List or Access Control List (ACL). - Also, check the IPSec crypto to ensure that the proposals match on both side
Interpret VPN Error Messages
The following table lists some of the common VPN error messages that are logged in the system log.
If error is this:
|
Try this:
|
|---|---|
IKE phase-1 negotiation is failed as initiator, main mode. Failed SA: x.x.x.x[500]-y.y.y.y[500] cookie:84222f276c2fa2e9:0000000000000000 due to timeout.
or
IKE phase 1 negotiation is failed. Couldn’t find configuration for IKE phase-1 request for peer IP x.x.x.x[1929] |
|
Received unencrypted notify payload (no proposal chosen) from IP x.x.x.x[500] to y.y.y.y[500], ignored...
or
IKE phase-1 negotiation is failed. Unable to process peer’s SA payload. |
Check the IKE Crypto profile configuration to verify that the proposals on both sides have a common encryption, authentication, and DH Group proposal.
|
pfs group mismatched:my: 2peer: 0
or
IKE phase-2 negotiation failed when processing SA payload. No suitable proposal found in peer’s SA payload.
|
Check the IPSec Crypto profile configuration to verify that:
|
IKE phase-2 negotiation failed when processing Proxy ID. Received local id x.x.x.x/x type IPv4 address protocol 0 port 0, received remote id y.y.y.y/y type IPv4 address protocol 0 port 0.
|
The VPN peer on one end is using policy-based VPN. You must configure a Proxy ID on the Palo Alto Networks firewall. See Create a Proxy ID to identify the VPN peers. .
|
The confusing part about the IPSec Tunnel status window is that there are actually 3 areas that show the current status. I have detailed the "status" below:
- Phase 1 - IKE status - Green indicates a valid IKE phase-1 SA or IKEv2 IKE SA. Red indicates that IKE phase-1 SA is not available or has expired.
- Phase 2 - IPSec status - Green indicates an IPSec phase-2 security association (SA) tunnel. Red indicates that IPSec phase-2 SA is not available or has expired.
- IPSec Tunnel Interface status - Green indicates that the tunnel interface is up (because tunnel monitor is disabled or because tunnel monitor status is UP and the monitoring IP address is reachable). Red indicates that the tunnel interface is down because the tunnel monitor is enabled and the remote tunnel monitoring IP address is unreachable.
I have personally seen the Phase 2 IPSec showing Green, with Phase 1 IKE showing a Red status, even though the tunnel is showing green, because it is still active. The next time that IKE is to be renegotiated, it may or may not have an issue, but it's good to be aware of.
To get more detailed information on what is going on when the IPSec Tunnel Interface is showing Red, you will need to go into your logs and look for any errors that may help indicate what the problem is. You also can click the "Tunnel Info" and "IKE Info" text to the right of the "bubble" status to get more info. (A window will appear showing the IKE or IPSec info).
Overview
The CLI commands in this document can be used to verify the status of an IPSEC tunnel, validate tunnel monitoring, clear the tunnel, and restore the tunnel.
Details
To monitor the tunnel or verify that the tunnel is active:
> show vpn flow
Note: For tunnel monitoring, a monitor status of down is an indicator that the destination IP being monitored is not reachable.
Example:
> show vpn flow
total tunnels configured: 1
filter - type IPSec, state any
total IPSec tunnel configured: 1
total IPSec tunnel shown: 1
id name state monitor local-ip peer-ip tunnel-i/f
--------------------------------------------------------------------------
4 tunnel active up 172.17.128.135 172.17.128.1
filter - type IPSec, state any
total IPSec tunnel configured: 1
total IPSec tunnel shown: 1
id name state monitor local-ip peer-ip tunnel-i/f
--------------------------------------------------------------------------
4 tunnel active up 172.17.128.135 172.17.128.1
To confirm that data is passing through the tunnel:
show vpn flow tunnel-id x << where x=id number from above display
Example:
The output below is at the bottom of the display and should be increasing as the traffic flows.
> show vpn flow tunnel-id 2
encap packets: 500
decap packets: 500
encap bytes: 54312
decap bytes: 54312
key acquire requests: 35
encap packets: 500
decap packets: 500
encap bytes: 54312
decap bytes: 54312
key acquire requests: 35
To view the details on the active IKE phase 1 SAs:
show vpn ike-sa gateway <gateway name>
Example:
> show vpn ike-sa gateway GW-to-Lab1
phase-1 SAs
GwID Peer-Address Gateway Name Role Mode Algorithm
---- ------------- --------------- ---------------------------
1 57.1.1.1 gw-to-Lab1 Init Main PSK/DH2/A128/SHA1
Show IKEv1 IKE SA: Total 1 gateways found.
phase-1 SAs
GwID Peer-Address Gateway Name Role Mode Algorithm
---- ------------- --------------- ---------------------------
1 57.1.1.1 gw-to-Lab1 Init Main PSK/DH2/A128/SHA1
Show IKEv1 IKE SA: Total 1 gateways found.
To view the details on the active IKE phase 2 SAs:
show vpn ipsec-sa tunnel <tunnel name>
Example:
> show vpn ipsec-sa tunnel IPVPN-tunnel1.1-to-LAB1
GwID TnID Peer-Address Tunnel(Gateway) Algorithm
---- ----- ------------ ----------------------- ------------
1 2 57.1.1.1 IPVPN-tunnel1.1-to-LAB1 ESP/A128/SHA1
Show IPSec SA: Total 1 tunnels found.
GwID TnID Peer-Address Tunnel(Gateway) Algorithm
---- ----- ------------ ----------------------- ------------
1 2 57.1.1.1 IPVPN-tunnel1.1-to-LAB1 ESP/A128/SHA1
Show IPSec SA: Total 1 tunnels found.
The following commands will tear down the VPN tunnel:
> clear vpn ike-sa gateway GW-to-Lab1
Delete IKEv1 IKE SA: Total 1 gateways found.
> clear vpn ipsec-sa tunnel IPVPN-tunnel1.1-to-LAB1
Delete IKEv1 IPSec SA: Total 1 tunnels found.
Delete IKEv1 IKE SA: Total 1 gateways found.
> clear vpn ipsec-sa tunnel IPVPN-tunnel1.1-to-LAB1
Delete IKEv1 IPSec SA: Total 1 tunnels found.
The following commands will bring up the VPN tunnel:
> test vpn ike-sa gateway GW-to-Lab1
Initiate IKE SA: Total 1 gateways found. 1 ike sa found.
> test vpn ipsec-sa tunnel IPVPN-tunnel1.1-to-LAB1
Initiate IPSec SA: Total 1 tunnels found. 1 ipsec sa found.
Initiate IKE SA: Total 1 gateways found. 1 ike sa found.
> test vpn ipsec-sa tunnel IPVPN-tunnel1.1-to-LAB1
Initiate IPSec SA: Total 1 tunnels found. 1 ipsec sa found.
Note: GW-to-Lab1 and IPVPN-tunnel1.1-to-LAB1 are example gateway and tunnel names, respectively.
Phase 1
Phase 2
owner: panagent
No comments:
Post a Comment