FortiGate IPsec Troubleshooting From IKE to Routing
An IPsec tunnel can be “up” and still fail to carry the traffic users need. That is why VPN troubleshooting should not stop at a green status indicator. The useful question is which stage of the path is working: peer reachability, IKE negotiation, IPsec security associations, selector matching, routing, firewall policy, NAT behavior, and return traffic.
Site-to-site IPsec and operational troubleshooting both belong to the skill set measured by the current FortiOS 7.6 Administrator exam. The most transferable method is to move from the tunnel’s control-plane establishment toward the actual data-plane flow instead of changing multiple VPN settings at once.
The broader Fortinet certification path covers more advanced network-security roles, but the same evidence order remains useful. First prove that the peers can negotiate. Then prove that the intended packet is eligible for the tunnel and has a valid path on both sides.
This stage-by-stage method prevents a common mistake: treating every VPN symptom as a cryptography problem. Many “VPN failures” are ordinary routing, policy, translation, or asymmetry problems that happen after negotiation succeeds.
Begin with peer reachability before debugging IKE
IKE cannot succeed if the peers cannot reach each other over the underlay. Confirm the local interface, public or transport address, upstream route, and any intermediate filtering. If the peer address itself is unreachable, changing proposals or pre-shared keys will not help.
This is especially important in multi-WAN or SD-WAN environments where the expected source interface may differ from the path the routing table currently prefers. Establish which interface and address the FortiGate should use before interpreting negotiation logs.
Phase 1 proves identity and establishes the secure control relationship
Phase 1 or IKE negotiation depends on compatible versions, authentication, encryption and integrity proposals, identities, and timing parameters. Mismatches at this stage usually leave clear negotiation evidence if logging and debugging are focused on the correct peer.
Rather than comparing whole configurations line by line, compare the negotiated parameters that matter. Determine whether the peers agree on IKE version, authentication method, proposal, and identity. If one side expects a different peer identity or credential, no amount of Phase 2 adjustment can compensate.
Phase 2 determines which traffic receives IPsec protection
After IKE succeeds, IPsec security associations protect data traffic according to negotiated parameters and traffic selectors. If selectors or protected networks do not align, the tunnel can appear established while the application flow never matches the expected SA.
Write the interesting traffic as source network, destination network, protocol, and direction. Then compare that packet with the selectors on both peers. A single subnet typo or reversed expectation can explain why negotiation looks healthy but byte counters do not move.
Routes determine whether traffic is sent toward the tunnel
FortiGate still needs a route that directs the protected destination toward the VPN interface or appropriate overlay path. Dynamic routing, static routes, policy routes, and SD-WAN can all influence which path wins.
If traffic never selects the tunnel interface, the VPN configuration may be completely correct and still remain unused. Route lookup should therefore be tested with the same destination address the application uses, not with a generic assumption that “the remote subnet is configured somewhere.”
Firewall policy must permit the traffic on both sides of the tunnel
Route-based IPsec designs commonly require firewall policies between the local network and the tunnel interface. The remote FortiGate or other VPN peer needs equivalent permission for the reverse direction according to its platform model. A missing policy often looks like a tunnel problem because the user knows only that the remote application is unreachable.
Confirm the exact policy hit and session rather than adding a broad any-any rule. That preserves security and tells you whether the packet actually reached the policy layer. The discipline is the same one used in firewall operations: prove where the packet was handled before changing enforcement.
NAT should be intentional across protected networks
Site-to-site VPN traffic often needs to preserve original private addresses so the remote network can route and authorize them. Accidental source NAT can therefore cause selectors, server ACLs, or return routing to fail. Other designs deliberately translate overlapping networks, in which case NAT becomes part of the VPN architecture rather than an error.
The troubleshooting record should state whether translation is expected. If it is not, verify that the matched policy does not alter the source. If it is, make sure both peers define protected networks using the addresses they actually see after translation.
Packet counters reveal which direction is broken
Tunnel and session counters help separate no-send, send-no-receive, and bidirectional traffic. If outbound encrypted bytes increase while inbound bytes remain still, the problem is likely beyond the local encryption step: remote policy, remote route, upstream filtering, or the return application path.
If neither direction moves when the application generates traffic, investigate route, selector, and policy matching locally. Directional evidence is more useful than repeatedly bouncing the tunnel because it narrows the failure domain.
Return-path asymmetry can break an otherwise correct VPN
The remote network must return traffic through the peer that owns the IPsec relationship. A second internet path, internal router, cloud route table, or overlapping default can send replies elsewhere. The FortiGate then never sees traffic needed to complete the session.
Trace the route from the remote application back to the original source. In hybrid environments, this may require coordination with cloud, WAN, or server teams. VPN troubleshooting is often cross-domain work because the encrypted tunnel is only one segment of the end-to-end path.
A successful VPN test proves data flow, not only negotiation
After a fix, validate with the actual application or a representative packet whose source and destination match the protected networks. Confirm route selection, policy hit, translation behavior, increasing tunnel counters, return traffic, and stable session state.
Then document the evidence that distinguished the fault. Was IKE failing? Did selectors exclude the traffic? Did a route send it elsewhere? Did a policy deny it? Did the remote network return asymmetrically? That record makes the next incident faster and prevents the team from treating every tunnel alarm as the same problem.
IPsec troubleshooting is easiest when the tunnel is treated as a sequence of dependencies rather than a single object. Peer reachability enables negotiation, negotiation creates SAs, selectors identify traffic, routing sends the packet to the tunnel, policy permits it, and the far side must provide a valid return path. Follow that sequence and the failure usually becomes visible.
Certificate-based VPNs add another dependency chain. The certificate must be valid for the intended identity, trusted by the peer, within its validity period, and supported by the configured authentication method. A tunnel that worked yesterday can fail after certificate renewal if the new chain, subject, or key does not match what the peer expects. Time synchronization matters here as well because an incorrect clock can make a valid certificate appear expired or not yet valid.
NAT traversal introduces a separate clue when one or both peers sit behind address translation. IKE and ESP handling can change so the peers encapsulate IPsec in UDP. If the network path changes or an upstream device blocks the required traffic, the failure may appear after a provider or edge-router change even though neither FortiGate VPN configuration was edited.
Dead Peer Detection and rekey behavior affect intermittent incidents. A tunnel may establish normally, pass traffic for hours, then fail during reauthentication or key rollover. Troubleshooting should capture the time of failure relative to SA lifetimes and DPD events. Restarting the tunnel can hide that pattern because it resets the timers without correcting the underlying mismatch.
Dynamic routing over IPsec adds useful resilience but also more control-plane evidence to inspect. The tunnel can be established while BGP or OSPF adjacency is down, leaving remote prefixes absent. Conversely, the route may remain briefly while the underlying tunnel has degraded. Operators should verify both the encrypted transport and the routing protocol that depends on it.
When multiple tunnels protect the same destinations, route preference and failover behavior become part of the VPN design. A secondary tunnel is valuable only if the remote peer, route table, policy, and monitoring all support the alternate path. Test planned failover with active application traffic so the team knows whether sessions survive, reconnect, or select an unexpected route.
MTU and fragmentation can create a particularly deceptive VPN symptom. Small pings and basic handshakes work, while larger application transfers stall because encapsulation increases packet size beyond what part of the path can carry. Path MTU discovery, MSS adjustment, and intermediate filtering of ICMP can all influence the behavior. Test packet sizes that resemble the application rather than relying only on a minimal connectivity probe.
Overlapping or summarized routes can create a tunnel that carries some destinations correctly and sends others toward a local interface or alternate WAN. Route lookup should therefore be performed for the exact failing host, not only the remote subnet’s first address. Longest-prefix matching can make a more specific route override the path the administrator expects from the high-level design.
Good VPN monitoring separates tunnel establishment from service health. An alert should distinguish an IKE negotiation failure, a missing Phase 2 association, a route loss, or a tunnel that remains established while the remote application is unreachable. Those conditions have different owners and recovery actions, and collapsing them into one “VPN down” state slows response.
Configuration backups from both peers are valuable evidence when a previously stable tunnel breaks after change. Comparing the last known-good state with current proposals, selectors, routes, and policies often narrows the fault faster than rebuilding the tunnel from memory.