Category Archives: FortiOS 5.4 Handbook

The complete handbook for FortiOS 5.4

FortiOS 5.4 HA new features

FortiOS 5.4 HA new features

FGCP supports BFD enabled BGP graceful restart after an HA failover (255574)

If an HA cluster is part of a Border Gateway Protocol (BGP) bidirectional forwarding detection (BFD) configuration where both the cluster and the BGP static neighbor are configured for graceful restart, after an HA failover BGP enters graceful restart mode and both the cluster and the BGP neighbor keep their BGP routes.

To support HA and BFD enabled BGP graceful:

  • From the cluster, configure the BFD enabled BGP neighbor as a static BFD neighbor using the config router bfd command.Set the BGP auto-start timer to 5 seconds so that after an HA failover BGP on the new primary unit waits for 5 seconds before connect to its BFD neighbors, and then registers BFD requests after establishing the connections. With static BFD neighbors, BFD requests and sessions can be created as soon as possible after the failover.The command get router info bfd requests shows the BFD peer requests.
  • The BFD session created for a static BFD neighbor/peer request initializes its state as INIT instead of DOWN and its detection time as bfd-required-min-rx * bfd-detect-mult msecs.
  • When a BFD control packet with a nonzero Your Discriminator (your_discr) value is received, if no session can be found to match the your_discr, instead of discarding the packet, other fields in the packet, such as addressing information, are used to choose one session that was just initialized, with zero as its remote discriminator.
  • When a BFD session in the UP state receives a control packet with zero as Your Discriminator and DOWN as State, the session changes its state to DOWN but will not notify this DOWN event to BGP and/or other registered clients.

 

FRUP is not supported by FortiOS 5.4 (295198)

With the changes to switch mode, FRUP is no longer available on the FortiGate-100D.

 

VOIP application control sessions are no longer blocked after an HA failover (273544)

After an HA failover, VoIP sessions that are being scanned by application control will now continue with only a minor interruption, if any. To support this feature, IPS UDP expectation tables are now synchronized between cluster units.

 

Firewall local-in policies are supported for the dedicated HA management interface (276779 246574)

To add local in polices for the dedicated management interface, enable ha-mgmt-inft-only and set intf to any. Enabling ha-mgmt-intf-only means the local-in policy applies only to the VDOM that contains the dedicated HA management interface.

config firewall local-in-policy edit 0

set ha-mgmt-intf-only enable set intf any

etc… end

 

 

HA heartbeat traffic set to the same priority level as data traffic (276665)

Local out traffic, including HA heartbeat traffic, is now set to high priority to make sure it is processed at the same priority level as data traffic. This change has been made because HA heartbeat traffic can be processed by NP6 processors that are also processing data traffic. When HA heartbeat traffic was set to a lower priority it may have be delayed or dropped by very busy NP6 processors resulting in HA failovers.

 

FGSP CLI command name changed (262340)

The FortiOS 5.2 command config system session-sync has been changed in FortiOS 5.4 to config system cluster-sync. Otherwise the command syntax is the same and the config system ha commands used for FGSP settings have not changed.

 

FGSP now supports synchronizing IPsec sessions (262340)

The FGSP now synchronizes IPsec tunnels between FortiGates in an FGSP configuration. IPsec tunnel synchronization synchronizes keys and other run time data between the FortiGates in an FGSP configuration. No additional configuration is required to synchronize IPsec sessions. Also you cannot disable IPsec tunnel synchronization.

The FGSP synchronizes IPsec keys and other runtime data but not actual tunnel sessions. This means that if one of the cluster units goes down the cluster unit that is still operating can quickly get IPsec tunnels re-established without re-negotiating them but all existing tunnel sessions on the failed FortiGate have to be restarted on the still operating FortiGate.

IPsec tunnel sync only supports dialup IPsec. The interfaces on both FortiGates that are tunnel endpoints must have the same IP addresses and external routers must be configured to load balance IPsec tunnel sessions to the FortiGates in the cluster.

 

Monitoring VLAN interfaces (220773)

When operating in HA mode and if you have added VLAN interfaces to the FortiGates in the cluster, you can use the following command to monitor all VLAN interfaces and send a message if one of the VLAN interfaces is found to be down.

config system ha-monitor

set monitor-vlan enable/disable

set vlan-hb-interval <interval_seconds>

set vlan-hb-lost-threshold <vlan-lost-heartbeat-threshold>

end

Once configured, this feature works by verifying that the primary unit can connect to the subordinate unit over each VLAN. This verifies that the switch that the VLAN interfaces are connected to is configured correctly for each VLAN. If the primary unit cannot connect to the subordinate unit over one of the configured VLANs the primary unit writes a link monitor log message indicating that the named VLAN went down (log message id 20099).

 

FortiGate HA cluster support for managed switches (276488 266084)

Added the capability to support managed switches from a FortiGate HA cluster. If a standby FortiGate becomes active, it automatically establishes connectivity with the managed switches. See Managing a FortiGate with a FortiSwitch for details.

 

HA cluster health displayed on the Unit Operation dashboard widget (260547)

The Unit Operation dashboard widget now includes the serial number and hostname of all of the FortiGate units in the cluster as well as an indication of the sync status of each cluster member.

 

Chapter 13 – High Availability

Chapter 13 – High Availability

This FortiOS Handbook chapter contains the following sections:

Solving the High Availability problem describes the high availability problem and introduces the FortiOS solutions described in this document (FGCP, VRRP, and standalone session synchronization).

An introduction to the FGCP introduces the FGCP clustering protocol and many of its features and terminology. FGCP configuration examples and troubleshooting describes configuring HA clusters and contains HA clustering configuration examples.

Virtual clusters describes configuring HA virtual clusters and contains virtual clustering configuration examples. Full mesh HA describes configuring FortiGate Full mesh HA and contains a full mesh HA configuration example. Operating a cluster describes how to operate a cluster and includes detailed information about how various FortiGate systems operate differently in a cluster.

HA and failover protection describes in detail how FortiGate HA device failover, link failover, and session failover work.

HA and load balancing describes how FGCP HA active-active load balancing works and how to configure it.

HA with FortiGate-VM and third-party products describes how FortiGate units interact with third-party products. VRRP describes FortiOS support of the Virtual Router Redundancy Protocol (VRRP) and its use for high availability.

FortiGate Session Life Support Protocol (FGSP) describes how to use the FGSP feature to support using external routers or load balancers to distribute or load balance sessions between two peer FortiGate units.

dianose npu np6 synproxy-stats (NP6 SYN-proxied sessions and unacknowledged SYNs)

dianose npu np6 synproxy-stats (NP6 SYN-proxied sessions and unacknowledged SYNs)

This command display information about NP6 syn-proxy sessions including the total number proxied sessions. As well the  Number of attacks, no ACK from client shows the total number of unacknowledge SYNs.

diagnose npu np6 synproxy-stats

DoS SYN-Proxy:

Number of proxied TCP connections : 39277346

Number of working proxied TCP connections : 182860

Number of retired TCP connections : 39094486

Number of attacks, no ACK from client : 208

 

diagnose hardware deviceinfo nic (number of packets dropped by an interface)

diagnose hardware deviceinfo nic <interfac-name> (number of packets dropped by an interface)

This command displays a wide variety of statistics for FortiGate interfaces. The fields Host Rx dropped and Host Tx dropped display the number of received and trasmitted packets that have been dropped.

diagnose hardware deviceinfo nic port2

============ Counters =========== Rx Pkts         :20482043

Rx Bytes        :31047522516

Tx Pkts         :19000495

Tx Bytes        :1393316953

Host Rx Pkts    :27324

Host Rx Bytes   :1602755

Host Rx dropped :0

Host Tx Pkts    :8741

Host Tx Bytes   :5731300

Host Tx dropped :0 sw_rx_pkts      :20482043 sw_rx_bytes     :31047522516 sw_tx_pkts      :19000495 sw_tx_bytes     :1393316953 sw_np_rx_pkts   :19000495 sw_np_rx_bytes  :1469318933 sw_np_tx_pkts   :20482042 sw_np_tx_bytes  :31129450620

 

diagnose npu np6 dce (number of dropped NP6 packets)

 

diagnose npu np6 dce <np6-id> (number of dropped NP6 packets)

This command displays the number of dropped packets for the selected NP6 processor.

  • IHP1_PKTCHK number of dropped IP packets
  • IPSEC0_ENGINB0 number of dropped IPSec
  • TPE_SHAPER number of dropped traffic sharper packets

diag npu np6 dce 1

IHP1_PKTCHK :0000000000001833 [5b] IPSEC0_ENGINB0 :0000000000000003 [80] TPE_SHAPER :0000000000000552 [94]

 

diagnose npu np6 sse-stats (number of NP6 sessions and dropped sessions)

diagnose npu np6 sse-stats <np6-id> (number of NP6 sessions and dropped sessions)

This command displays the total number of inserted, deleted and purged sessions processed by a selected NP6 processor. The number of dropped sessions of each type cam be determined by subtracting the number of successfull sessions from the total number of sessions. For example, the total number of dropped insert sessions is insert-total – insert-success.

diagnose npu np6 sse-stats 0

Counters SSE0 SSE1 Total
————— ————— ————— —————
active 0 0 0
insert-total 25 0 0
insert-success 25 0 0
delete-total 25 0 0
delete-success 25 0 0
purge-total 0 0 0
purge-success 0 0 0
search-total 40956 38049 79005
search-hit 37714 29867 67581
————— ————— ————— —————
pht-size 8421376 8421376  
oft-size 8355840 8355840  
oftfree 8355839 8355839  
PBA 3001  

diagnose sys mcast-session/session6 list (IPv4 and IPv6 multicast sessions)

diagnose sys mcast-session/session6 list (IPv4 and IPv6 multicast sessions)

This command lists all IPv4 or IPv6 multicast sessions. If a multicast session can be offloaded, the output includes the offloadable tag. If the multicast path can be offloaded one of the paths in the command output is tagged as offloaded.

The only way to determine the number of offloaded multicast sessions is to use the diagnose sys mcast- session/session6 list command and count the number of sessions with the offload tag.

diagnose sys mcast-session list

session info: id=3 vf=0 proto=17 172.16.200.55.51108->239.1.1.1.7878

used=2 path=11 duration=1 expire=178 indev=6 pkts=2 state:2cpu offloadable

npu-info in-pid=0 vifid=0 in-vtag=0 npuid=0 queue=0 tae=0

 

path: 2cpu policy=1, outdev=2 out-vtag=0

path: 2cpu policy=1, outdev=3 out-vtag=0

path: offloaded policy=1, outdev=7 out-vtag=0

path: policy=1, outdev=8

out-vtag=0

path: policy=1, outdev=9 out-vtag=0

path: policy=1, outdev=10 out-vtag=0

path: policy=1, outdev=11 out-vtag=0

path: policy=1, outdev=12 out-vtag=0

path: policy=1, outdev=13 out-vtag=0

path: 2cpu policy=1, outdev=64 out-vtag=0

path: 2cpu policy=1, outdev=68 out-vtag=0

 

diagnose npu np6 ipsec-stats (NP6 IPsec statistics)

diagnose npu np6 ipsec-stats (NP6 IPsec statistics)

The command output includes IPv4, IPv6, and NAT46 IPsec information:

  • spi_ses4 is the IPv4 counter
  • spi_ses6 is the IPv6 counter
  • 4to6_ses is the NAT46 counter

diagnose npu np6 ipsec-stats

vif_start_oid       03ed       vif_end_oid         03fc

IPsec Virtual interface stats:

sa_install 00000000000 sa_ins_fail 00000000000
sa_remove 00000000000 sa_del_fail 00000000000
4to6_ses_ins 00000000000 4to6_ses_ins_fail 00000000000
4to6_ses_del 00000000000 4to6_ses_del_fail 00000000000
spi_ses6_ins 00000000000 spi_ses6_ins_fail 00000000000
spi_ses6_del 00000000000 spi_ses6_del_fail 00000000000
spi_ses4_ins 00000000000 spi_ses4_ins_fail 00000000000
spi_ses4_del 00000000000 spi_ses4_del_fail 00000000000
sa_map_alloc_fail 00000000000 vif_alloc_fail 00000000000
sa_ins_null_adapter 00000000000 sa_del_null_adapter 00000000000
del_sa_mismatch 00000000000 ib_chk_null_adpt 00000000000
ib_chk_null_sa 00000000000 ob_chk_null_adpt 00000000000
ob_chk_null_sa 00000000000 rx_vif_miss 00000000000
rx_sa_miss 00000000000 rx_mark_miss 00000000000
waiting_ib_sa 00000000000 sa_mismatch 00000000000
msg_miss 00000000000    
np6_1:      
sa_install 00000000000 sa_ins_fail 00000000000
sa_remove 00000000000 sa_del_fail 00000000000
4to6_ses_ins 00000000000 4to6_ses_ins_fail 00000000000
4to6_ses_del 00000000000 4to6_ses_del_fail 00000000000
spi_ses6_ins 00000000000 spi_ses6_ins_fail 00000000000
spi_ses6_del 00000000000 spi_ses6_del_fail 00000000000
spi_ses4_ins 00000000000 spi_ses4_ins_fail 00000000000
spi_ses4_del 00000000000 spi_ses4_del_fail 00000000000
sa_map_alloc_fail 00000000000 vif_alloc_fail 00000000000
sa_ins_null_adapter 00000000000 sa_del_null_adapter 00000000000
del_sa_mismatch 00000000000 ib_chk_null_adpt 00000000000
ib_chk_null_sa 00000000000 ob_chk_null_adpt 00000000000
ob_chk_null_sa 00000000000 rx_vif_miss 00000000000
rx_sa_miss 00000000000 rx_mark_miss 00000000000
waiting_ib_sa 00000000000 sa_mismatch 00000000000
msg_miss 00000000000    

 

vif_get             00000000000        vif_get_expired     00000000000 vif_get_fail        00000000000        vif_get_invld      00000000000 vif_set             00000000000        vif_set_fail        00000000000 vif_clear           00000000000        vif_clear_fail      00000000000 np6_0: