About the heartbeat and synchronization

Heartbeat and synchronization traffic consists of TCP packets transmitted between the FortiMail units in the HA group through the primary and secondary heartbeat interfaces.

Heartbeat and synchronization traffic has three primary functions:

• to monitor the responsiveness of the HA group members
• to synchronize configuration changes from the primary unit to the secondary units

For exceptions to synchronized configuration items, see “Configuration settings that are not synchronized” on page 309.

• to synchronize mail data from the primary unit to the secondary unit (active-passive only)

Mail data consists of the FortiMail system mail directory, user home directories, and mail queue.

When the primary unit’s configuration changes, it immediately synchronizes the change to the secondary unit (or, in a config-only HA group, to the peer units) through the primary heartbeat interface. If this fails, or if you have inadvertently de-synchronized the secondary unit’s configuration, you can manually initiate synchronization. For details, see “click HERE to start a configuration/data sync” on page 316. You can also use the CLI command diagnose system ha sync on either the primary unit or the secondary unit to manually synchronize the configuration. For details, see the FortiMail CLI Reference.

During normal operation, the secondary unit expects to constantly receive heartbeat traffic from the primary unit. Loss of the heartbeat signal interrupts the HA group, and, if it is active-passive in style, generally triggers a failover. For details, see “Failover scenario 1: Temporary failure of the primary unit” on page 331.

Exceptions include system restarts and the execute reload CLI command. In case of a system reboot or reload of the primary unit, the primary unit signals the secondary unit to wait for the primary unit to complete the restart or reload. For details, see “Failover scenario 2: System reboot or reload of the primary unit” on page 332.

Periodically, the secondary unit checks with the primary unit to see if there are any configuration changes on the primary unit. If there are configuration changes, the secondary unit will pull the configuration changes from the primary unit, generate a new configuration, and reload the new configuration. In this case, both the primary and secondary units send alert email. For details, see “Failover scenario 3: System reboot or reload of the secondary unit” on page 333.

Behavior varies by your HA mode when the heartbeat fails:

• Active-passive HA

A new primary unit is elected: the secondary unit becomes the new primary unit and assumes the duty of processing of email. During the failover, no mail data or configuration changes are lost, but some in-progress email deliveries may be interrupted. These interrupted deliveries may need to be restarted, but most email clients and servers can gracefully handle this. Additional failover behaviors may be configured. For details, see “On failure” on page 322.

Maintain the heartbeat connection. If the heartbeat is accidentally interrupted for an active-passive HA group, such as when a network cable is temporarily disconnected, the secondary unit will assume that the primary unit has failed, and become the new primary unit. If no failure has actually occurred, both FortiMail units will be operating as primary units simultaneously. For details on correcting this, see “click HERE to restore configured operating mode” on page 317.

• Config-only HA

Each secondary unit continues to operate normally. However, with no primary unit, changes to the configuration are no longer synchronized. You must manually configure one of the secondary units to operate as the primary unit, synchronizing its changes to the remaining secondary units.

For failover examples and steps required to restore normal operation of the HA group in each case, see “Example: Failover scenarios” on page 330.

Configuration settings that are not synchronized

All configuration settings on the primary unit are synchronized to the secondary unit, except the following:

Table 32:HA settings not synchronized

Operation mode	You must set the operation mode (gateway, transparent, or server) of each HA group member before configuring HA. For details, see “Operation mode” on page 175.
Host name	The host name distinguishes members of the cluster. For details, see “Host name” on page 368.
Static route	Static routes are not synchronized because the HA units may be in different networks (see “Configuring static routes” on page 258).

Interface         Each FortiMail unit in the HA group must be configured with different configuration network interface settings for connectivity purposes. For details, see

“Configuring the network interfaces” on page 247.

(gateway and server

mode only)               Exceptions include some active-passive HA settings which affect the interface configuration for failover purposes. These settings are synchronized. For details, see “Virtual IP Address” on page 342.

Management IP address (transparent mode only)	Each FortiMail unit in the HA group should be configured with different management IP addresses for connectivity purposes. For details, see “About the management IP” on page 245.
SNMP system information	Each FortiMail unit in the HA group will have its own SNMP system information, including the Description, Location, and Contact. For details, see “Configuring the network interfaces” on page 247.
RAID configuration	RAID settings are hardware-dependent and determined at boot time by looking at the drives (for software RAID) or the controller (hardware RAID), and are not stored in the system configuration. Therefore, they are not synchronized.
Main HA configuration	The main HA configuration, which includes the HA mode of operation (such as master or slave), is not synchronized because this configuration must be different on the primary and secondary units. For details, see “Configuring the HA mode and group” on page 319.

Table 32:HA settings not synchronized

HA Daemon configuration	The following HA daemon settings are not synchronized: •      Shared password •      Backup mail data directories •      Backup MTA queue directories You must add the shared HA password to each unit in the HA group. All units in the HA group must use the same shared password to identify the group. Synchronized HA daemon options that are active-passive HA settings affect how often the secondary unit tests the primary unit and how the secondary unit synchronizes configuration and mail data. Because HA daemon settings on the secondary unit control how the HA daemon operates, in a functioning HA group you would change the HA daemon configuration on the secondary unit to change how the HA daemon operates. The HA daemon settings on the primary unit do not affect the operation of the HA daemon. Note: In some active-passive HA groups, you may want to have different HA daemon configurations on the primary and secondary units. For example, after a failover, if the failed primary unit restarts and becomes a secondary unit, you might not want the new secondary unit to synchronize with the new primary unit in the same way as when the HA group is functioning normally.
HA service monitoring configuration	In active-passive HA, the HA service monitoring configuration is not synchronized. The remote service monitoring configuration on the secondary unit controls how the secondary unit checks the operation of the primary unit. The local services configuration on the primary unit controls how the primary unit tests the operation of the primary unit. For details, see “Configuring service-based failover” on page 328. Note: You might want to have a different service monitoring configuration on the primary and secondary units. For example, after a failover you may not want service monitoring to operate until you have fixed the problems that caused the failover and have restarted normal operation of the HA group.
System appearance	The appearance settings you configured under System > Customization > Appearance are not synchronized.
Config-only HA	In config-only HA, the following settings are not synchronized: •      the local domain name (see “Local domain name” on page 368) •      the quarantine report host name (see “Web release host name/IP” on page 604) •      User-level black/white lists (but they can be synchronized manually by clicking the link on the HA status page. They are also synchronized from onfig master to config slave when the slave unit joins the HA cluster for the first time.) System and domain-level black/white lists are synchronized. Note that before v5.0.2 release, domain-level black/white lists are not automatically synchronized either.