Update FortiGate firmware

Updating or upgrading a firewall is similar to upgrading the operating system so you should make the same preparations. Make sure everything is backed up and you have a plan in case something doesn’t work. Make a checklist to confirm that the update is successful. Finally, ensure you have enough time to do the update.

This is a summary of the steps for updating FortiGate firmware:

1. Backup and store the old configuration on another server.

Do a full configuration backup using the CLI. This should already be part of your disaster recovery plan. If the upgrade fails, be sure you have a plan to get the firewall back up and running.

2. Have copy of old firmware available.

This should also be part of your disaster recovery plan. If the upgrade fails, you might be able to switch the active partition. But be prepared for the worst case scenario where you need your old firmware.

3. Have a disaster recovery option on standby, especially for a remote site.

This should be part of your plan in a critical failure. In this scenario, this is your plan if your firewall doesn’t come back up after the upgrade. In this case, you need access to the console port to find out why, such as if the DHCP or the IP has changed, or the OS is corrupt. You must have access to the console to find out. If there is no simple fix, be prepared for a format and TFTP reload.

4. Read the release notes, including the upgrade path and bug information.

Be sure to read the release notes, preferably more than once. The release notes contain lots of important information, known bugs, fixed bugs, upgrade issues such as lost configuration settings.

5. Double check everything.

For example, double check that your TFTP server is working, your console connection functions properly, you have read the release notes and understand everything that affects the upgrade for your FortiGate models, you have backed up your configuration, you have covered everything you might need for the upgrade.

6. Perform the upgrade.

The upgrade itself usually doesn’t take very long, usually just a few minutes. But make sure you schedule enough time for the entire process and possible contingencies. If the upgrade is successful, you need time to check and confirm that all important functions are working, such as VPNs etc. If the upgrade fails, you need time to sort things out.

Sample upgrade

This is an example of upgrading the FortiGate from FortiOS 6.0.4 to 6.2.0.

To view the FortiOS firmware:

1. Go to Dashboard.

The System Information widget shows the current firmware version.

To check if a new FortiOS firmware version is available:

1. Go to System > Firmware.

If a new firmware version is available, a notice appears in the Current version section.

When a new FortiOS version is released, it may not be listed on your FortiGate right away. You can download the firmware from Fortinet Support, then use Upload Firmware to upgrade your FortiGate.

To upgrade to the latest version from FortiGuard:

1. Go to System > Firmware.
2. In the FortiGuard Firmware section, click Latest.

If you see a message saying there is no valid upgrade path for this firmware version, click All available and select a suitable firmware version for your FortiGate.

3. Click Release Notes and read the release notes for that version.

Release Notes are also available from the Fortinet Documentation Library.

4. Click Backup config and upgrade and follow the prompts.
5. Save the backup of your configuration in case you need to restore it after the upgrade.

To upgrade to the latest version from local PC:

1. Ensure you have downloaded the firmware from Fortinet Support.
2. Go to System > Firmware.
3. In the Upload Firmware section, click Browse and select the firware.
4. Click Backup config and upgrade and follow the prompts.
5. Save the backup of your configuration in case you need to restore it after the upgrade.

FortiGate uploads and installs the firmware, and then restarts and displays the login screen.

See the procedure above to view the FortiOS firmware to ensure you are running the new firmware version.

Administrators

Administrator profiles

Introduction

By default, the FortiGate has a super administrator account, called admin. Additional administrators can be added for various functions, each with a unique username, password, and set of access privileges.

Administrator profiles define what the administrator can do when logged into the FortiGate. When you set up an administrator account, you also assign an administrator profile which dictates what the administrator sees. Depending on the nature of the administrator’s work, access level or seniority, you can allow them to view and configure as much or as little as is required.

Super_admin profile

This profile has access to all components of FortiOS, including the ability to add and remove other system administrators. For certain administrative functions, such as backing up and restoring the configuration, super_admin access is required. To ensure that there is always a method to administer the FortiGate, the super_admin profile can’t be deleted or modified.

The super_admin profile is used by the default admin account. It is recommended that you add a password and rename this account once you have set up your FortiGate. In order to rename the default account, a second admin account is required.

Creating customized profiles

To create a profile in the GUI:

1. Go to System > Admin Profiles.
2. Select Create New.
3. Configure the following settings: l l Access permissions. l Override idle timeout.
4. Select OK.

To create a profile in the CLI:

config system accprofile

edit “sample”

set secfabgrp read-write set ftviewgrp read-write set authgrp read-write set sysgrp read-write set netgrp read-write set loggrp read-write set fwgrp read-write set vpngrp read-write set utmgrp read-write set wanoptgrp read-write set wifi read-write

next

end

Edit profiles

To edit a profile in the GUI:

1. Go to System > Admin Profiles.
2. Choose the profile to be edited and select Edit.
3. Select OK to save any changes made.

To edit a profile in the CLI:

config system accprofile edit “sample”

set secfabgrp read

next

end

Delete profiles

To delete a profile in the GUI:

1. Go to System > Admin Profiles.
2. Choose the profile to be deleted and select Delete.
3. Select OK.

To delete a profile in the CLI:

config system accprofile

delete “sample” end

Add a local administrator

By default, FortiGate has one super admin named admin. You can create more administrator accounts with difference privileges.

To create an administrator account in the GUI:

1. Go to System > Administrators.
2. Select Create New > Administrator.
3. Specify the Username.
4. Set Type to Local User.
5. Set the password and other fields.
6. Click OK.

To create an administrator account in the CLI:

config system admin edit <Admin_name>

set accprofile <Profile_name> set vdom <Vdom_name>

set password <Password for this admin>

next end

Remote authentication for administrators

Administrators can use remote authentication, such as LDAP, to connect to the FortiGate.

Setting up remote authentication for administrators includes the following steps:

1. Configure the LDAP server on page 153
2. Add the LDAP server to a user group on page 154
3. Configure the administrator account on page 154

Configure the LDAP server

To configure the LDAP server in the GUI:

1. Go to User& Device > LDAP Servers and select Create New.
2. Enter the server Name, ServerIP address or Name.
3. Enter the Common Name Identifier and Distinguished Name.
4. Set the Bind Type to Regular and enter the Username and Password.
5. Click OK.

To configure the LDAP server in the CLI:

config user ldap

edit <ldap_server_name>

set server <server_ip> set cnid “cn”

set dn “dc=XYZ,dc=fortinet,dc=COM”

set type regular

set username “cn=Administrator,dc=XYA, dc=COM” set password <password>

next

end

Add the LDAP server to a user group

After configuring the LDAP server, create a user group that include the LDAP server you configured.

To create a user group in the GUI:

1. Go to User& Device > UserGroups and select Create New.
2. Enter a Name for the group.
3. In the Remote groups section, select Create New.
4. Select the Remote Server from the dropdown list.
5. Click OK.

To create a user group in the CLI:

config user group

edit <Group_name>

set member “ldap_server_name”

next

end

Configure the administrator account

After configuring the LDAP server and adding it to a user group, create a new administrator. For this administrator, instead of entering a password, use the new user group and the wildcard option for authentication.

To create an administrator in the GUI:

1. Go to System > Administrators.
2. Select Create New > Administrator.
3. Specify the Username.
4. Set Type to Match a useron a remote servergroup.
5. In Remote UserGroup, select the user group you created.
6. Select Wildcard.

The Wildcard option allows LDAP users to connect as this administrator.

7. Select an AdministratorProfile.
8. Click OK.

To create an administrator in the CLI:

config system admin edit <admin_name>

set remote-auth enable set accprofile super_admin set wild card enable set remote-group ldap

end

Other methods of administrator authentication

Administrator accounts can use different methods for authentication, including RADIUS, TACACS+, and PKI.

RADIUS authentication for administrators

To use a RADIUS server to authenticate administrators, you must:

• Configure the FortiGate to access the RADIUS server. l Create the RADIUS user group. l Configure an administrator to authenticate with a RADIUS server.

TACACS+ authentication for administrators

To use a TACACS+ server to authenticate administrators, you must:

• Configure the FortiGate to access the TACACS+ server. l Create a TACACS+ user group. l Configure an administrator to authenticate with a TACACS+ server.

PKI certificate authentication for administrators

To use PKI authentication for an administrator, you must:

• Configure a PKI user. l Create a PKI user group. l Configure an administrator to authenticate with a PKI certificate.

Password policy

Brute force password software can launch more than just dictionary attacks. It can discover common passwords where a letter is replaced by a number. For example, if p4ssw0rd is used as a password, it can be cracked.

Using secure passwords is vital for preventing unauthorized access to your FortiGate. When changing the password, consider the following to ensure better security:

• Do not use passwords that are obvious, such as the company name, administrator names, or other obvious words or phrases.
• Use numbers in place of letters, for example, passw0rd. l Administrator passwords can be up to 64 characters. l Include a mixture of numbers, and upper and lower case letters. l Use multiple words together, or possibly even a sentence, for example keytothehighway. l Use a password generator.
• Change the password regularly and always make the new password unique and not a variation of the existing password, such as changing from password to password1.
• Make note of the password and store it in a safe place away from the management computer, in case you forget it; or ensure at least two people know the password in the event one person becomes unavailable. Alternatively, have two different admin logins.

FortiGate allows you to create a password policy for administrators and IPsec pre-shared keys. With this policy, you can enforce regular changes and specific criteria for a password policy including:

• Minimum length between 8 and 64 characters.
• If the password must contain uppercase (A, B, C) and/or lowercase (a, b, c) characters. l If the password must contain numbers (1, 2, 3). l If the password must contain special or non-alphanumeric characters (!, @, #, $, %, ^, &, *, (, and )). l Where the password applies (admin or IPsec or both). l The duration of the password before a new one must be specified.

If you add a password policy or change the requirements on an existing policy, the next time that administrator logs into the FortiGate, the administrator is prompted to update the password to meet the new requirements before proceeding to log in.

To create a system password policy the GUI:

1. Go to System > Settings.
2. In the Password Policy section, change the Password scope to Admin, IPsec, or Both.
3. Specify the password options.
4. Click Apply.

To create a system password policy the CLI:

config system password-policy

status      Enable/disable setting a password policy for locally defined administrator passwords and IPsec VPN pre-shared keys. apply-to Apply password policy to administrator passwords or IPsec preshared keys or both. Separate entries with a space.

minimum-length	Minimum password length (8 – 128, default = 8).
min-lower-case-letter default = 0).	Minimum number of lowercase characters in password (0 – 128,
min-upper-case-letter default = 0).	Minimum number of uppercase characters in password (0 – 128,
min-non-alphanumeric 128, default = 0).	Minimum number of non-alphanumeric characters in password (0 –
min-number default = 0).	Minimum number of numeric characters in password (0 – 128,
change-4-characters (This attribute	Enable/disable changing at least 4 characters for a new password

overrides reuse-password if both are enabled). expire-status     Enable/disable password expiration.

reuse-password          Enable/disable reusing of password (if both reuse-password and

change-4-characters are enabled, change-4-characters overrides). end

System management introduction

This topic contains information about FortiGate administration that you can do after installing the FortiGate in your network.

Basic system settings

Administrator

By default, FortiGate has an administrator account with the username admin and no password. To prevent unauthorized access to the FortiGate, we highly recommended that you protect this account with a password.

Administrator profile

An administrator profile defines what the administrator can do on the FortiGate. You can set up different administrator profiles depending on the nature of the administrator’s work, access level, or seniority. When you set up an administrator account, assign the administrator profile for what that administrator can do.

Interface

Both the physical and virtual interface allow traffic to flow between internal networks, and between the Internet and internal networks. FortiGate has options for setting up interfaces and groups of sub-networks that can scale as your organization grows. You can create and edit VLAN, EMAC-VLAN, switch interface, zone, and so on.

Advanced system settings

Password policy

Set up a password policy for administrators and IPsec pre-shared keys. A password policy can enforce password criteria and change frequency.

SNMP

The Simple Network Management Protocol (SNMP) allows you to monitor hardware on your network. You can configure hardware such as the FortiGate SNMP agent to report system information and traps. SNMP traps alert you to events that happen such as when a log disk is full or a virus is detected.

DHCP server

You can configure one or more DHCP servers on any FortiGate interface. A DHCP server dynamically assigns IP addresses to hosts on the network connected to the interface. Host computers must be configured to obtain their IP addresses using DHCP.

VDOM

You can use virtual domains (VDOMs) to divide a FortiGate into multiple virtual devices that function independently. For each separate VDOM, you can create different configurations including firewall policies, routing, VPNs, and security profiles.

Troubleshooting

Tracking SD-WAN sessions

You can check the destination interface in FortiView in order to see which port the traffic is being forwarded to.

The example below demonstrates a source-based load-balance between two SD-WAN members.

• If the source IP address is an even number, it will go to port13.
• If the source IP address is an odd number, it will go to port12.

For information on other features of FortiView, see FortiView on page 91.

Understanding SD-WAN related logs

This topic lists the SD-WAN related logs and explains when the logs will be triggered.

Health-check detects a failure:

• When health-check detects a failure, it will record a log:

34: date=2019-03-23 time=17:26:06 logid=”0100022921″ type=”event” subtype=”system” level=”critical” vd=”root” eventtime=1553387165 logdesc=”Routing information changed” name=”test” interface=”R150″ status=”down” msg=”Static route on interface R150 may be removed by health-check test. Route: (10.100.1.2->10.100.2.22 ping-down)”

• When health-check detects a recovery, it will record a log:

32: date=2019-03-23 time=17:26:54 logid=”0100022921″ type=”event” subtype=”system” level=”critical” vd=”root” eventtime=1553387214 logdesc=”Routing information changed” name=”test” interface=”R150″ status=”up” msg=”Static route on interface R150 may be added by health-check test. Route: (10.100.1.2->10.100.2.22 ping-up)”

Health-check has an SLA target and detects SLA qualification changes:

• When health-check has an SLA target and detects SLA changes, and changes to fail:

5: date=2019-04-11 time=11:48:39 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1555008519816639290 logdesc=”Virtual WAN Link status” msg=”SD-WAN Health Check(ping) SLA(1): number of pass members changes from 2 to 1.”

• When health-check has an SLA target and detects SLA changes, and changes to pass:

2: date=2019-04-11 time=11:49:46 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1555008586149038471 logdesc=”Virtual WAN Link status” msg=”SD-WAN Health Check(ping) SLA(1): number of pass members changes from 1 to 2.”

SD-WAN calculates a link’s session/bandwidth over/under its ratio and stops/resumes traffic:

• When SD-WAN calculates a link’s session/bandwidth over its configured ratio and stops forwarding traffic:

3: date=2019-04-10 time=17:15:40 logid=”0100022924″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1554941740185866628 logdesc=”Virtual WAN Link volume status” interface=”R160″ msg=”The member(3) enters into conservative status with limited ablity to receive new sessions for too much traffic.” l When SD-WAN calculates a link’s session/bandwidth according to its ratio and resumes forwarding traffic:

1: date=2019-04-10 time=17:20:39 logid=”0100022924″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1554942040196041728 logdesc=”Virtual WAN Link volume status” interface=”R160″ msg=”The member(3) resume normal status to receive new sessions for internal adjustment.”

The SLA mode service rule’s SLA qualified member changes:

• When the SLA mode service rule’s SLA qualified member changes. In this example R150 fails the SLA check, but is still alive:

14: date=2019-03-23 time=17:44:12 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553388252 logdesc=”Virtual WAN Link status” msg=”Service2() prioritized by SLA will be redirected in seq-num order 2(R160) 1(R150).” 15: date=2019-03-23 time=17:44:12 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553388252 logdesc=”Virtual WAN Link status” interface=”R150″ msg=”The member1(R150) SLA order changed from 1 to 2. ”

16: date=2019-03-23 time=17:44:12 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553388252 logdesc=”Virtual WAN Link status” interface=”R160″ msg=”The member2(R160) SLA order changed from 2 to 1. ”

• When the SLA mode service rule’s SLA qualified member changes. In this example R150 changes from fail to pass:

1: date=2019-03-23 time=17:46:05 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553388365 logdesc=”Virtual WAN Link status” msg=”Service2() prioritized by SLA will be redirected in seq-num order 1(R150) 2(R160).” 2: date=2019-03-23 time=17:46:05 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553388365 logdesc=”Virtual WAN Link status” interface=”R160″ msg=”The member2(R160) SLA order changed from 1 to 2. ” 3: date=2019-03-23 time=17:46:05 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553388365 logdesc=”Virtual WAN Link status” interface=”R150″ msg=”The member1(R150) SLA order changed from 2 to 1. ”

The priority mode service rule member’s link status changes:

• When priority mode service rule member’s link status changes. In this example R150 changes to better than R160, and both are still alive:

1: date=2019-03-23 time=17:33:23 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553387603 logdesc=”Virtual WAN Link status” msg=”Service2() prioritized by packet-loss will be redirected in seq-num order 1(R150) 2 (R160).”

2: date=2019-03-23 time=17:33:23 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553387603 logdesc=”Virtual WAN Link status” interface=”R160″ msg=”The member2(R160) link quality packet-loss order changed from 1 to 2.

”

3: date=2019-03-23 time=17:33:23 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553387603 logdesc=”Virtual WAN Link status” interface=”R150″ msg=”The member1(R150) link quality packet-loss order changed from 2 to 1. ” l When priority mode service rule member’s link status changes. In this example R160 changes to better than R150, and both are still alive:

6: date=2019-03-23 time=17:32:01 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553387520 logdesc=”Virtual WAN Link status” msg=”Service2() prioritized by packet-loss will be redirected in seq-num order 2(R160) 1 (R150).”

7: date=2019-03-23 time=17:32:01 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553387520 logdesc=”Virtual WAN Link status” interface=”R150″ msg=”The member1(R150) link quality packet-loss order changed from 1 to 2.

”

8: date=2019-03-23 time=17:32:01 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553387520 logdesc=”Virtual WAN Link status” interface=”R160″ msg=”The member2(R160) link quality packet-loss order changed from 2 to 1. ”

SD-WAN member is used in service and it fails the health-check:

• When SD-WAN member fails the health-check, it will stop forwarding traffic:

6: date=2019-04-11 time=13:33:21 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1555014801844089814 logdesc=”Virtual WAN Link status” interface=”R160″ msg=”The member2(R160) link is unreachable or miss threshold. Stop forwarding traffic. ”

• When SD-WAN member passes the health-check again, it will resume forwarding logs:

2: date=2019-04-11 time=13:33:36 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1555014815914643626 logdesc=”Virtual WAN Link status” interface=”R160″ msg=”The member2(R160) link is available. Start forwarding traffic. ”

Load-balance mode service rule’s SLA qualified member changes:

• When load-balance mode service rule’s SLA qualified member changes. In this example R150 changes to not meet SLA:

2: date=2019-04-11 time=14:11:16 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1555017075926510687 logdesc=”Virtual WAN Link status” msg=”Service1(rule2) will be load balanced among members 2(R160) with available routing.” 3: date=2019-04-11 time=14:11:16 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1555017075926508676 logdesc=”Virtual WAN Link status”

interface=”R150″ msg=”The member1(R150) SLA order changed from 1 to 2. ” 4: date=2019-04-11 time=14:11:16 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1555017075926507182 logdesc=”Virtual WAN Link status” interface=”R160″ msg=”The member2(R160) SLA order changed from 2 to 1. ”

• When load-balance mode service rule’s SLA qualified member changes. In this example R150 changes to meet SLA:

1: date=2019-04-11 time=14:33:23 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1555017075926510668 logdesc=”Virtual WAN Link status” msg=”Service1(rule2) will be load balanced among members 1(R150) 2(R160) with available routing.”

2: date=2019-03-23 time=14:33:23 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553387603592651068 logdesc=”Virtual WAN Link status” interface=”R160″ msg=”The member2(R160) link quality packet-loss order changed from 1 to 2.

”

3: date=2019-03-23 time=14:33:23 logid=”0100022923″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553387603592651068 logdesc=”Virtual WAN Link status” interface=”R150″ msg=”The member1(R150) link quality packet-loss order changed from 2 to 1. ”

SLA link status logs, generated with interval sla-fail-log-period or sla-pass-log-period:

l When SLA fails, SLA link status logs will be generated with interval sla-fail-log-period:

7: date=2019-03-23 time=17:45:54 logid=”0100022925″ type=”event” subtype=”system” level=”notice” vd=”root” eventtime=1553388352 logdesc=”Link monitor SLA information” name=”test” interface=”R150″ status=”up” msg=”Latency: 0.016, jitter: 0.002, packet loss: 21.000%, inbandwidth: 0Mbps, outbandwidth: 200Mbps, bibandwidth: 200Mbps, sla_map: 0x0″ l When SLA passes, SLA link status logs will be generated with interval sla-pass-log-period:

5: date=2019-03-23 time=17:46:05 logid=”0100022925″ type=”event” subtype=”system” level=”information” vd=”root” eventtime=1553388363 logdesc=”Link monitor SLA information” name=”test” interface=”R150″ status=”up” msg=”Latency: 0.017, jitter: 0.003, packet loss:

0.000%, inbandwidth: 0Mbps, outbandwidth: 200Mbps, bibandwidth: 200Mbps, sla_map: 0x1″

SD-WAN related diagnose commands

This topic lists the SD-WAN related diagnose commands and related output.

To check SD-WAN health-check status:

FGT # diagnose sys virtual-wan-link health-check Health Check(server):

Seq(1): state(alive), packet-loss(0.000%) latency(15.247), jitter(5.231) sla_map=0x0

Seq(2): state(alive), packet-loss(0.000%) latency(13.621), jitter(6.905) sla_map=0x0

FGT  # diagnose sys virtual-wan-link health-check Health Check(ping):

Seq(1): state(alive), packet-loss(0.000%) latency(0.683), jitter(0.082) sla_map=0x0 Seq(2): state(dead), packet-loss(100.000%) sla_map=0x0

FGT # diagnose sys virtual-wan-link health-check google Health Check(google):

Seq(1): state(alive), packet-loss(0.000%) latency(14.563), jitter(4.334) sla_map=0x0

Seq(2): state(alive), packet-loss(0.000%) latency(12.633), jitter(6.265) sla_map=0x0

To check SD-WAN member status:

l When SD-WAN load-balance mode is source-ip-based/source-dest-ip-based.

FGT # diagnose sys virtual-wan-link member

Member(1): interface: port13, gateway: 10.100.1.1 2004:10:100:1::1, priority: 0, weight: 0

Member(2): interface: port15, gateway: 10.100.1.5 2004:10:100:1::5, priority: 0, weight: 0 l When SD-WAN load-balance mode is weight-based.

FGT # diagnose sys virtual-wan-link member

Member(1): interface: port13, gateway: 10.100.1.1 2004:10:100:1::1, priority: 0, weight: 33

Member(2): interface: port15, gateway: 10.100.1.5 2004:10:100:1::5, priority: 0, weight: 66 l When SD-WAN load-balance mode is measured-volume-based. l Both members are under volume and still have room:

FGT # diagnose sys virtual-wan-link member

Member(1): interface: port13, gateway: 10.100.1.1 2004:10:100:1::1, priority: 0, weight: 33

Config volume ratio: 33, last reading: 8211734579B, volume room 33MB

Member(2): interface: port15, gateway: 10.100.1.5 2004:10:100:1::5, priority: 0, weight: 66

Config volume ratio: 66, last reading: 24548159B, volume room 66MB l Some members are overloaded and some still have room:

FGT # diagnose sys virtual-wan-link member

Member(1): interface: port1, gateway: 10.10.0.2, priority: 0, weight: 0

Config volume ratio: 10, last reading: 10297221000B, overload volume 1433MB

Member(2): interface: port2, gateway: 10.11.0.2, priority: 0, weight: 38 Config volume ratio: 50, last reading: 45944239916B, volume room 38MB l When SD-WAN load balance mode is usage-based/spillover. l When no spillover occurs:

FGT # diagnose sys virtual-wan-link member

Member(1): interface: port13, gateway: 10.100.1.1 2004:10:100:1::1, priority: 0, weight: 255

Egress-spillover-threshold: 400kbit/s, ingress-spillover-threshold: 300kbit/s Egress-overbps=0, ingress-overbps=0

Member(2): interface: port15, gateway: 10.100.1.5 2004:10:100:1::5, priority: 0, weight: 254

Egress-spillover-threshold: 0kbit/s, ingress-spillover-threshold: 0kbit/s Egress-overbps=0, ingress-overbps=0 l When member has reached limit and spillover occurs:

FGT # diagnose sys virtual-wan-link member

Member(1): interface: port13, gateway: 10.100.1.1 2004:10:100:1::1, priority: 0, weight: 255

Egress-spillover-threshold: 400kbit/s, ingress-spillover-threshold: 300kbit/s Egress-overbps=1, ingress-overbps=1

Member(2): interface: port15, gateway: 10.100.1.5 2004:10:100:1::5, priority: 0, weight: 254

Egress-spillover-threshold: 0kbit/s, ingress-spillover-threshold: 0kbit/s

Egress-overbps=0, ingress-overbps=0

• You can also use the diagnose netlink dstmac list command to check if you are over the limit.

FGT # diag netlink dstmac list port13

dev=port13 mac=08:5b:0e:ca:94:9d rx_tcp_mss=0 tx_tcp_mss=0 egress_overspill_ threshold=51200 egress_bytes=103710 egress_over_bps=1 ingress_overspill_threshold=38400 ingress_bytes=76816 ingress_over_bps=1 sampler_rate=0

To check SD-WAN service rules status:

• Manual mode service rules.

FGT # diagnose sys virtual-wan-link service

Service(1): Address Mode(IPV4) flags=0x0

TOS(0x0/0x0), Protocol(0: 1->65535), Mode(manual) Members:

1: Seq_num(2), alive, selected

Dst address: 10.100.21.0-10.100.21.255 l Auto mode service rules.

FGT # diagnose sys virtual-wan-link service

Service(1): Address Mode(IPV4) flags=0x0

TOS(0x0/0x0), Protocol(0: 1->65535), Mode(auto), link-cost-factor(latency), link-costthreshold(10), health-check(ping) Members:

1: Seq_num(2), alive, latency: 0.011

2: Seq_num(1), alive, latency: 0.018, selected Dst address: 10.100.21.0-10.100.21.255 l Priority mode service rules.

FGT # diagnose sys virtual-wan-link service

Service(1): Address Mode(IPV4) flags=0x0

TOS(0x0/0x0), Protocol(0: 1->65535), Mode(priority), link-cost-factor(latency), linkcost-threshold(10), health-check(ping) Members:

1: Seq_num(2), alive, latency: 0.011, selected

2: Seq_num(1), alive, latency: 0.017, selected Dst address: 10.100.21.0-10.100.21.255 l Load-balance mode service rules.

FGT # diagnose sys virtual-wan-link service

Service(1): Address Mode(IPV4) flags=0x0

TOS(0x0/0x0), Protocol(0: 1->65535), Mode(load-balance) Members:

1: Seq_num(1), alive, sla(0x1), num of pass(1), selected

2: Seq_num(2), alive, sla(0x1), num of pass(1), selected Dst address: 10.100.21.0-10.100.21.255 l SLA mode service rules.

FGT # diagnose sys virtual-wan-link service

Service(1): Address Mode(IPV4) flags=0x0 TOS(0x0/0x0), Protocol(0: 1->65535), Mode(sla) Members:

1: Seq_num(1), alive, sla(0x1), cfg_order(0), cost(0), selected

2: Seq_num(2), alive, sla(0x1), cfg_order(1), cost(0), selected Dst address: 10.100.21.0-10.100.21.255

To check interface logs from the past 15 minutes:

FGT (root) # diagnose sys virtual-wan-link intf-sla-log R150

Timestamp: Fri Apr 12 11:08:36 2019, used inbandwidth: 0bps, used outbandwidth: 0bps, used bibandwidth: 0bps, tx bytes: 860bytes, rx bytes: 1794bytes.

Timestamp: Fri Apr 12 11:08:46 2019, used inbandwidth: 1761bps, used outbandwidth: 1710bps, used bibandwidth: 3471bps, tx bytes: 2998bytes, rx bytes: 3996bytes.

Timestamp: Fri Apr 12 11:08:56 2019, used inbandwidth: 2452bps, used outbandwidth: 2566bps, used bibandwidth: 5018bps, tx bytes: 7275bytes, rx bytes: 7926bytes.

Timestamp: Fri Apr 12 11:09:06 2019, used inbandwidth: 2470bps, used outbandwidth: 3473bps, used bibandwidth: 5943bps, tx bytes: 13886bytes, rx bytes: 11059bytes.

Timestamp: Fri Apr 12 11:09:16 2019, used inbandwidth: 2433bps, used outbandwidth: 3417bps, used bibandwidth: 5850bps, tx bytes: 17946bytes, rx bytes: 13960bytes.

Timestamp: Fri Apr 12 11:09:26 2019, used inbandwidth: 2450bps, used outbandwidth: 3457bps, used bibandwidth: 5907bps, tx bytes: 22468bytes, rx bytes: 17107bytes.

To check SLA logs in the past 15 minutes:

FGT (root) # diagnose sys virtual-wan-link sla-log ping 1

Timestamp: Fri Apr 12 11:09:27 2019, vdom root, health-check ping, interface: R150, status:

up, latency: 0.014, jitter: 0.003, packet loss: 16.000%.

Timestamp: Fri Apr 12 11:09:28 2019, vdom root, health-check ping, interface: R150, status:

up, latency: 0.015, jitter: 0.003, packet loss: 15.000%.

Timestamp: Fri Apr 12 11:09:28 2019, vdom root, health-check ping, interface: R150, status:

up, latency: 0.014, jitter: 0.003, packet loss: 14.000%.

Timestamp: Fri Apr 12 11:09:29 2019, vdom root, health-check ping, interface: R150, status: up, latency: 0.015, jitter: 0.003, packet loss: 13.000%.

To check application control used in SD-WAN and the matching IP addresses:

FGT # diagnose sys virtual-wan-link internet-service-app-ctrl-list

Ctrl application(Microsoft.Authentication 41475):Internet Service ID(4294836224)

Protocol(6), Port(443)

Address(2): 104.42.72.21 131.253.61.96

Ctrl application(Microsoft.CDN 41470):Internet Service ID(4294836225)

Ctrl application(Microsoft.Lync 28554):Internet Service ID(4294836226)

Ctrl application(Microsoft.Office.365 33182):Internet Service ID(4294836227)

Ctrl application(Microsoft.Office.365.Portal 41468):Internet Service ID(4294836228)

Ctrl application(Microsoft.Office.Online 16177):Internet Service ID(4294836229)

Ctrl application(Microsoft.OneNote 40175):Internet Service ID(4294836230)

Ctrl application(Microsoft.Portal 41469):Internet Service ID(4294836231)

Protocol(6), Port(443)

Address(8): 23.58.134.172 131.253.33.200 23.58.135.29 204.79.197.200 64.4.54.254

23.59.156.241 13.77.170.218 13.107.22.200

Ctrl application(Microsoft.Sharepoint 16190):Internet Service ID(4294836232)

Ctrl application(Microsoft.Sway 41516):Internet Service ID(4294836233)

Ctrl application(Microsoft.Tenant.Namespace 41471):Internet Service ID(4294836234)

To check IPsec aggregate interface when SD-WAN uses the per-packet distribution feature:

# diagnose sys ipsec-aggregate list agg1 algo=L3 member=2 run_tally=2 members: vd1-p1 vd1-p2

To check BGP learned routes and determine if they are used in SD-WAN service:

FGT # get router info bgp network

FGT # get router info bgp network 10.100.11.0

BGP routing table entry for 10.100.10.0/24

Paths: (2 available, best 1, table Default-IP-Routing-Table) Advertised to non peer-group peers:

172.10.22.2

20

10.100.20.2 from 10.100.20.2 (6.6.6.6)

Origin EGP metric 200, localpref 100, weight 10000, valid, external, best

Community: 30:5

Last update: Wen Mar 20 18:45:17 2019

FGT # get router info route-map-address

Extend-tag: 15, interface(wan2:16)

10.100.11.0/255.255.255.0

FGT # diagnose firewall proute list list route policy info(vf=root):

id=4278779905 vwl_service=1(DataCenter) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sportt=0:65535 iif=0 dport=1-65535 oif=16 source wildcard(1): 0.0.0.0/0.0.0.0

destination wildcard(1): 10.100.11.0/255.255.255.0

 

Using BGP tags with SD-WAN rules

SD-WAN rules can use Border Gateway Protocol (BGP) learned routes as dynamic destinations.

In this example, a customer has two ISP connections, wan1 and wan2. wan1 is used primarily for direct access to internet applications, and wan2 is used primarily for traffic to the customer’s data center.

The customer could create an SD-WAN rule using the data center’s IP address range as the destination to force that traffic to use wan2, but the data center’s IP range is not static. Instead, a BGP tag can be used.

For this example, wan2’s BGP neighbor advertises the data center’s network range with a community number of 30:5.

This example assumes that SD-WAN is enable on the FortiGate, wan1 and wan2 are added as SD-WAN members, and a policy and static route have been created. See Creating the SD-WAN interface on page 105 for details.

To configure BGP tags with SD-WAN rules:

1. Configure the community list:

config router community-list edit “30:5” config rule edit 1 set action permit set match “30:5”

next

end

next

end

2. Configure the route map:

config router route-map edit “comm1” config rule edit 1 set match-community “30:5” set set-route-tag 15

next

end

next

end

3. Configure BGP:

config router bgp set as xxxxx set router-id xxxx config neighbor edit “10.100.20.2” set soft-reconfiguration enable set remote-as xxxxx set route-map-in “comm1”

next

end

end

4. Configure a firewall policy:

config firewall policy edit 1 set name “1” set srcintf “dmz”

set dstintf “”virtual-wan-link”” set srcaddr “all” set dstaddr “all” set action accept set schedule “always” set service “ALL”

set nat enable

next

end

5. Edit the SD-WAN configuration:

config system virtual-wan-link set status enable config members edit 1 set interface “wan1” set gateway 172.16.20.2

next edit 2 set interface “wan2”

next

end config service edit 1 set name “DataCenter” set mode manual set route-tag 15 set members 2

next

end

end

Troubleshooting

Check the network community

Use the get router info bgp network command to check the network community:

# get router info bgp network

BGP table version is 5, local router ID is 1.1.1.1

Status codes: s suppressed, d damped, h history, * valid, > best, i – internal, S Stale

Origin codes: i – IGP, e – EGP, ? – incomplete

Network Next Hop Metric LocPrf Weight RouteTag Path *> 0.0.0.0/0 10.100.1.5 32768 0 ?

*> 1.1.1.1/32 0.0.0.0 32768 0 ?

*> 10.1.100.0/24 172.16.203.2 32768 0 ?

*> 10.100.1.0/30 0.0.0.0 32768 0 ?

*> 10.100.1.4/30 0.0.0.0 32768 0 ?

*> 10.100.1.248/29 0.0.0.0 32768 0 ? *> 10.100.10.0/24 10.100.1.5 202 10000 15 20 e *> 172.16.200.0/24 0.0.0.0 32768 0 ?

*> 172.16.200.200/32

0.0.0.0 32768 0 ?

*> 172.16.201.0/24 172.16.200.4 32768 0 ? *> 172.16.203.0/24 0.0.0.0 32768 0 ?

*> 172.16.204.0/24 172.16.200.4 32768 0 ?

*> 172.16.205.0/24 0.0.0.0 32768 0 ?

*> 172.16.206.0/24 0.0.0.0 32768 0 ?

*> 172.16.207.1/32 0.0.0.0 32768 0 ?

*> 172.16.207.2/32 0.0.0.0 32768 0 ?

*> 172.16.212.1/32 0.0.0.0 32768 0 ?

*> 172.16.212.2/32 0.0.0.0 32768 0 ?

*> 172.17.200.200/32

0.0.0.0 32768 0 ? *> 172.27.1.0/24 0.0.0.0 32768 0 ?

*> 172.27.2.0/24 0.0.0.0 32768 0 ?

*> 172.27.5.0/24 0.0.0.0 32768 0 ?

*> 172.27.6.0/24 0.0.0.0 32768 0 ?

*> 172.27.7.0/24 0.0.0.0 32768 0 ?

*> 172.27.8.0/24 0.0.0.0 32768 0 ?

*> 172.29.1.0/24 0.0.0.0 32768 0 ?

*> 172.29.2.0/24 0.0.0.0 32768 0 ? *> 192.168.1.0 0.0.0.0 32768 0 ?

Total number of prefixes 28

# get router info bgp network 10.100.11.0

BGP routing table entry for 10.100.10.0/24

Paths: (2 available, best 1, table Default-IP-Routing-Table) Advertised to non peer-group peers:

172.10.22.2

20

10.100.20.2 from 10.100.20.2 (6.6.6.6)

Origin EGP metric 200, localpref 100, weight 10000, valid, external, best

Community: 30:5 <<<<===========================

Last update: Wen Mar 20 18:45:17 2019

Check dynamic BGP addresses

Use the get router info route-map-address command to check dynamic BGP addresses:

# get router info route-map-address

Extend-tag: 15, interface(wan2:16)

10.100.11.0/255.255.255.0

Check dynamic BGP addresses used in policy routes

Use the diagnose firewall proute list command to check dynamic BGP addresses used in policy routes:

# diagnose firewall proute list list route policy info(vf=root):

id=4278779905 vwl_service=1(DataCenter) flags=0x0 tos=0x00 tos_mask=0x00 protocol=0 sportt=0:65535 iif=0 dport=1-65535 oif=16 source wildcard(1): 0.0.0.0/0.0.0.0 destination wildcard(1): 10.100.11.0/255.255.255.0

Forward error correction on VPN overlay networks

This topic shows an SD-WAN with forward error correction (FEC) on VPN overlay networks. FEC can be used to lower packet loss ratio by consuming more bandwidth. It uses six parameters in IPsec phase1/phase1-interface setting.

l fec-ingress. Disabled by default. l fec-egress. Disabled by default. l fec-base. <1-100>. Default=20. l fec-redundant. <1-100>. Default=10. l fec-send-timeout. <1-1000>. Default=8. l fec-receive-timeout.<1-10000>. Default=5000.

For example, a customer has tow ISP connections, wan1 and wan2. Using these two connections, create two IPsec VPN interfaces as SD-WAN members. Configure FEC on each VPN interface to lower packet loss ratio by retransmitting the packets using its backend algorithm.

Sample topology

To configure IPsec VPN:

config vpn ipsec phase1-interface edit “vd1-p1” set interface “wan1” set peertype any set net-device disable set proposal aes256-sha256 set dhgrp 14 set remote-gw 172.16.201.2 set psksecret ftnt1234 set fec-egress enable set fec-send-timeout 8 set fec-base 20 set fec-redundant 10 set fec-ingress enable set fec-receive-timeout 5000

next edit “vd1-p2” set interface “wan2” set peertype any set net-device disable set proposal aes256-sha256 set dhgrp 14 set remote-gw 172.16.202.2

set psksecret ftnt1234 set fec-egress enable set fec-send-timeout 8 set fec-base 20 set fec-redundant 10 set fec-ingress enable set fec-receive-timeout 5000

next

end

config vpn ipsec phase2-interface edit “vd1-p1”

set phase1name “vd1-p1”

next edit “vd1-p2”

set phase1name “vd1-p2”

next

end

To configure the interface:

config system interface

edit “vd1-p1”

set ip 172.16.211.1 255.255.255.255 set remote-ip 172.16.211.2 255.255.255.255

next edit “vd1-p2”

set ip 172.16.212.1 255.255.255.255 set remote-ip 172.16.212.2 255.255.255.255

next

end

To configure the firewall policy:

config firewall policy edit 1

set name “1” set srcintf “dmz” set dstintf “”virtual-wan-link”” set srcaddr “all” set dstaddr “all” set action accept set schedule “always” set service “ALL” set nat enable

next

end To configure SD-WAN:

config system virtual-wan-link

set status enable config members

edit 1

set interface “vd1-p1” set gateway 172.16.211.2 next

edit 1 set interface “vd2-p2” set gateway 172.16.212.2

next

end

end

To use the diagnose command to check VPN FEC status:

# diagnose vpn tunnel list

list all ipsec tunnel in vd 0

—————————————————–name=vd1 ver=1 serial=1 172.16.200.1:0->172.16.200.2:0

bound_if=11 lgwy=static/1 tun=intf/0 mode=auto/1 encap=none/3600 options[0e10]=create_dev frag-rfc fec-egress fec-ingress accept_traffic=1

proxyid_num=1 child_num=0 refcnt=11 ilast=8 olast=8 ad=/0 stat: rxp=0 txp=0 rxb=0 txb=0

dpd: mode=on-demand on=1 idle=20000ms retry=3 count=0 seqno=0 natt: mode=none draft=0 interval=0 remote_port=0 fec-egress: base=20 redundant=10 remote_port=50000      <<<<<<<<<<<<<<<<<<<<<< fec-ingress: base=20 redundant=10    <<<<<<<<<<<<<<<<<<<<<< proxyid=demo proto=0 sa=1 ref=2 serial=1

src: 0:10.1.100.0/255.255.255.0:0 dst: 0:173.1.1.0/255.255.255.0:0

SA: ref=3 options=10226 type=00 soft=0 mtu=1390 expire=42897/0B replaywin=2048 seqno=1 esn=0 replaywin_lastseq=00000000 itn=0 qat=0

life: type=01 bytes=0/0 timeout=42899/43200

dec: spi=181f4f81 esp=aes key=16 6e8fedf2a77691ffdbf3270484cb2555 ah=sha1 key=20 f92bcf841239d15d30b36b695f78eaef3fad05c4

enc: spi=0ce10190 esp=aes key=16 2d684fb19cbae533249c8b5683937329 ah=sha1 key=20 ba7333f89cd34cf75966bd9ffa72030115919213

dec:pkts/bytes=0/0, enc:pkts/bytes=0/0