FortiWLC – Alarms

Alarms

When alarms are generated, the user has the option to either Acknowledge or Clear them by simply checking the box alongside the desired alarm and clicking the appropriate button towards the bottom of the window.

  • Clear—Moves the alarm from the Active Alarms table into the Alarm History table.
  • Acknowledge—Marks the alarm as acknowledged in the UserAcknowledged column.

As seen in the figure above, the Active Alarms table provides several columns, as described below.

489

TABLE 35: Active Alarm Columns

Column Description
Alarm Name The name of the alarm triggered.
Severity The severity level; can range from Information, Minor, Major, Critical.
Source The type of device that triggered the alarm (controller, AP).
FDN The name of the device that triggered the alarm.
Raised At The date and time at which the alarm was triggered.
Detail Detailed information regarding the alarm, including identifying device details.
UserAcknowledged Indicates whether the alarm has been flagged as Acknowledged.
Modifying Alarm Definitions

While FortiWLC (SD) provides a list of pre-configured alarms, users can also customize the alarms to the needs of their environment via the Alarms > Definition tab.

Figure 86: Alarm Definitions

As shown above, each alarm has a predetermined severity level, trigger condition, and threshold, but these values can be modified by clicking the small pencil icon next to the desired alarm. This will pop up the Alarm Configuration window, as seen in Figure 87 on page 491.

 

Figure 87: Editing an Alarm

Use the drop-downs provided in the window to tailor the alarm to the deployment’s needs and click Save when finished. If desired, the user can click Reload Default to reset the alarm’s configuration to its original values.

The Threshold field’s units will vary depending on the alarm selected—for example, when modifying AP Memory Usage High, the Threshold is measured in percentage of overall system memory (and defaults to 70%). However, in an alarm such as Link Down, no threshold is needed at all, as it is a binary alarm (i.e., it is triggered when a link to an AP goes down—there is no percentage involved).

List of Alarms
No. Alarm Severity Source Explanation
1. Alarm link up information all controller models Physical link on controller is up.
2. Alarm link down critical all controller models Physical link on the controller is down; check the connection.
3. Alarm auth fail information controller models An administrator failed to log in to the GUI due to an authentication failure.
4. AP down critical all AP models An AP is down. Possible reasons for this are an AP reboot, an AP crash, or an Ethernet cable from the controller may be down. Also the AP may have connected to another controller.
5. Radio Failure critical all AP models An alarm is generated when the Radio fails to turn operational during Initial bootup. This is occurred due to some Hardware issue on the AP Radio.
6. Rogue AP detected critical all controller models A rogue AP has been detected on the network.

The message looks something like this: Rogue

AP Detected               Critical  06/04/2010

10:04:51  CONTROLLER (1:24194)  ROGUE AP DETECTED. Station mac=0c:60:76:2d:fe:d9 bss=00:02:6f:3a:fd:89 by AP Ben-Cubei (18)

See the chapter Rogue AP Detection and Mitigation.

7. AP software version mismatch critical all AP models The software version on the AP does not match the version on the controller. Automatic AP upgrade must have been turned off. Update the AP from the controller with either the CLI command upgrade ap same <ap id> force or upgrade ap same all force. You can also turn automatic upgrade back on by with the CLI command autoap-upgrade enable.
8. AP init failure major all AP models AP initialization failed.

 

No. Alarm Severity Source Explanation
9. Software license expired major all controller

models

Controller software license has expired. To obtain additional licenses, see www.merunetworks.com/ license.
10. 802.1X auth failure major, minor, information all controller

models

RADIUS server authentication failed. To find out why, look at the RADIUS server log for the error message and also check the station log. If this happens only occasionally, you can ignore it. However, if this message appears repeatedly, the authentication failures could prevent a station from entering the network. In this case, check the RADIUS server to make sure the client and server have the same credentials.
11. MIC failure AP major all controller models The Michael MIC Authenticator Tx/Rx Keys provided in the Group Key Handshake are only used if the network is using TKIP to encrypt the data. A failure of the Michael MIC in a packet usually indicates that the WPA WPSK password is wrong.
12. MIC countermeasure activation major all controller

models

Two consecutive MIC failures have occurred (see above).
13. RADIUS Server Switchover major all controller

models

A switchover from the Primary  Authentication

RADIUS Server to the Secondary Authentication RADIUS Server occurred. When this message occurs, the Primary RADIUS server is configured but not reachable and the Secondary RADIUS server is both configured and reachable.

This message is generated only for 802.1x switchover, not for Captive Portal switchover.

An example looks like this:

RADIUS Server Switchover        Major     06/07/ 2010 14:09:57  RADIUS Server switches over from Primary <172.18.1.7> to Secondary

<172.18.1.3> for Profile <wpa>

 

No. Alarm Severity Source Explanation
14. RADIUS Server Switchover Failed major all controller

models

A switchover from the Primary  Authentication

RADIUS Server to the Secondary Authentication RADIUS Server  failed because the secondary server is not configured. When this message occurs, the Primary RADIUS server is configured but not reachable and the Secondary RADIUS server is not configured.

This message is generated only for 802.1x switchover failure, not for Captive Portal switchover failure.

An example looks like this:

RADIUS Server Switchover Failed Major     06/

07/2010 14:02:47  Primary RADIUS Server

<172.18.1.7> failed. No valid Secondary RADIUS

Server present. Switchover FAILED for Profile

<wpa> Alarms Table(1 entry)

15. Restore Primary RADIUS Server major all controller

models

A switchover from the Secondary Authentication

RADIUS Server to the Primary Authentication RADIUS Server occurred. This alarm was generated while doing RADIUS fall back to the primary server after 15 minutes.

This message is generated only for 802.1x primary RADIUS restore, not for Captive Portal restore.

An example looks like this:

Restore Primary RADIUS Server   Major     06/07/ 2010 15:54:10  Security Profile <wpa> restored back to the Primary RADIUS server <172.18.1.7>

 

No. Alarm Severity Source Explanation
16. Acct RADIUS server switchover major all controller

models

A switchover from either Accounting RADIUS Server (primary or secondary) to the other one occurred. This message is generated only for 802.1x switchover, not for Captive Portal switchover.

An example when the primary to secondary switch occurred looks like this:

Accounting RADIUS Server Switch Major     06/ 07/2010 14:39:00  Accounting RADIUS Server switches over from Primary <172.18.1.7> to Secondary <172.18.1.3> for Profile <wpa>

17. Acct RADIUS server switchover failed major all controller models An attempted switchover from one Accounting RADIUS Server to the other server failed.When this message occurs, the Primary Accounting RADIUS server is configured but not reachable and the Secondary Accounting RADIUS server is not configured.

This message is generated only for 802.1x switchover failure, not for Captive Portal switchover fail lure.

An example looks like this:

Accounting RADIUS Server Switch Major     06/

07/2010 14:22:26  Primary Accounting RADIUS

Server <172.18.1.7> failed. No valid Secondary

Accounting RADIUS Server present. Switchover

FAILED for Profile <wpa>

18. Master down critical all controller models N+1 Master controller is down and no longer in control; the slave controller will now take over.
19. Master up critical all controller models N+1 Master controller is up and running; this controller will now take control away from the slave controller.
20. CAC limit reached major all controller models Admission control in ATM networks is known as Connection Admission Control (CAC) – this process determines which traffic is admitted into a network. If this message occurs, the maximum amount of traffic is now occurring on the network and no more can be added.

 

FortiWLC – Fault Management

Fault Management

Alarm and event information can be found on the Monitor > Fault Management page. By default, the Active Alarms table is displayed, which indicates any alarms that have been recently triggered.

Figure 85: Fault Management Table

The Fault Management page provides information regarding two major types of events in FortiWLC (SD): Alarms and Events. Refer to their respective sections below for additional details.

FortiWLC – Troubleshooting

Troubleshooting

  • Where Do I Start?
  • Error Messages
  • System Logs
  • System Diagnostics
  • Capturing Packets
  • FTP Error Codes

Where Do I Start?

We recommend that you start troubleshooting as follows:

Web UI or CLI? Problem Involves? Strategy
Web UI stations View station log history by clicking Monitor > Diagnostics > Station
Web UI radios View radio log history by clicking Monitor > Diagnostics > Radio

Where Do I Start?

Web UI or CLI? Problem Involves? Strategy
CLI stations View station-log history with one of these commands:

station-log show-mac=<affected MAC address> station-log show (if the MAC is not known)

If the problem is reproducible/occurring continually, log your terminal session, enter the station-log interface and add the affected MAC address using the command station add <MAC>. If you DON’T know the MAC address, enter event all all to capture all events for all MAC addresses.

CLI controller View controller-log history with the command diagnostics-controller

If the problem is reproducible/occurring continually, log your terminal session, enter the station-log interface with the command station-log, and add the affected MAC address using the command station add <MAC>. If you DON’T know the MAC address, type event all all to capture all events for all MAC addresses.

Error Messages

The following are common error messages that may occur either at the controller or at an AP.

Error Messages

Message Text Explanation
[07/20 13:02:11.122]

1m[35m**Warning**[0m

WMAC: Wif(0):SetTsf()

TSF[00000000:000006e3] ->

[00000033:77491cfd]thr[0000

0000:03938700]

May be observed on the AP command line or in trace log output from an AP after a full diagnostics gather.

The SetTsf() messages indicate that the AP has adjusted its TSF (TSF stands for Time Synchronization Function and is really the AP’s clock) forward by more than a certain threshold (the threshold is 5 seconds). The specific case above indicates that the AP has just booted up and adjusted its TSF value to its neighboring AP’s TSF value.

You can tell that the AP just booted because its current TSF is a low value (i.e. 6e3 microseconds). During initialization, the AP will synchronize its TSF to the TSF of its neighbors whenever the neighbors support a BSSID in common with this AP. That is a requirement to support Virtual Cell.

[07/31 14:01:33.506]

*****ERROR***** QOS: FlowMgr failed while processing flow request, reason= 5, srcMac[00:23:33:41:ed:27], dstMac[00:00:00:00:00:00].

May be observed in the controller’s CLI interface.

This error occurs when there is an attempt to either set up or remove an AP flow on a station that has started a phone call. “reason=5” means the cited station is not assigned to the AP where the attempt to set up/ remove the flow was made.

The presumed impact is that the stations (presumably phones) get lower than normal call quality since there are no QoS flows established on behalf of the MAC address.

Received non-local pkt on AP! This message may be observed on the serial console of a controller or in the dmesg.txt output included with a controller’s diagnostics. This message indicates that a Ethernet type 0x4001 or UDP port 5000 packet (L2 and L3 COMM respectively) was received by the controller’s Ethernet, but was not actually destined for the controller’s MAC or IP address.

FortiWLC – Configuring SNMP

Configuring SNMP

The SNMP Agent offers the network administrator performance management and fault management features, with the collection of statistics as well as notification of unusual events via traps.

This information applies to all controller models and the following AP series:

  • AP400
  • AP1000

The Wireless LAN System SNMP Agent can inter-operate with 3rd party Network Management Systems (NMS) such as HP OpenView, and present alarm and trap information to configured management stations.

Fortinet FortiWLC (SD) supports several versions of SNMP protocols. On Fortinet software, all versions (SNMPv1, SNMPv2c, and SNMPv3) of the Internet-Standard Management Framework share the same basic structure and components. Furthermore, all versions of the specifications of the Internet-Standard Management Framework follow the same architecture.

No Feature RFCs
1 SNMPv1 RFC-1155, RFC-1157
2 SNMPv2c RFC-1901, RFC-1905, RFC-1906
3 SNMPv3 RFC-1905, RFC-1906, RFC-2571, RFC-2574, RFC-2575
4 MIB-II RFC-1213
5 Fortinet Private MIB Fortinet Wireless LAN Proprietary MIB

Note that Fortinet FortiWLC (SD) doesn’t support write operation through SNMP. You need to provision any required configuration through the CLI or Web UI.

445

Features

The following protocols are supported for the read function only (not write):

  • RFC-1214
  • SNMPv1/v2c/v3
  • Fortinet WLAN systems

SNMP Architecture

Figure 77: SNMP Network Management Architecture

The Wireless LAN System SNMP network management architecture follows the client-server architecture as illustrated in the diagram. The SNMP model of a managed network consists of the following elements:

  • One or more managed nodes. In the illustration, the controller is among the managed nodes in the SNMP-based managed network. The SNMP agent is resident in the managed node. It collects statistics from the access points and combines them before sending them to the SNMP manager via MIB variables. Configuration information set via SNMP is also propagated to the access points by the SNMP agent.
  • At least one management station containing management applications.
  • Management information in each managed node, that describes the configuration, state, statistics, and that controls the actions of the managed node.
  • A management protocol, which the managers and agents use to exchange management messages. In an SNMP managed network, the management protocol is SNMP (Simple Network Management Protocol). This defines the format and meaning of the messages

Features

 

communicated between the managers and agents. Fortinet Wireless LAN System provides support for traps, gets, and MIB walk functions only.

Neither read nor write privilege gives the SNMP manager access to the community strings. The controller can have an unlimited number of read and read/write community strings.

MIB Tables

The MIB tables supported by the Wireless LAN System SNMP implementation can be downloaded from the controller and then copied to an off-box location. The MIB Tables are also available on the Fortinet web site. A summary of the Wireless LAN System MIB Enterprise tables are:

mwstatistics.1 mwGlobalStatistics.1 * mwIf80211StatsTable.1 mwGlobalStatistics.2 * mwIfStatsTable.1 mwIfStatsEntry.1 mwGlobalStatistics.6 * mwStationStatsTable.1 mwStationStatsEntry.1 mwGlobalStatistics.7 * mwApStationStatsTable.1 mwApStationStatsEntry.1 mwGlobalStatistics.8 * mwCacApStatsTable.1 mwCacApStatsEntry.1 mwGlobalStatistics.9 * mwCacBssStatsTable.1 mwCacBssStatsEntry.1 mwStatistics.2 * mwTop10Statistics.1 mwTop10ApStationProblemTable.1 mwTop10ApStationProblemEntry.1 mwTop10Statistics.2 mwTop10ApStationRxtxTable.1 mwTop10ApStationRxtxEntry.1 mwTop10Statistics.3 mwTop10ApProblemTable.1 mwTop10ApProblemEntry.1 mwGlobalStatistics.4 mwTop10ApRxtxTable.1 mwTop10ApRxtxEntry.1 mwStatistics.1 mwPhoneTable.1 mwPhoneEntry.1 mwStatistics.2 mwPhoneCallTable.1 mwPhoneCallEntry.1 mwStatistics.3 mwStatusTable.1 mwStatusEntry.1

Global statistics use 64 bit counters in FortiWLC (SD) 4.0 and later

SNMP Architecture

Download the MIB Tables for Management Applications

If you are using a third-party SNMP-based Network Manager program, you will need to integrate the Fortinet Wireless LAN System proprietary MIB tables that allow the manager program to manage controllers and APs. The MIB tables are available in a compressed (zipped) file that can be copied from the controller to an off-box location.

To download the enterprise MIB Tables, contained in the file mibs.tar.gz, located in the images directory, use the following CLI commands:

controller# cd image controller# copy mibs.tar.gz off‐box_location

To download the enterprise MIB Tables using the Web UI, follow these steps:

  1. Open a Web Browser(IE or Firefox), enter the system IP address (example: https:// 172.29.0.133) and then enter a user name and password (factory default user name/ password is admin/admin).
  2. Click Configuration > Wired > SNMP > Download MIB Files.
  3. When the download is done, you will see the file listed in the Downloads list.
  4. Save the file mibs(x).tar.gz.

FortiGate 7060E Running FortiOS 5.4.9

So, just a heads up for those that are running the 7060E series. The new code (that isn’t available unless you ask support for it) is 5.4.9 build 8110 and it is a life saver. Fixes a bunch of crazy bugs and nuances that were really making me bash my head into the wall when trying to manage this cluster I have.

Hop on as soon as you can and enjoy the easy life!

 

**Edited out the obvious typos and spelling mistakes. I need you guys to stay on me when I’ve been posting after taking my pain meds!