Troubleshooting static routing

When there are problems with your network that you believe to be static routing related, there are a few basic tools available to locate the problem.

These tools include:

• Ping
• Traceroute
• Examine routing table contents

Ping

Beyond the basic connectivity information, ping can tell you the amount of packet loss (if any), how long it takes the packet to make the round trip, and the variation in that time from packet to packet.

If there is no packet loss detected, your basic network connectivity is OK. If there is some packet loss detected, you should investigate:

• Possible ECMP, split horizon, network loops
• Cabling to ensure no loose connections

If there is total packet loss, you should investigate:

• Hardware – ensure cabling is correct, and all equipment between the two locations is accounted for
• Addresses and routes – ensure all IP addresses and routing information along the route is configured as expected
• Firewalls – ensure all firewalls are set to allow PING to pass through

To ping from a Windows PC

1. Go to a DOS prompt. Typically you go to Start > Run, enter cmd and select OK.

2. Enter ping 11.101.100 to ping the default internal interface of the FortiGate unit with four packets.

 

To ping from an Apple computer

1. Open the Terminal.

2. Enter ping 11.101.100.

3. If the ping fails, it will stop after a set number of attempts. If it succeeds, it will continue to ping repeatedly. Press

Control+C to end the attempt and see gathered data.

To ping from a Linux PC

1. Go to a command line prompt.

2. Enter “/bin/etc/ping 11.101.101”.

 

Traceroute

Where ping will only tell you if it reached its destination and came back successfully, traceroute will show each step of its journey to its destination and how long each step takes. If ping finds an outage between two points, traceroute can be used to locate exactly where the problem is.

 

To use traceroute on an Windows PC

1. Go to a DOS prompt. Typically you go to Start > Run, enter “cmd” and select OK.

2. Enter “tracert com” to trace the route from the PC to the Fortinet website.

To use traceroute from an Apple computer

1. Open the Terminal.

2. Enter traceroute com.

3. The terminal will list the number of steps made. Upon reaching the destination, it will list three asterisks per line.

Press Control+C to end the attempt.

 

To use traceroute on a Linux PC

1. Go to a command line prompt.

2. Enter “/bin/etc/traceroute com”.

The Linux traceroute output is very similar to the MS Windows traceroute output.

 

Examine routing table contents

The first place to look for information is the routing table.

The routing table is where all the currently used routes are stored for both static and dynamic protocols. If a route is in the routing table, it saves the time and resources of a lookup. If a route isn’t used for a while and a new route needs to be added, the oldest least used route is bumped if the routing table is full. This ensures the most recently used routes stay in the table. Note that if your FortiGate unit is in Transparent mode, you are unable to perform this step.

If the FortiGate is running in NAT mode, verify that all desired routes are in the routing table: local subnets, default routes, specific static routes, and dynamic routing protocols.

To check the routing table in the web-based manager, use the Routing Monitor — go to Router > Monitor > Routing Monitor. In the CLI, use the command get router info routing-table all.

Route priority

After the FortiGate unit selects static routes for the forwarding table based on their administrative distances, the priority field of those routes determines routing preference. Priority is a Fortinet value that may or may not be present in other brands of routers.

You can configure the priority field through the CLI or the web-based manager. Priority values can range from 0 to

4 294 967 295. The route with the lowest value in the priority field is considered the best route. It is also the primary route.

To change the priority of a route – web-based manager

1. Go to Router > Static > Static Routes.

2. Select the route entry, and select Edit.

3. Select Advanced.

4. Enter the Priority value.

5. Select OK.

 

To change the priority of a route – CLI

The following command changes the priority to 5 for a route to the address 10.10.10.1 on the port1

interface.

config router static edit 1

set device port1

set gateway 10.10.10.10 set dst 10.10.10.1

set priority 5 end

If there are other routes set to priority 10, the route set to priority 5 will be preferred. If there are routes set to priorities less than 5, those other routes will be preferred instead.

In summary, because you can use the CLI to specify which sequence numbers or priority field settings to use when defining static routes, you can prioritize routes to the same destination according to their priority field settings. For a static route to be the preferred route, you must create the route using the config router static CLI command and specify a low priority for the route. If two routes have the same administrative distance and the same priority, then they are equal cost multipath (ECMP) routes.

Since this means there is more than one route to the same destination, it can be confusing which route or routes to install and use. However, if you have enabled load balancing with ECMP routes, then different sessions will resolve this problem by using different routes to the same address.

Multipath routing and determining the best route

Multipath routing occurs when more than one entry to the same destination is present in the routing table. When multipath routing happens, the FortiGate unit may have several possible destinations for an incoming packet, forcing the FortiGate unit to decide which next-hop is the best one.

It should be noted that some IP addresses will be rejected by routing protocols. These are called Martian addresses. They are typically IP addresses that are invalid and not routable because they have been assigned an address by a misconfigured system, or are spoofed addresses.

Two methods to manually resolve multiple routes to the same destination are to lower the administrative distance of one route or to set the priority of both routes. For the FortiGate unit to select a primary (preferred) route, manually lower the administrative distance associated with one of the possible routes. Setting the priority on the routes is a FortiGate unit feature and may not be supported by non-Fortinet routers.

Administrative distance is based on the expected reliability of a given route. It is determined through a combination of the number of hops from the source and the protocol used. A hop is when traffic moves from one router to the next. More hops from the source means more possible points of failure. The administrative distance can be from 1 to 255, with lower numbers being preferred. A distance of 255 is seen as infinite and will not be installed in the routing table.

Here is an example to illustrate how administration distance works — if there are two possible routes traffic can take between two destinations with administration distances of 5 (always up) and 31 (sometimes not available), the traffic will use the route with an administrative distance of 5. If for some reasons the preferred route (admin distance of 5) is not available, the other route will be used as a backup.

Different routing protocols have different default administrative distances. These different administrative distances are based on a number of factors of each protocol such as reliability, speed, and so on. The default administrative distances for any of these routing protocols are configurable.

 

Default administrative distances for routing protocols and connections

 

Routing protocol	Default administrative distance
Direct physical connection	1
Static	10
EBGP	20
OSPF	110
IS-IS	115
RIP	120
IBGP	200

Another method to determine the best route is to manually change the priority of both routes in question. If the next-hop administrative distances of two routes on the FortiGate unit are equal, it may not be clear which route the packet will take. Manually configuring the priority for each of those routes will make it clear which next-hop will be used in the case of a tie. The priority for a route be set in the CLI, or when editing a specific static route, as described in the next section. Lower priority routes are preferred. Priority is a Fortinet value that may or may not be present in other brands of routers.

All entries in the routing table are associated with an administrative distance. If the routing table contains several entries that point to the same destination (the entries may have different gateways or interface associations), the FortiGate unit compares the administrative distances of those entries first, selects the entries having the lowest distances, and installs them as routes in the FortiGate unit forwarding table. As a result, the FortiGate unit forwarding table contains only those routes having the lowest distances to every possible destination. While only static routing uses administrative distance as its routing metric, other routing protocols such as RIP can use metrics that are similar to administrative distance.

 

Static routing security

Securing the information on your company network is a top priority for network administrators. Security is also required as the routing protocols used are internationally known standards that typically provide little or no inherent security by themselves.

The two reasons for securing your network are the sensitive and proprietary information on your network, and also your external bandwidth. Hackers not only can steal your information, but they can also steal your bandwidth. Routing is a good low level way to secure your network, even before UTM features are applied.

Routing provides security to your network in a number of ways including obscuring internal network addresses with NAT and blackhole routing, using RPF to validate traffic sources, and maintaining an access control list (ACL) to limit access to the network.

This section includes:

• Network Address Translation (NAT)
• Access Control List (ACL)
• Blackhole Route
• Reverse path lookup

Network Address Translation (NAT)

Network address translation (NAT) is a method of changing the address from which traffic appears to originate. This practice is used to hide the IP address on a company’s internal networks, and helps prevent malicious attacks that use those specific addresses.

This is accomplished by the router connected to that local network changing all the IP addresses to its externally connected IP address before sending the traffic out to the other networks, such as the Internet. Incoming traffic uses the established sessions to determine which traffic goes to which internal IP address. This also has the benefit of requiring only the router to be very secure against external attacks, instead of the whole internal network as would be the case without NAT. Securing one computer is much cheaper and easier to maintain.

1. Configuring NAT on your FortiGate unit includes the following steps.

2. Configure your internal network. For example use the 11.101.0 subnet.

3. Connect your internal subnet to an interface on your FortiGate unit. For example use port1.

4. Connect your external connection, for example an ISP gateway of 20.120.2, to another interface on your

Fortigate unit, for example port2.

Configure security policies to allow traffic between port1 and port2 on your FortiGate unit, ensuring that the NAT

feature is enabled.

The above steps show that traffic from your internal network will originate on the 10.11.101.0 subnet and pass on to the 172.20.120.0 network. The FortiGate unit moves the traffic to the proper subnet. In doing that, the traffic appears to originate from the FortiGate unit interface on that subnet — it does not appear to originate from where it actually came from.

NAT “hides” the internal network from the external network. This provides security through obscurity. If a hacker tries to directly access your network, they will find the Fortigate unit, but will not know about your internal network. The hacker would have to get past the security-hardened FortiGate unit to gain access to your internal network. NAT will not prevent hacking attempts that piggy back on valid connections between the internal network and the outside world. However other UTM security measures can deal with these attempts.

Another security aspect of NAT is that many programs and services have problems with NAT. Consider if someone on the Internet tries to initiate a chat with someone on the internal network. The outsider only can access the FortiGate unit’s external interface unless the security policy allows the traffic through to the internal network. If allowed in, the proper internal user would respond to the chat. However if its not allowed, the request to chat will be refused or time-out. This is accomplished in the security policy by allowing or denying different protocols.

 

Access Control List (ACL)

An access control list (ACL) is a table of addresses that have permission to send and receive data over a router’s interface or interfaces. The router maintains an ACL, and when traffic comes in on a particular interface it is buffered, while the router looks up in the ACL if that traffic is allowed over that port or not. If it is allowed on that incoming interface, then the next step is to check the ACL for the destination interface. If the traffic passes that check as well the buffered traffic is delivered to its accentuation. If either of those steps fail the ACL check, the traffic is dropped and an error message may be sent to the sender. The ACL ensures that traffic follows expected paths, and any unexpected traffic is not delivered. This stops many network attacks. However, to be effective the ACL must be kept up to date —when employees or computers are removed from the internal network their IP addresses must also be removed from the ACL. For more information on the ACL, see the router chapter of the FortiGate CLI Reference.

 

Blackhole Route

A blackhole route is a route that drops all traffic sent to it. It is very much like /dev/null in Linux programming. Blackhole routes are used to dispose of packets instead of responding to suspicious inquiries. This provides added security since the originator will not discover any information from the target network.

Blackhole routes can also limit traffic on a subnet. If some subnet addresses are not in use, traffic to those addresses (traffic which may be valid or malicious) can be directed to a blackhole for added security and to reduce traffic on the subnet.

The loopback interface, a virtual interface that does not forward traffic, was added to enable easier configuration of blackhole routing. Similar to a normal interface, this loopback interface has fewer parameters to configure, and all traffic sent to it stops there. Since it cannot have hardware connection or link status problems, it is always available, making it useful for other dynamic routing roles. Once configured, you can use a loopback interface in security policies, routing, and other places that refer to interfaces. You configure this feature only from the CLI. For more information, see the system chapter of the FortiGate CLI Reference.

 

Reverse path lookup

Whenever a packet arrives at one of the FortiGate unit’s interfaces, the unit determines whether the packet was received on a legitimate interface by doing a reverse lookup using the source IP address in the packet header. This is also called anti-spoofing. If the FortiGate unit cannot communicate with the computer at the source IP address through the interface on which the packet was received, the FortiGate unit drops the packet as it is likely a hacking attempt.

If the destination address can be matched to a local address (and the local configuration permits delivery), the FortiGate unit delivers the packet to the local network. If the packet is destined for another network, the Fortigate unit forwards the packet to a next-hop router according to a policy route and the information stored in the FortiGate forwarding table.

Building the routing table

In the factory default configuration, the FortiGate unit routing table contains a single static default route. You can add routing information to the routing table by defining additional static routes.

It is possible that the routing table is faced with several different routes to the same destination—the IP addresses of the next-hop router specified in those routes or the FortiGate interfaces associated with those routes may vary. In this situation, the “best” route is selected from the table.

The FortiGate unit selects the “best” route for a packet by evaluating the information in the routing table. The “best” route to a destination is typically associated with the shortest distance between the FortiGate unit and the closest gateway, also known as a next-hop router. In some cases, the next best route may be selected if the best route is unavailable.

The FortiGate unit installs the best available routes in the unit’s forwarding table, which is a subset of the unit’s routing table. Packets are forwarded according to the information in the forwarding table.

Viewing the routing table in the CLI

In the CLI, you can easily view the static routing table just as in the web-based manager or you can view the full routing table.

When viewing the list of static routes using the CLI command get route static, it is the configured static routes that are displayed. When viewing the routing table using the CLI command get router info routing-table all, it is the entire routing table information that is displayed including configured and learned routes of all types. The two are different information in different formats.

If VDOMs are enabled on your FortiGate unit, all routing related CLI commands must be performed within a VDOM and not in the global context.

 

To view the routing table

# get router info routing-table all

Codes: K – kernel, C – connected, S – static, R – RIP, B – BGP O – OSPF, IA – OSPF inter area

N1 – OSPF NSSA external type 1, N2 – OSPF NSSA external type 2

E1 – OSPF external type 1, E2 – OSPF external type 2

i – IS-IS, L1 – IS-IS level-1, L2 – IS-IS level-2, ia – IS-IS inter area

* – candidate default

S* 0.0.0.0/0 [10/0] via 192.168.183.254, port2

S   1.0.0.0/8 [10/0] via 192.168.183.254, port2

S  2.0.0.0/8 [10/0] via 192.168.183.254, port2

C  10.142.0.0/23 is directly connected, port3

B  10.160.0.0/23 [20/0] via 10.142.0.74, port3, 2d18h02m

C 192.168.182.0/23 is directly connected, port2

 

Examining an entry:

B 10.160.0.0/23 [20/0] via 10.142.0.74, port3, 2d18h02m

B                                                  BGP. The routing protocol used.

10.160.0.0/23                              The destination of this route including netmask.

[20/0]                                           20 indicates and administrative distance of 20 out of a range of 0 to 255.

0 is an additional metric associated with this route, such as in OSPF

10.142.0.74                                 The gateway, or next hop.

port3                                           The interface used by this route.

2d18h02m                                  How old this route is, in this case almost three days old.

 

To view the kernel routing table

# get router info kernel

tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->10.11.201.0/24 pref=10.11.201.4 gwy=0.0.0.0 dev=5(external1)

tab=254 vf=0 scope=253 type=1 proto=2 prio=0 0.0.0.0/0.0.0.0/0->172.20.120.0/24 pref=172.20.120.146 gwy=0.0.0.0 dev=6(internal)

The parts of the routing table entry are:

 

tab                                               Table number. This will be either 254 (unicast) or 255 (multicast).

vf                                                 Virtual domain of the firewall. This is the vdom index number. If vdoms are not enabled, this number will be 0.

type                                             Type of routing connection. Valid values include:

0 – unspecific

1 – unicast

2 – local

3 – broadcast

4 – anycast

5 – multicast

6 – blackhole

7 – unreachable

8 – prohibited

Type of installation. This indicates where the route came from. Valid values include:

proto

0 – unspecific

2 – kernel

11 – ZebOS routing module

14 – FortiOS

15 – HA

16 – authentication based

17 – HA1

prio                                             Priority of the route. Lower priorities are preferred.

–>10.11.201.0/24

(->x.x.x.x/mask)

The IP address and subnet mask of the destination

pref                                             Preferred next hop along this route

gwy                                             Gateway – the address of the gateway this route will use

dev                                              Outgoing interface index. This number is associated with the interface for this route, and if VDOMs are enabled the VDOM will be included here as well. If an interface alias is set for this interface it will also be displayed here.

Default route

The default route is used if either there are no other routes in the routing table or if none of the other routes apply to a destination. Including the gateway in the default route gives all traffic a next-hop address to use when leaving the local network. The gateway address is normally another router on the edge of the local network.

All routers, including FortiGate units, are shipped with default routes in place. This allows customers to set up and become operational more quickly. Beginner administrators can use the default route settings until a more advanced configuration is warranted.

FortiGate units come with a default static route with an IPv4 address of 0.0.0.0, an administration distance of 10, and a gateway IPv4 address.

Adding a static route

To add or edit a static route, go to Router > Static > Static Routes and select Create New.

Destination IP / Mask               Enter the destination IP address and netmask.

A value of 0.0.0.0/0.0.0.0 is universal.

Device                                         Select the name of the interface which the static route will connect through.

Gateway                                     Enter the gateway IP address.

Distance

Enter the distance value, which will affect which routes are selected first by different protocols for route management or load balancing. The default is

10.

Priority                                       Enter the priority if desired, which will artificially weight the route during route selection. The higher the number, the less likely the route is to be selected over others. The default is 0.

Routing table

When two computers are directly connected, there is no need for routing because each computer knows exactly where to find the other computer. They communicate directly.

Networking computers allows many computers to communicate with each other. This requires each computer to have an IP address to identify its location to the other computers. This is much like a mailing address – you will not receive your postal mail at home if you do not have an address for people to send mail to. The routing table on a computer is much like an address book used to mail letters to people in that the routing table maintains a list of how to reach computers. Routing tables may also include information about the quality of service (QoS) of the route, and the interface associated with the route if the device has multiple interfaces.

Looking at routing as delivering letters is more simple than reality. In reality, routers loose power or have bad cabling, network equipment is moved without warning, and other such events happen that prevent static routes from reaching their destinations. When any changes such as these happen along a static route, traffic can no longer reach the destination — the route goes down. Dynamic routing can address these changes to ensure traffic still reaches its destination. The process of realizing there is a problem, backtracking and finding a route that is operational is called convergence. If there is fast convergence in a network, users won’t even know that re- routing is taking place.

The routing table for any device on the network has a limited size. For this reason, routes that aren’t used are replaced by new routes. This method ensures the routing table is always populated with the most current and most used routes—the routes that have the best chance of being reused. Another method used to maintain the routing table’s size is if a route in the table and a new route are to the same destination, one of the routes is selected as the best route to that destination and the other route is discarded.

Routing tables are also used in unicast reverse path forwarding (uRPF). In uRPF, the router not only looks up the destination information, but also the source information to ensure that it exists. If there is no source to be found, then that packet is dropped because the router assumes it to be an error or an attack on the network.

The routing table is used to store routes that are learned. The routing table for any device on the network has a limited size. For this reason, routes that aren’t used are replaced by new routes. This method ensures the routing table is always populated with the most current and most used routes — the routes that have the best chance of being reused. Another method used to maintain the routing table’s size is if a route in the table and a new route are to the same destination, one of the routes is selected as the best route to that destination and the other route is discarded.

Some actions you can perform on the routing table include:

• Viewing the routing table in the web-based manager
• Viewing the routing table in the CLI
• Searching the routing table

Viewing the routing table in the web-based manager

By default, all routes are displayed in the Routing Monitor list. The default static route is defined as 0.0.0.0/0, which matches the destination IP address of “any/all” packets.

To display the routes in the routing table, go to Router > Monitor > Routing Monitor.

The figure below shows the Routing Monitor list belonging to a FortiGate unit that has interfaces named “port1”, “port4”, and “lan”. The names of the interfaces on your FortiGate unit may be different.

Advanced Static Routing

Advanced static routing includes features and concepts that are used in more complex networks. Dynamic routing is not addressed in this section.

This section includes: Routing concepts

Static routing tips

Policy routing

Transparent mode static routing

Static routing example

 

Routing concepts

Many routing concepts apply to static routing. However without first understanding these basic concepts, it is difficult to understand the more complex dynamic routing.

This section includes:

• Routing in VDOMs
• Default route
• Adding a static route
• Routing table
• Building the routing table
• Static routing security
• Multipath routing and determining the best route
• Route priority
• Troubleshooting static routing

 

Routing in VDOMs

Routing on FortiGate units is configured per-VDOM. This means if VDOMs are enabled, you must enter a VDOM to do any routing configuration. This allows each VDOM to operate independently, with its own default routes and routing configuration.

In this guide, the procedures assume your FortiGate unit has VDOMs disabled. This is stated in the assumptions for the examples. If you have VDOMs enabled you will need to perform the following steps in addition to the procedure’s steps.

 

To route in VDOMs – web-based manager

Select the VDOM that you want to view or configure at the bottom of the main menu.

 

 

To route in VDOMs – CLI

Before following any CLI routing procedures with VDOMs enabled, enter the following commands. For this example, it is assumed you will be working in the root VDOM. Change root to the name of your selected VDOM

as needed.

config vdom edit root

Following these commands, you can enter any routing CLI commands as normal.