How to check number of sessions used by UTM proxy

Each FortiGate model has a set limit of the maximum number of sessions the UTM proxy supports. The UTM proxy handles all the traffic for the following protocols: HTTP, SMTP, POP3, IMAP, FTP, and NNTP. If the proxy for a protocol fills up its session table, the FortiGate unit will enter conserve mode, where it behaves differently, until entries and memory free up again.

 

Conserve or failopen mode

Once you reach the limit, depending on your FortiGate unit’s conserve mode configuration, no new sessions are created until an old ones end. You can configure your FortiGate unit’s behavior when memory is running low or the proxy connection limit has been reached. There are two related commands for this in the CLI:

config system global

set av-failopen-session {enable | disable}

set av-failopen { idledrop | off | one-shot | pass}

end

av-failopen-session must be enabled to set the behavior for these conditions. When it is enabled, and a proxy for a protocol runs out of room in its session table that protocol goes into failopen mode and behaves as defined in the av-failopen command.

av-failopen determines the behavior of the proxy until entries are free in the session table again for that proxy.

• idledrop — This option removes idle sessions from the session table, starting with the clients that have the most sessions currently open. This method assumes that idle sessions are not being used and it will not cause problems to close these sessions. This is usually true, but some applications may have problems with this and start complaining about either not having or being able to open a session. If this occurs, try another method to check if this is really the problem. This is a secure option as no unscanned traffic is allowed to pass.
• off — This option turns off accepting any new AV sessions, but will continue to process any existing AV sessions that are currently active. All the protocols listed (HTTP, SMTP, POP3, IMAP, FTP, and NNTP) are scanned by FortiGate Antivirus. If AV scanning is enabled, av-failopen off is selected, and the proxy session table fills up, then no new sessions of that type will be accepted. For example, if POP3 session table is filled and email AV scanning is enabled, no more POP3 connections will be allowed until the session table gets some free space. This is a secure option because no unscanned traffic is allowed to pass.
• one-shot — When memory is low, bypass the antivirus system. The name one-shot comes from the fact that once you are in one-shot av-failopen mode, you must set av-failopen to either pass or off to restart AV scanning. This is a very unsecure option because it allows all traffic without AV scanning, and it never reverts to normal without manual assistance.
• pass — When memory is low, bypass the antivirus system much as one-shot. The difference is that when memory is freed up, the system will start AV scanning automatically again. This is an unsecure option because it allows

 

traffic to pass without AV scanning. However, it is better than one-shot because it automatically restarts AV scanning when possible.

If the proxy session table is full for one or more protocols and your FortiGate unit enters into conserve or failopen mode, it will appear as if you have lost connections, network services are intermittent or non-existent, and yet other services work normally for a while until their sessions end and they join the queue of session-starved applications.

 

Checking sessions in use

To make troubleshooting this type of problem easier, sessions are broken down by which protocol they use. This provides you with statistics and errors specific to one of the protocols.

Due to the amount of output from this command, you should connect to the CLI with a terminal program, such as puTTY, that logs output. Otherwise, you will likely not be able to access all the output information from the command.

In the following output, only the HTTP entries are displayed. The other protocols have been removed in an attempt to shorten the output. There will be separate entries for each supported protocol (HTTP, SMTP, POP3, IMAP, FTP, and NNTP) in each section of the output.

 

To check sessions in use and related errors – CLI

FGT# # get test proxyworker 4

Worker[0] HTTP Common

Current Connections 8/8032

Max Concurrent Connections 76

Worker Stat

Running time (HH:MM:SS:usec) 29:06:27:369365

Time in loop scanning 2:08:000198

Error Count (accept) 0

Error Count (read) 0

Error Count (write) 0

Error Count (poll) 0

Error Count (alloc) 0

Last Error 0

Acceptor Read 6386

Acceptor Write 19621

Acceptor Close 0

 

HTTP Stat

Bytes sent 667012 (kb) Bytes received 680347 (kb) Error Count (alloc) 0

Error Count (accept) 0

Error Count (bind) 0

Error Count (connect) 0

Error Count (socket) 0

Error Count (read) 134

Error Count (write) 0

 

Error Count (retry) 40

Error Count (poll) 0

Error Count (scan reset) 2

Error Count (urlfilter wait) 3

Last Error 104

Web responses clean 17950

Web responses scan errors 23

Web responses detected 16

Web responses infected with worms 0

Web responses infected with viruses 0

Web responses infected with susp 0

Web responses file blocked 0

Web responses file exempt 0

Web responses bannedword detected 0

Web requests oversize pass 16

Web requests oversize block 0

Last Server Scan errors 102

URL requests exempt 0

URL requests blocked 0

URL requests passed 0

URL requests submit error 0

URL requests rating error 0

URL requests rating block 0

URL requests rating allow 10025

URL requests infected with worms 0

Web requests detected 0

Web requests file blocked 0

Web requests file exempt 0

POST requests clean 512

POST requests scan errors 0

POST requests infected with viruses 0

POST requests infected with susp 0

POST requests file blocked 0

POST requests bannedword detected 0

POST requests oversize pass 0

POST requests oversize block 0

Web request backlog drop 0

Web response backlog drop 0

 

Worker Accounting

poll=721392/649809/42 pollfail=0 cmdb=85 scan=19266 acceptor=25975

 

HTTP Accounting

setup_ok=8316 setup_fail=0 conn_ok=0 conn_inp=8316 urlfilter=16553/21491/20 uf_lookupf=0

scan=23786 clt=278876 srv=368557

 

SMTP Accounting

setup_ok=12 setup_fail=0 conn_ok=0 conn_inp=12

scan=12 suspend=0 resume=0 reject=0 spamadd=0 spamdel=0 clt=275 srv=279

 

POP3 Accounting

setup_ok=30 setup_fail=0 conn_ok=0 conn_inp=30 scan=3 clt=5690 srv=5836

 

IMAP Accounting

setup_ok=0 setup_fail=0 conn_ok=0 conn_inp=0 scan=0 clt=0 srv=0

 

FTP Accounting

setup_ok=0 setup_fail=0 conn_ok=0 conn_inp=0

scan=0 clt=0 srv=0 datalisten=0 dataclt=0 datasrv=0

 

NNTP Accounting

setup_ok=0 setup_fail=0 conn_ok=0 conn_inp=0 scan=0 clt=0 srv=0

 

The output from this command falls into the following sections:

 

• HTTP Common current connections — There is an entry for each protocol that displays the connections currently used, and the maximum connections allowed. This maximum is for the UTM proxy, which means all the protocols connections combined cannot be larger than this number. To support this, note that the maximum

session count for each protocol is the same. You may also see a line titled Max Concurrent Connections for each protocol. This number is the maximum connections of this type allowed at one time. If VDOMs are enabled, this value is defined either on the global or per-VDOM level at VDOM > Global Resources.

• Worker Stat — This is statistics about the UTM proxy including how long it has been running, and how many errors it has found.
• HTTP Stat — This section includes statistics about the HTTP protocol proxy. This is a very extensive list covering errors, web responses, and any UTM positive matches. There are similar sections for each protocol, but the specific entries in each vary based on what UTM scanning is looking for in each — spam control for email, file transfer blocking for FTP, and so on.
• Worker Accounting — Lists accounting information about the UTM proxy such as polling statistics, how many sessions were scanned, and how many were just accepted. This information can tell you if expect AV scanning is taking place or not. Under normal operation there should be no errors or fails.
• HTTP Accounting — The accounting sections for each protocol provide information about successful session creation, failures, how many sessions are being scanned or filtered, and how many are client or server originated. If setup_fail is larger than zero, run the command again to see if it is increasing quickly. If it is, your FortiGate unit may be in conserve mode.

 

Related commands

To dump memory usage:

# get test proxyworker 1

 

To display statistics per VDOM:

# get test proxyworker 4444

 

To restart the proxy:

# get test proxyworker 99