Reliable webfilter statistics

Introduction

FortiOS 6.2.0 provides command line tools to view the webfilter statistics report. These command line tools currently fall into either proxy-based or flow-based webfilter statistics commands.

Proxy-based webfilter statistics report

l The proxy-based webfilter statistics command line tools are as follows. These commands are available in both global or per-VDOM command lines.

#diagnose wad filter <—-define the interested objects for output (global) # diag wad ? console-log   Send WAD log messages to the console. debug  Debug setting. stats       Show statistics.

filter    Filter for listing sessions or tunnels. <—-use filter to filter-out interested object and output kxp    SSL KXP diagnostics. user  User diagnostics. memory    WAD memory diagnostics.

restore   Restore configuration defaults. history   Statistics history. session   Session diagnostics. tunnel       Tunnel diagnostics. webcache  Web cache statistics. worker    Worker diagnostics. csvc   Cache service diagnostics.

#diagnose wad stat filter list/clear <—-list/clear WebFiltering/DLP statistics report l In the example below, there are two VDOMs using proxy-based policies which have webfilter profiles enabled. The command line can be used to view the proxy-based webfilter statistics report.

(global) # diag wad filter ? list   Display current filter. clear     Erase current filter settings. src      Source address range to filter by. dst     Destination address range to filter by.

sport     Source port range to filter by. dport   Destination port range to filter by. vd   Virtual Domain Name. <—-filter for per-vdom or global statistics report explicit-policy   Index of explicit-policy. -1 matches all. firewall-policy Index of firewall-policy. -1 matches all. drop-unknown-session   Enable drop message unknown sessions. negate   Negate the specified filter parameter. protocol    Select protocols to filter by.

FGT_600D-ICAP-NAT (global) # diag wad filter vd <vdom>    Virtual Domain Name. ALL   all vdoms root      vdom vdom1 vdom

FGT_600D-ICAP-NAT (global) # diag wad filter vd root <—-filter-out root vdom statistics

Drop_unknown_session is enabled.

FGT_600D-ICAP-NAT (global) # diag wad stats filter list filtering of vdom root <—-Displayed the WF statistics for root vdom

dlp          = 0     <—-Number of Reuqest that DLP Sensor processed;

content-type = 0     <—-Number of Reuqest that matching content-type filter;

urls:
examined = 6 examined;	<—-Number of Request that Proxy Web-Filter(all wad daemons)
allowed = 3	<—-Number of Request that be allowed in the examined requests;
blocked = 0	<—-Number of Request that be blocked in the examined requests;
logged = 0	<—-Number of Request that be logged in the examined requests;

overridden = 0 <—-Number of Request that be overrided to another webfilter

profile in the examined requests;

FGT_600D-ICAP-NAT (global) # diag wad filter vd vdom1 <—-filter-out vdom1 statistics

FGT_600D-ICAP-NAT (global) # diag wad stats filter list filtering of vdom vdom1 <—-Displayed the WF statistics for vdom1 dlp   = 0 content-type = 0 urls:

examined = 13 allowed = 2 blocked = 9 logged = 8 overridden = 0 FGT_600D-ICAP-NAT (global) # diag wad filter vd ALL

FGT_600D-ICAP-NAT (global) # diag wad stats filter list

filtering of all accessible vdoms <—-global statistics is sum of two VDOMs dlp     = 0 content-type = 0 urls:

examined = 19 allowed = 5 blocked = 9 logged = 8 overridden = 0

Flow-based webfilter statistics report

• The flow-based webfilter statistics command line tools are as follows. These commands are available in global command lines only.

(global) # diag test app ipsmonitor IPS Engine Test Usage:

1: Display IPS engine information 2: Toggle IPS engine enable/disable status 3: Display restart log 4: Clear restart log 5: Toggle bypass status 6: Submit attack characteristics now 10: IPS queue length 11: Clear IPS queue length 12: IPS L7 socket statistics 13: IPS session list 14: IPS NTurbo statistics 15: IPSA statistics 18: Display session info cache 19: Clear session info cache 21: Reload FSA malicious URL database 22: Reload whitelist URL database 24: Display Flow AV statistics 25: Reset Flow AV statistics 27: Display Flow urlfilter statistics 28: Reset Flow urlfilter statistics
29: Display global Flow urlfilter statistics Statistics	<—-List the Flow Web Filtering
30: Reset global Flow urlfilter statistics Statistics 96: Toggle IPS engines watchdog timer 97: Start all IPS engines 98: Stop all IPS engines 99: Restart all IPS engines and monitor	<—-Reset the Flow Web Filtering

• In the example below, there are two VDOMs using flow-based policies which have webfilter profiles enabled. The command line can be used to view the flow-based webfilter statistics report.

(global) # diag test app ipsmonitor 29 Global URLF states: request: 14 <—-Number of Requests that Flow Web-Filter(all ips engines) received; response: 14 <—-Number of Response that Flow Web-Filter(all ips engines) sent; pending: 0       <—-Number of Requests that under processing at that moment; request error: 0       <—-Number of Request that have error; response timeout: 0 <—-Number of response that ips engine not been received in-

time;

blocked: 12    <—-Number of Request that Flow Web-Filter blocked; allowed: 2  <—-Number of Request that Flow Web-Filter allowed;

File filter for webfilter

Introduction

File Filter is a new feature introduced in FortiOS 6.2, and provides the Web filter profile with the capability to block files passing through a FortiGate based on file type. In addition, the configuration for file type filtering has been greatly simplified. In previous FortiOS versions, File Filtering could only be achieved by configuring a DLP (Data Leak Prevention) Sensor.

In FortiOS 6.2, HTTP and FTP File Filtering is configurable in Web filter profile, and SMTP, POP3, IMAP file-filtering is configurable in Email filter profile. Currently, File Filtering in Web filter profile is based on file type (file’s meta data) only, and not on file size or file content. Users will still need to configure a DLP sensor to block files based on size or content such as SSN numbers, credit card numbers or regexp.

FTP inspection and GUI configuration have yet to be implemented. In addition, Web filter File Filtering will only work on proxy mode policies.

File Types Supported

File Filter in Web filter profile supports the following file types:

File Type Name	Description
all	Match any file
7z	Match 7-zip files
arj	Match arj compressed files
cab	Match Windows cab files
lzh	Match lzh compressed files
rar	Match rar archives
tar	Match tar files
zip	Match zip files
bzip	Match bzip files
gzip	Match gzip files
bzip2	Match bzip2 files
xz	Match xz files
bat	Match Windows batch files
msc	Match msc files
uue	Match uue files
mime	Match mime files
base64	Match base64 files
binhex	Match binhex files

 

File Type Name	Description
bin	Match bin files
elf	Match elf files
exe	Match Windows executable files
hta	Match hta files
html	Match html files
jad	Match jad files
class	Match class files
cod	Match cod files
javascript	Match javascript files
msoffice	Match MS-Office files. For example, doc, xls, ppt, and so on.
msofficex	Match MS-Office XML files. For example, docx, xlsx, pptx, and so on.
fsg	Match fsg files
upx	Match upx files
petite	Match petite files
aspack	Match aspack files
prc	Match prc files
sis	Match sis files
hlp	Match Windows help files
activemime	Match activemime files
jpeg	Match jpeg files
gif	Match gif files
tiff	Match tiff files
png	Match png files
bmp	Match bmp files
ignored	Match ignored files
unknown	Match unknown files
mpeg	Match mpeg files
mov	Match mov files
mp3	Match mp3 files
wma	Match wma files
File Type Name	Description
wav	Match wav files
pdf	Match pdf files
avi	Match avi files
rm	Match rm files
torrent	Match torrent files
msi	Match Windows Installer msi bzip files
mach-o	Match Mach object files
dmg	Match Apple disk image files
.net	Match .NET files
xar	Match xar archive files
chm	Match Windows compiled HTML help files
iso	Match ISO archive files
crx	Match Chrome extension files

Configure File Filter from CLI

Using CLI, configuration for File Filtering is nested inside Web filter profile’s configuration.

In File filtering configuration, file filtering functionality and logging is independent of the Web filter profile.

To block or log a file type, configure file filter entries. Within each entry, specify a file-type, action (log|block), protocol to inspect (http|ftp), direction we want to inspect traffic (incoming|outgoing|any), and match only encrypted files. In addition, in each file filter entry we can specify multiple file types. File filter entries are ordered, however, blocked will take precedence over log.

In the CLI example below, we want to file filter the following using Web filter profile:

1. Block PDFs from entering our leaving our network (filter1).
2. Log the download of some graphics file-types via HTTP (filter2).
3. Block EXE files from leaving to our network via FTP (filter3).

config webfilter profile edit “webfilter-file-filter” config file-filter
set status enable filtering	<– Allow user to disable/enable file
set log enable file filtering	<– Allow user to disable/enable logging for
set scan-archive-contents enable such as ZIP, RAR etc. config entries edit “filter1”	<– Allow scanning of files inside archives
set comment “Block PDF files” set protocol http ftp     <– Inspect HTTP and FTP traffic set action block <– Block file once file type is matched

set direction any <– Inspect both incoming and outgoing traffic set encryption any    <– Inspect both encrypted and un-encrypted

files set file-type “pdf” <– Choosing the file type to match next edit “filter2” set comment “Log graphics files”

set protocol http <– Inspect only HTTP traffic set action log   <– Log file once file type is matched set direction incoming <– Only inspect incoming traffic set encryption any

set file-type “jpeg” “png” “gif” <– Multiple file types can be configured

in a single entry

next edit “filter3” set comment “Block upload of EXE files”

set protocol ftp  <– Inspect only FTP traffic set action log

set direction outgoing   <– Inspect only outgoing traffic set encryption any set file-type “exe”

next

end

end

end

After configuring File Filter in Webfilter profile we must apply it to a firewall policy using the following command:

config firewall policy edit 1 set name “client-to-internet” set srcintf “dmz” set dstintf “wan1” set srcaddr “all” set dstaddr “all” set action accept set schedule “always” set service “ALL” set utm-status enable set utm-inspection-mode proxy set logtraffic all set webfilter profile “webfilter-filefilter” set profile-protocol-options “protocol” set ssl-ssh-profile “protocols”

set nat enable

next end

Log Example

GUI > VDOM > Log & Report > Web Filter:

External resources for webfilter

Introduction

External Resources is a new feature introduced in FortiOS 6.0, which provides a capability to import an external blacklist which sits on an HTTP server. This feature helps FortiGate retrieve a dynamic URL/Domain Name/IP Address/Malware hash list from an external HTTP server periodically. FortiGate uses these external resources as web filter’s remote categories, DNS filter’s remote categories, policy address objects or AntiVirus profile’s malware definitions. If the external resource is updated, FortiGate objects will update dynamically.

External Resource are categorized into 4 types:

• URL list (Type=category) l Domain Name List (Type=domain) l IP Address list (Type=address) l Malware hash list (Type=malware)

For Web Filter profile, it can use category type external resources. Category type external resources file is a URL entries list in a plain text file.

When a category type external resource is configured in Web Filter profile, it will be treated as a Remote Category. If the URL in a HTTP/HTTPS request matches the entry inside this external resource file, it will be treated as the Remote Category and follow the action configured for this category in Web Filter profile.

External resource type category also can be used in ssl-ssh-profile configuration for category-based SSL-Exempt. When a Remote Category is configured in ssl-ssh-profile SSL-Exempt, if a HTTPS request’s URL matches in the Remote Category’s entry list, HTTPS request with destination for this URL can be exempted from SSL Deep Inspection. External Resources File Format

External Resources File should follow the following requirements:

• The external resource file is a plain text format file and each URL list/IP Address/Domain Name occupies a single line. l The file is limited to 10M, line is limited 128K (128 x 1024 entries), and the line length limit is 4K characters. l The entries limited also follow table size limitation defined by CMDB per model. l The external resource update period can be set to 1 minute, hourly, daily, weekly, or monthly (43200 min, 30 days).
• The external resource type as category (URL list) and domain (Domain Name list) share the category number range 192-221 (total 30 categories). l There’s no duplicated entry validation for external resources file (entry inside each file or inside different files).

For URL list (Type=category):

Scheme is optional, and will be truncated if found (http://, https:// is not needed).

Wildcard (*) is supported (from 6.2). It supports the ‘*’ at beginning and ending of URL, and not in the middle of URL as follows:

+ support *.domain2.com, domain.com.* + not support: domain3.*.com IDN (International Domain Name) and UTF encoding URL is supported (from 6.2).

IPv4,IPv6 format URL is supported. IPv6 in URL list must in [ ] form.

Configure External Resources from CLI

We can use CLI to configure the external resources files that is located on external HTTP Server. Under Global, configure the external resource file location and specify the resource type.

Web Filter will use category type external resources as Remote Categories. In the following example, it is configured a file Ext-Resource-Type-as-Category-1.txt as type as category, it will be treated in Web Filter as Remote Category, the category name configured as Ext-Resource-Type-as-Category-1 and category-id as 192:

config system external-resource edit “Ext-Resource-Type-as-Category-1”

set type category <—-

set category 192 <—-

set resource “http://172.16.200.66/external-resources/Ext-Resource-Type-as-Category-

1.txt” set refresh-rate 1

next

end

Now in each VDOM, category type external resource can be used in Web Filter as Remote Cateogry. In the example above, URL list in “Ext-Resource-Type-as-Category-1.txt” file will be treated as remote category (category-id 192). Configure the action for this remote category in Web Filter profile and apply it in the policy:

config webfilter profile edit “webfilter” config ftgd-wf unset options config filters edit 1 set category 2 set action warning

next ……

edit 24 set category 192 <—set action block

next edit 25 set category 221 set action warning

next edit 26 set category 193

next

end

end

set log-all-url enable

next

end

config firewall policy edit 1 set name “WebFilter” set srcintf “port10” set dstintf “port9” set srcaddr “all” set dstaddr “all” set action accept set schedule “always” set service “ALL” set utm-status enable set logtraffic all set webfilter-profile “webfilter” set profile-protocol-options “protocol” set ssl-ssh-profile “protocols”

set nat enable

next end

Configure External Resources from GUI

Configure, edit or view the Entries for external resources from GUI.

1. GUI > Global > Fabric Connectors page:
2. GUI > Global > Fabric Connectors page > Create New. Click Create New button, and select Threat Feeds Type FortiGuard
3. GUI > Global > Fabric Connectors page. Enter the Resource Name, URL Location of the resource file, resource authentication credential, Refresh Rate or comment, and click OK to finish the Threat Feeds configuration.
4. GUI > Global > Fabric Connectors page. After a few minutes, double-click the Threat Feeds Object you just configured. It is shown in the Edit Click View Entries to view the entry list in the external resources file:
5. GUI > VDOM > Web Filter Profile page. The configured external resources is shown and configured in each Web

Filter Profile:

Log Example

If a HTTP/HTTPS request URL matched in Remote Category’s entry list, it will override its original FGD URL rating and it is treated as Remote Category.

GUI > VDOM > Log & Report > Web Filter:

CLI Example:

1: date=2019-01-18 time=15:49:15 logid=”0316013056″ type=”utm” subtype=”webfilter” eventtype=”ftgd_blk” level=”warning” vd=”vdom1″ eventtime=1547855353 policyid=1 sessionid=88922 srcip=10.1.100.18 srcport=39886 srcintf=”port10″ srcintfrole=”undefined” dstip=216.58.193.67 dstport=443 dstintf=”port9″ dstintfrole=”undefined” proto=6 service=”HTTPS” hostname=”www.fortinet.com” profile=”webfilter” action=”blocked” reqtype=”direct” url=”/” sentbyte=752 rcvdbyte=10098 direction=”outgoing” msg=”URL belongs to a denied category in policy” method=”domain” cat=192 catdesc=”Ext-Resource-Type-as-Category-1″

Remote Category in ssl-ssh-profile category-based SSL-Exempt

Remote Category can be applied in ssl-ssh-profile category-based SSL-Exempt.

GUI > VDOM > Security Profiles > SSL/SSH Inspection:

HTTPS Request URL matched in this Remote Category will be exempted from SSL Deep Inspection.

Log example:

3: date=2019-01-18 time=16:06:21 logid=”0345012688″ type=”utm” subtype=”webfilter” eventtype=”ssl-exempt” level=”information” vd=”vdom1″ eventtime=1547856379 policyid=1 sessionid=90080 srcip=10.1.100.18 srcport=39942 srcintf=”port10″ srcintfrole=”undefined” dstip=216.58.193.67 dstport=443 dstintf=”port9″ dstintfrole=”undefined” proto=6 service=”HTTPS” hostname=”www.fortinet.com” profile=”webfilter” action=”passthrough” reqtype=”direct” url=”/” sentbyte=517 rcvdbyte=0 direction=”outgoing” msg=”The SSL session was exempted.” method=”domain” cat=192 catdesc=”Ext-Resource-Type-as-Category-1″ urlsource=”exempt_type_user_cat”

Local Category and Remote Category Priority

Web Filter can have both local category and remote category at the same time. There’s no duplication check between local category URL override and remote category resource file. For example, a URL like www.example.com may be shown both in remote category entry list and in FortiGate’s local category URL override configuration. We recommend avoiding this scenario since FortiGate does not check for duplicates. However, if a URL is duplicated in both local category and remote category, it is rated as local category.

Advanced Filters 2

Safe search

This feature applies to popular search sites and prevents explicit websites and images from appearing in search results.

Supported search sites are: l Google l Yahoo l Bing l Yandex

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Search Engines
2. Enable Enforce ‘Safe Search’ on Google, Yahoo!, Bing, Yandex.

To enable this feature in the CLI:

config webfilter profile edit “webfilter” config web set safe-search url

end

next

end

YouTube education filters

Use these features to limit users’ access to YouTube channels, such as in an education environment where you want students and users to be able to access YouTube education videos but not other YouTube videos.

Restrict YouTube access

Formerly, YouTube for Schools was a way to access educational videos inside a school network. This YouTube feature lets schools access educational videos on YouTube EDU and to specify the videos accessible within the school network.

When Google stopped supporting YouTube for Schools on July 1, 2016, YouTube safe search also stopped working.

Google provides information on restricting YouTube content such as Restrict YouTube content available to G Suite users. At this time, the options Google offers to restrict inappropriate content includes: DNS, HTTP headers, and Chromebooks..

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Search Engines
2. Enable Restrict YouTube Access and select Strict or Moderate.

To enable this feature in the CLI:

config webfilter profile edit “webfilter” config web set youtube-restrict strict end

next

end

YouTube channel filtering

This web filtering feature is also called Restrict YouTube access to specific channels. Use this feature to block or only allow matching YouTube channels.

The following identifiers are used: given <channel-id>, affect on: www.youtube.com/channel/<channel-id> www.youtube.com/user/<user-id> matches channel-id from <meta itemprop=”channelId” content=”UCGzuiiLdQZu9wxDNJHO_JnA”>

www.youtube.com/watch?v=<string> matches channel-id from <meta itemprop=”channelId” content=”UCGzuiiLdQZu9wxDNJHO_JnA”>

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Proxy Options
2. Enable Restrict YouTube access to specific channels.
3. Select Create New and specify the Channel ID, for example, UCGzuiiLdQZu9wxDNJHO_JnA.
4. Select OK and the option shows the Channel ID and its Link.

To enable this feature in the CLI:

config webfilter profile  edit “webfilter”

set youtube-channel-status whitelist <– whitlist: only allow the traffic belongs to this channel id and relative identifiers

blacklist: only block the traffic belongs to

this channel id and relative identifiers and allow the other traffic pass  config youtube-channel-filter

edit 1

set channel-id “UCGzuiiLdQZu9wxDNJHO_JnA”  next

end

next end

Log all search keywords

Use this feature to log all search phrases.

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Search Engines
2. Enable Log all search keywords.

To enable this feature in the CLI:

config webfilter profile edit “webfilter” config web set log-search enable

end

next

end

Restrict Google account usage to specific domains

Use this feature to block access to some Google accounts and services while allowing access to accounts in the domains in the exception list.

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Proxy Options
2. Enable Restrict Google account usage to specific domains.
3. Select the + button and enter the domains that Google can access, for example, www.fortinet.com.

When you try to use Google services like Gmail, only traffic from the domain of www.fortinet.com can go through. Traffic from other domains is blocked.

HTTP POST Action

Select the action to take with HTTP POST traffic. HTTP POST is the command used by your browser when you send information, such as a form you have filled-out or a file you are uploading to a web server.

The action options are Allow or Block. The default is Allow.

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Proxy Options
2. For HTTP POST Action, select Allow or Block.

To enable this feature in the CLI:

config webfilter profile edit “webfilter” set post-action [normal/block] config ftgd-wf unset options

end

next end

Remove Java applets, remove ActiveX, and remove cookies

The Remove Java Applets feature filters java applets from web traffic. Websites using java applets might not function properly if you enable this filter.

The Remove ActiveX feature filters ActiveX scripts from web traffic. Websites using ActiveX might not function properly with if you enable this filter.

The Remove Cookies feature filters cookies from web traffic. Websites using cookies might not function properly if you enable this filter.

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Proxy Options
2. Select the filters you want to use: Remove Java Applets, Remove ActiveX, and/or Remove Cookies.

To enable this feature in the CLI:

config webfilter profile  edit “webfilter”

set options activexfilter cookiefilter javafilter <– enable one or more of activexfilter cookiefilter javafilter.

config ftgd-wf

unset options

end

next end

Advanced Filters 1

Block malicious URLs discovered by FortiSandbox

To use this feature, you must be registered to a FortiSandbox and be connected to it.

This feature blocks malicious URLs that FortiSandbox finds.

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Static URL Filter
2. Enable Block malicious URLs discovered by FortiSandbox.

To enable this feature in the CLI:

config webfilter profile edit “webfilter” config web set blacklist enable end

next

end

Allow websites when a rating error occurs

If you don’t have a FortiGuard license but you have enabled services that need a FortiGuard license, such as FortiGuard filter, then you’ll get a rating error message.

Use this setting to allow access to websites that return a rating error from the FortiGuard Web Filter service.

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Rating Options
2. Enable Allow websites when a rating erroroccurs.

To enable this feature in the CLI:

config webfilter profile edit “webfilter” config ftgd-wf set options error-allow

end

next

end

Rate URLs by domain and IP address

If you enable this feature, in addition to only sending domain information to FortiGuard for rating, FortiGate always sends both the URL domain name and the TCP/IP packet’s IP address (except for private IP addresses) to FortiGuard for the rating.

FortiGuard server might return a different category of IP address and URL domain. If they are different, FortiGate uses the rating weight of the IP address or domain name to determine the rating result and decision. This rating weight is hard-coded in FortiGate.

For example, if we use a spoof IP of Google as www.irs.gov, FortiGate will send both the IP address and domain name to FortiGuard to get the rating. In this example, we get two different ratings, one is search engine and portals which belongs to the IP of Google, another is government and legal organizations which belongs to www.irs.gov. As the search engine and portals has a higher weight than government and legal organizations, this traffic will be rated as search engine and portals and not rated as government and legal organizations.

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Rating Options
2. Enable Rate URLs by domain and IP address.

To enable this feature in the CLI:

config webfilter profile edit “webfilter” config ftgd-wf set options rate-server-ip

end

next

end

Block invalid URLs

Use this feature to block websites when their SSL certificate CN field does not contain a valid domain name.

For example, this option blocks URLs which contains spaces. If there is a space in the URL, it must be written as: http://www.example.com/space%20here.html.

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Static URL Filter
2. Enable Block invalid URLs .

To enable this feature in the CLI:

config webfilter profile edit “webfilter” set options block-invalid-url

next

end

Rate images by URL

This feature enable FortiGate to retrieve ratings for individual images in addition to websites. Images in a blocked category are not displayed even if they are part of a site in an allowed category. Blocked images are replaced with blank placeholders. These image file types are rated: GIF, JPEG, PNG, BMP, and TIFF.

This feature requires a valid FortiGuard license, otherwise rating errors will occur. By default, this feature is enabled.

For example, if the Other Adult Materials category is blocked, before enabling Rate images by URL, the image is not blocked:

After enabling Rate images by URL, images in the Other Adult Materials category are blocked. For example:

To enable this feature in the GUI:

1. Go to Security Profiles > Web Filter and go to the Rating Options
2. Enable Rate images by URL.

To enable this feature in the CLI:

config webfilter profile edit “webfilter” config ftgd-wf unset options set rate-image-urls enable

end

next

end

Web content filter of webfilter

You can control access to web content by blocking web pages containing specific words or patterns. This helps to prevent access to pages with questionable material. You can specify words, phrases, patterns, wildcards and Perl regular expressions to match content on web pages. You can use multiple web content filter lists and select the best web content filter list for each web filter profile.

Pattern type

When you have created the web filter content list, you need to add web content patterns to it. There are two types of patterns: wildcard and regular expression.

Wildcard

Use the wildcard setting to block or exempt one word or text strings of up to 80 characters. You can also use wildcard symbols such as ? or * to represent one or more characters. For example, a wildcard expression forti*.com matches fortinet.com and forticare.com. The * represents any character appearing any number of times.

Regular expression

Use the regular expression setting to block or exempt patterns of Perl expressions which use some of the same symbols as wildcard expressions but for different purposes. In regular expressions, * represents the character before the symbol. For example, forti*.com matches fortiii.com but not fortinet.com or fortiice.com. In this case, the symbol * represents i appearing any number of times.

The maximum number of web content patterns in a list is 5000.

Content evaluation

The web content filter feature scans the content of every web page that is accepted by a security policy. The system administrator can specify banned words and phrases and attach a numerical value, or score, to the importance of those words and phrases. When the web content filter scan detects banned content, it adds the scores of banned words and phrases found on that page. If the sum is higher than a threshold set in the web filter profile, FortiGate blocks the page.

The default score for web content filter is 10 and the default threshold is 10. This means that by default, a web page is blocked by a single match.

Banned words or phrases are evaluated according to the following rules:

• The score for each word or phrase is counted only once, even if that word or phrase appears many times in the web page.
• The score for any word in a phrase without quotation marks is counted. l The score for a phrase in quotation marks is counted only if it appears exactly as written.

Sample of applying banned pattern rules

The following table is an example of how rules are applied to the contents of a web page. For example, a web page contains only this sentence:

The score for each word or phrase is counted only once, even if that word or phrase appears many times in the web page.

Banned pattern	Assigned score	Score added to the sum for the entire page	Threshold score	Comment
word	20	20	20	Appears twice but only counted once. Web page is blocked.
word phrase	20	40	20	Each word appears twice but only counted once giving a total score of 40. Web page is blocked.
word sentence	20	20	20	“word” appears twice, “sentence” does not appear, but since any word in a phrase without quotation marks is counted, the score for this pattern is 20. Web page is blocked.
“word sentence”	20	0	20	This phrase does not appear exactly as written. Web page is allowed.
“word or phrase”	20	20	20	This phrase appears twice but is counted only once. Web page is blocked.

Sample configuration

To configure web content filter in the GUI:

1. Go to Security Profiles > Web Filter and go to the Static URL Filter
2. Enable Content Filter to display its options.
3. Select Create New to display the content filter options.
4. For Pattern Type, select RegularExpression and enter fortinet in the Pattern
   • Leave Language as Western. l Set Action to Block.
   • Set Status to Enable.
5. Select OK to see the updated Static URL Filter
6. Validate the configuration by visiting a website with the word fortinet, for example, www.fortinet.com. The website is blocked and a replacement page displays.

To configure web content filter in the CLI:

1. Create a content table:

config webfilter content

edit 1                           <– the id of this content

set name “webfilter”

config entries

edit “fortinet”            <– the banned word set pattern-type regexp  <– the type is regular expression set status enable set lang western

set score 10             <– the score for this word is 10 set action block

next

end

next end

2. Attach the content table to the webfilter profile:

config webfilter profile

edit “webfilter”

config web

set bword-threshold 10  <– the threshold is 10

set bword-table 1       <– the id of content table we created in the previous step

end

config ftgd-wf

unset options

end

next end

Quota of webfilter

In addition to using category and classification blocks and overrides to limit user access to URLs, you can set a daily quota by category, category group, or classification. Quotas allow access for a specified length of time or a specific bandwidth, and is calculated separately for each user. Quotas are reset everyday at midnight.

Quotas can be set only for the actions of Monitor, Warning, or Authenticate. When the quota is reached, the traffic is blocked and the replacement page displays.

Sample topology

Sample configuration of setting a quota

This example shows setting a time quota for a category, for example, the Education category.

To configure a quota in the GUI:

1. Go to Security Profiles > Web Filter and go to the FortiGuard category based filter
2. Open the General Interest -Personal section by selecting the + icon beside it.
3. Select Education and then select Monitor.
4. In the Category Usage Quota section, select Create New.
5. In the right pane, select the Category field and then select Education.
6. For the Quota Type, select Time and set the Total quota to 5 minute(s).
7. Select OK and the Category Usage Quota section displays the quota.
8. Validate the configuration by visiting a website in the education category, for example https://www.harvard.edu/.

You can view websites in the education category.

9. Check the used and remaining quota in Monitor> FortiGuard Quota.
10. When the quota reaches its limit, traffic is blocked and the replacement page displays.

To configure a quota in the CLI:

config webfilter profile edit “webfilter” config ftgd-wf

unset options

config filters

edit 1

set category 30 <– the id of education category  next

end

config quota

edit 1

set category 30

set type time

set duration 5m

next

end end

next

end

FortiGuard filter of webfilter

To use this service, you must have a valid subscription on your FortiGate.

FortiGuard filter enhances the web filtering features supplied with your FortiGate unit by sorting billions of web pages into a wide range of categories that users can allow or block.

FortiGuard web filtering services includes over 45 million individual website rating that applies to more than two billion pages. When FortiGuard filter is enabled in a webfilter and is applied to firewall policies, if a request for a web page appears in traffic controlled by one of the firewall policies, the URL is sent to the nearest FortiGuard server. The URL category or rating is returned. If the category is blocked, the FortiGate shows a replacement message in place of the requested page. If the category is not blocked, the page request is sent to the requested URL as normal.

FortiGuard webfilter action

You can select one of the following FortiGuard webfilter actions:

FortiGuard webfilter Action	Description
Allow	Permit access to the sites in the category.
Block	Prevent access to the sites in the category. Users trying to access a blocked site sees a replacement message indicating the site is blocked.
Monitor	Permits and logs access to sites in the category. You can enable user quotas when you enable this action.
Warning	Displays a message to the user allowing them to continue if they choose.
Authenticate	Requires the user to authenticate with the FortiGate before allowing access to the category or category group.

FortiGuard webfilter categories

FortiGuard has many webfilter categories including two local categories and a special remote category. For more information on the different categories, see the table below.

FortiGuard webfilter category	Where to find more information
All URL categories	https://fortiguard.com/webfilter/categories.
Remote category	External resources for webfilter on page 329.

The priority of categories is local category > external category > FortiGuard built-in category. If a URL is configured as a local category, it only follows the behavior of local category and not external or FortiGuard built-in category.

Sample configuration of blocking a web category

This example shows blocking a website based on its category (rating), for example, information technology.

To block a category in the GUI:

1. Go to Security Profiles > Web Filter and go to the FortiGuard category based filter
2. Open the General Interest -Business section by clicking the + icon beside it.
3. Select Information Technology and then select Block.

To block a category in the CLI:

config webfilter profile

edit “webfilter”

config ftgd-wf

unset options

config filters

edit 1

set category 52    <– the pre-set id of “information technology” caterogy

set action block   <– set action to block  next

end

end

next end

To validate that you have blocked a category:

1. Go to a website belonging to the blocked category, for example, www.fortinet.com, and you see a blocked page and the category that is blocked.

To view the log of a blocked website in the GUI:

1. Go to Log & Report > Web Filter.

To view the log of a blocked website in the CLI:

FGT52E-NAT-WF # execute log filter category utm-webfilter

FGT52E-NAT-WF # execute log display

1: date=2019-04-22 time=13:46:25 logid=”0316013056″ type=”utm” subtype=”webfilter” eventtype=”ftgd_blk” level=”warning” vd=”vdom1″ eventtime=1555965984972459609 policyid=1 sessionid=659263 srcip=10.1.200.15 srcport=49234 srcintf=”wan2″ srcintfrole=”wan” dstip=54.183.57.55 dstport=80 dstintf=”wan1″ dstintfrole=”wan” proto=6 service=”HTTP” hostname=”www.fortinet.com” profile=”webfilter” action=”blocked” reqtype=”direct” url=”/” sentbyte=386 rcvdbyte=0 direction=”outgoing” msg=”URL belongs to a denied category in policy” method=”domain” cat=52 catdesc=”Information Technology”

Sample configuration of issuing a warning

This example shows issuing a warning when a user visits a website based on its category (rating), for example, information technology.

To configure a warning in the GUI:

1. Go to Security Profiles > Web Filter and go to the FortiGuard category based filter
2. Open the General Interest -Business section by clicking the + icon beside it.
3. Select Information Technology and then select Warning.
4. Set the Warning Interval which is the interval when the warning page appears again after the user chooses to continue.

To configure a warning in the CLI:

config webfilter profile edit “webfilter” config ftgd-wf unset options config filters edit 1 set category 52

set action warning  <– set action to warning

next

end

end

next end

To validate that you have configured the warning:

1. Go to a website belonging to the selected category, for example, www.fortinet.com, and you see a warning page where you can choose to Proceed or Go Back.

Sample configuration of authenticating a web category

This example shows authenticating a website based on its category (rating), for example, information technology.

To authenticate a category in the GUI:

1. Go to Security Profiles > Web Filter and go to the FortiGuard category based filter
2. Open the General Interest -Business section by clicking the + icon beside it.
3. Select Information Technology and then select Authenticate.
4. Set the Warning Interval which is the interval when the authentication page appears again after authentication.
5. Click the + icon beside Selected User Group and select a user group. You must have a valid user group to use this feature.

To authenticate a category in the CLI:

config webfilter profile edit “webfilter” config ftgd-wf

unset options

config filters edit 1

set category 52

set action authenticate         <– set the action of authenticate set auth-usr-grp “local_group”  <– user to authenticate

next

end end

next

end

To validate that you have configured authentication:

1. Go to a website belonging to the selected category, for example, www.fortinet.com. First, you see a warning page where you can choose to Proceed or Go Back.
2. Click Proceed to check that the authentication page appears.
3. Enter the username and password of the user group you selected, and click Continue.

If the credentials are correct, the traffic is allowed through.

Sample customization of the replacement page

When the FortiGuard webfilter action is Block, Warning, or Authenticate, there is a Customize option for you to customize the replace page.

To customize the replace page:

1. Go to Security Profiles > Web Filter and go to the FortiGuard category based filter
2. Right-click the item and select Customize.
3. A pane appears for you to customize the page.