FortiSIEM Real Time Performance Probe

Real Time Performance Probe

This section describes how to probe monitored devices for real time performance metrics.

Available metrics

GUI launch locations

Running a real time probe

Example – Real time Interface Statistics Display

Available metrics

CPU utilization

Memory utilization

Network interface statistics

Uptime

Disk utilization

SNMP Ping Statistics

Process Utilization

GUI launch locations

Real time Performance Metrics option is available from the following GUI locations

CMDB > Device > IP Address > Right click

CMDB > Device > Interfaces > Name > Right click

Incident > Incident Source and Incident Target > Right click

Running a real time probe

From any of the above locations, select Real Time Performance Metrics

Select the parameters

Select Job Name as the metric of interest

Select polling Frequency in seconds

Select the number of Runs as the number of times the device will be polled

Select the Collector which should communicate to the device

Depending on the job name, you may also need to select a Filter. For example, select Interface Name for Network Interface Statistics.

Example – Real time Interface Statistics Display

 

 

Using FortiSIEM Workbooks with Tableau Visual Analytics Desktop and Server

Using AccelOps Workbooks with Tableau Visual Analytics Desktop and Server

You can use any of the workbooks provided by AccelOps, which are attached to this page, to create visualizations of AccelOps data.

  1. Download a workbook attached to this page to your local device where Tableau Visual Analytics Desktop is installed.
  2. In Visual Analytics Desktop, go to File > Open….
  3. Browse to the file you downloaded and open it.
  4. You can make any changes you want to the workbook, but you can upload it to the server and start using it as is. Follow the instructions in the Publish the Workbook section of Creating a Single Sheet Workbook to publish to the Tableau Visual Analytics Sever, and add user permissions as described in Adding Users to Workbooks.

Drag and drop to upload or browse for files

Adding Users to Workbooks

Only the workbook publisher can give access to specific users during report creation time. As the AccelOps Visual Analytics Server Administrator, you can add users to the system and view which workbooks users can access.

Adding Users to Visual Analytics Server

Viewing User Access to Workbooks

Adding Users to Visual Analytics Server

  1. Log in to AccelOps Visual Analytics Server.
  2. In the Admin tab click Users.
  3. Click Add.
  4. Enter the user name as it appears in Active Directory.
  5. Select the License Level for the user and assign User Rights as necessary.
  6. Click OK.

Viewing User Access to Workbooks

  1. Log in to Visual Analytics Server.
  2. In the Admin tab click Users.
  3. Select a user name to see the workbooks that the user can access.

FortiSIEM Creating and Managing Workbooks

Creating and Managing Workbooks

This section contains information on using Visual Analytics Desktop to create sheets and workbooks that are based on AccelOps reports, and then publishing them for others to use.

Viewing Workbooks

Creating and Publishing Workbooks

Creating a Single Sheet Workbook

Creating a Multiple Sheet Workbook

Using AccelOps Workbooks with Tableau Visual Analytics Desktop and Server Adding Users to Workbooks

Viewing Workbooks
  1. Log in to Visual Analytics Server.
  2. Click the Content tab and select Workbooks.
  3. Click on a workbook.

The workbook along with the various worksheets are displayed.

  1. Select a workbook or worksheet.
  2. You will be prompted for credentials that will allow the workbook or worksheet to access database information. Enter the Admin credential that you used to set up Accelops Report Server and click OK.
  3. When your credential is accepted, the chart associated with the selected workbook or worksheet will be displayed.
Creating and Publishing Workbooks

Workbooks are collections of AccelOps reports that have been synced to AccelOps Report Server, and which are then the basis for charts and dashboards that can be published to Visual Analytics Server for access by other users. Information in this section describes how to create single and multiple sheets of report information, and then make them accessible to other users.

Creating a Single Sheet Workbook

Creating a Multiple Sheet Workbook

Using AccelOps Workbooks with Tableau Visual Analytics Desktop and Server

Creating a Single Sheet Workbook

These instructions demonstrate how to create a single-sheet workbook that will chart the CPU and memory utilization trend for various servers. This example uses the Servers by CPU, Memory report and its associated table, but any report with a table in the reportdb database can also be used. The Tableau Desktop online Help also contains extensive information about building sheets and workbooks with the Tableau Desktop editor, which powers the AccelOps Visual Analytics Desktop.

Prerequisites

Procedure

Create the Sheet

Create the Workbook

Publish the Workbook

Prerequisites

Follow the instructions in Syncing an AccelOps Report with Report Server to sync the report you want to use for your worksheet.

You will need to know the name of the parent table for your synced report. Follow the instructions in Viewing reportdb Organization to find the table that corresponds to your report.

Procedure

Create the Sheet

  1. Launch Tableau Visual Analytics Desktop.
  2. Connect to AccelOps Report Server with the Username and Password that you used during Report Server installation. For Database, enter reportdb. For Port, enter 30000.

Connecting to Port 30000

It’s important to make sure you enter the correct port to connect to the reportdb database. If you leave this option blank you will connect to the default PostgreSQL database port, which will connect you with phoenixdb instead of reportdb. For more information about the databases contained in Report Server, see Report Server Architecture: phoenixdb and reportdb.

  1. Under Tables, select the parent table for your report.

For the steps following, we will use the Servers by CPU, Memory table and its associated columns.

  1. Drag the table to the View pane and click Update Now.

The data in the table will load into the pane below. Note that the table columns match closely to the Report Display Columns in AccelOps.

  1. For Connection, select Live.
  2. Click Go to Worksheet.

In the worksheet view you will see that a set of Dimensions and Measures are populated for the table.

  1. Under Measures, select Report Time and drag it to the Dimensions section to create Report Time as a calculated measurement.
  2. Under Dimensions, right-click on Report Time to edit the calculation formula and convert it to a human-readable format from UNIX time. The formula should look like DATEADD(‘second’,INT([Report Time]),#1969-12-31 16:00:00#)
  3. Drag Report Time from Dimensions to Columns.
  4. Under Columns, right-click on Report Time and select Exact Date.

You should now see dates and time increments in your chart as the X-axis.

  1. Under Measures, select and drag AVG(cpuUtil) and AVG(memUtil) to Rows.
  2. Set the aggregation of both AVG(cpuUtil) and AVG(memUtil) to AVG. For example, AVG(AVG(cpuUtil)) and AVG(AVG(memUtil)).

You should now see both measures on the Y-axis of your chart.

  1. Under Dimensions, drag Host Name to the Color section under Marks. Each host will be assigned a color and added to the chart.
  2. Change the chart display name for AVG(cpuUtil) and AVG(memUtil) by clicking on each in the Y-axis to launch the Edit Y-Axis dialog.

You can now edit the Title and Range, as well as other attributes, for each measure.

  1. Under Data, click on the data source to open the Options menu, then click Refresh.
  2. Rename the sheet by clicking on the data source to open the Options menu, then select Rename and enter a new name.

Your sheet is now complete. Hover your mouse over a trend line to view information about a specific host.

Create the Workbook

  1. Click the Dashboard tab on the bottom of the Sheet editor to open the Dashboard
  2. Under Dashboard, select an appropriate Size and screen resolution.
  3. Under Dashboard, select the sheet and drag it into the display pane.
  4. Open the Dashboard options menu and select Rename.

Change the name of the dashboard from Server CPU/Memory Trend to Server Performance.

  1. In the File menu, select Save. Publish the Workbook
  2. In the Server menu, select Sign In…
  3. Enter the IP address and port number for the Visual Analytics Server.
  4. Enter the Username and Password for the Visual Analytics Server admin user, and then click Sign In.
  5. In the Server menu, select Publish Workbook.
  6. Enter attributes for the workbook, such the associated Project, Name, View Permissions, and Views to Share.

See Adding Users to Workbooks for more information about user permissions for workbooks.

  1. Click Publish.

Creating a Multiple Sheet Workbook

These instructions demonstrate how to create a multiple-sheet workbook that will contain a set of charts related to Network Health. This example uses the Network Devices by Ping RTT, Network Interfaces By Utilization, and Network Devices By CPU, Memory reports, but any report with an associated table and views in the reportdb database could be used. The Tableau Desktop online Help also contains extensive information about building sheets and workbooks with the Tableau Desktop editor, which powers the AccelOps Visual Analytics Desktop.

Prerequisites

Procedure

Create a View

Create a Workbook that Uses the View

Create the Workbook

Publish the Workbook

Prerequisites

Follow the instructions in Syncing an AccelOps Report with Report Server to sync the reports you want to use for your worksheet. You will need to know the name of the parent table for your synced reports. Follow the instructions in Viewing reportdb Organization to find the table that corresponds to your report.

Procedure

Create a View

Each report you want to include in your workbook corresponds to a table in the AccelOps reportdb. These tables need to be joined to cross-link the information that will appear in your workbook. In the case of a Network Health workbook that includes the sheets Network Devices by Ping RTT, Network Interfaces By Utilization, and Network Devices By CPU, Memory, the joining keys are host name and time.

  1. Follow the instructions in Viewing reportdb Organization to find the parent tables for the reports you want to join.

For each report there is one parent table and multiple child tables containing data for a particular month.

  1. Create a SQL statement in pgAdmin to join the tables.

In this example data is captured for one day. This enables quick generation of the data visualization.

SELECT cpu.report_time, cpu.”hostName”, cpu.”hostIpAddr”, cpu.”AVG(cpuUtil)”, cpu.”AVG(memUtil)”,        uptime.”SUM(sysDownTime)”, uptime.”AVG(avgDurationMSec)”, uptime.”LAST(sysUpTime)”,        uptime.”SUM(pollIntv)”, util.”intfName”, util.”intfAlias”,        util.”AVG(inIntfUtil)” AS “totalAvgInIntfUtil”, util.”AVG(outIntfUtil)” AS “totalAvgOutIntfUtil”,        util.”AVG(recvBitsPerSec)” AS “totalAvgRecvBitsPerSec”,        util.”AVG(sentBitsPerSec)” AS “totalAvgSentBitsPerSec”,        util.”AVG(outQLen)”, util.”AVG(intfSpeed64)”

FROM “Network Devices By CPU, Memory_1278492569_1” cpu,

“Network Devices by Ping RTT_2021056235_1” uptime,

“Network Interfaces By Utilization_382117475_1″ util

 

WHERE ((cpu.report_time * 1000)::double precision * ’00:00:00.001′::interval + ‘1969-12-31 16:00:00-08’::timestamp with time zone) >= (now() – 1::double precision * ‘1 day’::interval)      AND ((uptime.report_time * 1000)::double precision * ’00:00:00.001′::interval + ‘1969-12-31 16:00:00-08’::timestamp with time zone) >= (now() – 1::double precision * ‘1 day’::interval)      AND ((util.report_time * 1000)::double precision * ’00:00:00.001′::interval + ‘1969-12-31 16:00:00-08’::timestamp with time zone) >= (now() – 1::double precision * ‘1 day’::interval)      AND cpu.report_time = uptime.report_time AND cpu.”hostName” = uptime.”hostName” AND uptime.report_time = util.report_time AND uptime.”hostName” = util.”hostName”;

 

  1. Click the Play icon in pgAdmin to execute the query.

Make sure the output pane contains data that is the result of the query execution.

  1. Modify the SQL statement to create a view.

Add this command at the top of the SQL statement:

Add this command at the bottom of the SQL statement:

Your complete SQL statement should look like this:

CREATE OR REPLACE VIEW ph_network_health_view AS

SELECT cpu.report_time, cpu.”hostName”, cpu.”hostIpAddr”, cpu.”AVG(cpuUtil)”, cpu.”AVG(memUtil)”,        uptime.”SUM(sysDownTime)”, uptime.”AVG(avgDurationMSec)”, uptime.”LAST(sysUpTime)”,        uptime.”SUM(pollIntv)”, util.”intfName”, util.”intfAlias”,        util.”AVG(inIntfUtil)” AS “totalAvgInIntfUtil”, util.”AVG(outIntfUtil)” AS “totalAvgOutIntfUtil”,        util.”AVG(recvBitsPerSec)” AS “totalAvgRecvBitsPerSec”,        util.”AVG(sentBitsPerSec)” AS “totalAvgSentBitsPerSec”,        util.”AVG(outQLen)”, util.”AVG(intfSpeed64)”

FROM “Network Devices By CPU, Memory_1278492569_1” cpu,

“Network Devices by Ping RTT_2021056235_1” uptime,

“Network Interfaces By Utilization_382117475_1″ util

 

WHERE ((cpu.report_time * 1000)::double precision * ’00:00:00.001′::interval + ‘1969-12-31 16:00:00-08’::timestamp with time zone) >= (now() – 1::double precision * ‘1 day’::interval)      AND ((uptime.report_time * 1000)::double precision * ’00:00:00.001′::interval + ‘1969-12-31 16:00:00-08’::timestamp with time zone) >= (now() – 1::double precision * ‘1 day’::interval)      AND ((util.report_time * 1000)::double precision * ’00:00:00.001′::interval + ‘1969-12-31 16:00:00-08’::timestamp with time zone) >= (now() – 1::double precision * ‘1 day’::interval)      AND cpu.report_time = uptime.report_time AND cpu.”hostName” = uptime.”hostName” AND uptime.report_time = util.report_time AND uptime.”hostName” = util.”hostName”;

grant select on ph_network_health_view TO public;

 

  1. In pgAdmin, click the Play icon to execute the statement.
  2. Using pgAdmin, navigate to the Views and make sure the ph_network_health_view has been created.
  3. Right-click on ph_network_health_view to open the Options menu, then select View Data > View Last 100 Rows to make sure the view contains data.

Create a Workbook that Uses the View

  1. Launch AccelOps Visual Analytics Desktop.
  2. Connect to AccelOps Report Server with the Username and Password that you used during Report Server installation. For Database, enter reportdb. For Port, enter 30000.
  3. Under Tables, enter the name of the view you created in the search box to locate the view.

 

  1. Drag the view into the Join pane and click Update Now. The data in the view will load into the pane below.

 

  1. For Connection, select Live.
  2. Click Go to Worksheet.

In the worksheet view you will see that a set of Dimensions and Measures are populated for the view.

An example worksheet showing CPU and Memory Utilization with several Dimensions and Measures populated from the original table.

 

  1. For each report in your workbook you can now create an individual sheet, as described in Creating a Single Sheet Workbook.

Create the Workbook

  1. Click the Dashboard tab on the bottom of the Sheet editor to open the Dashboard
  2. Drag each sheet you’ve created into the Join

 

An example of three worksheets loaded into the Dashboard Join pane.

  1. Under Dashboard, select an appropriate Size and screen resolution.
  2. Open the Dashboard Options menu and select Rename.
  3. In the File menu, select Save. Publish the Workbook
  4. In the Server menu, select Sign In…
  5. Enter the IP address and port number for the Visual Analytics Server.
  6. Enter the Username and Password for the Visual Analytics Server admin user, and then click Sign In.
  7. In the Server menu, select Publish Workbook.
  8. Enter attributes for the workbook, such the associated Project, Name, View Permissions, and Views to Share.

See Adding Users to Workbooks for more information about user permissions for workbooks.

 

  1. Click Publish.

 

FortiSIEM Installing and Configuring Tableau Server

Installing and Configuring Tableau Server

Prerequisites

Installation

Activation

Configuration

 

Prerequisites

Before you begin installing Tableau Server, make sure you have read the section on Tableau Server in Requirements for Visual Analytics Report Server. This contains information on the Administrator Account and Ports that you will need during the configuration process. You may want to also consult the Tableau Server Administration Guide before you begin the installation process.

Installation
  1. Download the installation file from Tableau Software.
  2. Double-click the installation file to launch the Setup Wizard.
  3. When the Setup Wizard launches, click Next to begin the installation process.
  4. Enter a Destination Location where you want to install the server files, and then click Next.
  5. When the system verification process completes, click Next.
  6. Enter a location for the Start Menu folder, or use the default location, and then click Next.
  7. Click Install to complete the installation process.
  8. Click Next to begin the server activation process.
Activation
  1. If you are evaluating Tableau Server, click Start trial now. Otherwise, click Activate the product to enter a license key.
  2. If you enter a license key, click Activate.
  3. Click Continue to launch the Tableau Server configuration process.
Configuration
  1. In the Configuration dialog, enter a User Name and Password for the domain admin account that you will use to administer the Tableau Server.
  2. If necessary, enter a Gateway port through which you will connect to the server over HTTP.
  3. Click OK.

The initialization process will launch and complete within several minutes.

  1. Click Finish to complete the configuration process.
  2. Launch the Tableau Server user interface by entering the URI for the server in a browser window.

The URI will be be in the format of http://<Windows_Server_IP_Address>:<Port_Number_Used_In_Step_2> 6.  Sign in to the server by entering the credentials for the domain admin account that you created in Step 1, and then click Sign In.

  1. Click the Admin tab and select Maintenance.
  2. Under Status, check to make sure that all systems are up and running.

You are now ready to install Tableau Desktop. After you have completed the Desktop installation process and connect to Report Server for the first time to create a sheet, as described in Creating a Single Sheet Workbook, you will also establish the connection between AccelOps Report Server and Tableau Server.

 

 

FortiSIEM Working with the Report Server

Working with the Report Server

This section contains information on AccelOps Report Server architecture, viewing and querying CMDB and Event data in contained in the Report Server databases, and database maintenance.

Report Server Architecture: phoenixdb and reportdb

Working with CMDB Data in AccelOps Report Server

Viewing phoenixdb Organization

Querying Incident Data in AccelOps Report Server

Reference: Attribute Columns in the ph_incident_view Table

Sample Incident Queries

Querying Other CMDB Tables in AccelOps Report Server

Querying Device Vendor and Model Distribution for Discovered Devices Querying Discovered Devices

Working with Event Data in AccelOps Report Server

Viewing reportdb Organization

Syncing an AccelOps Report with Report Server

Deleting a Report from AccelOps Report Server

Modifying an Existing Report in AccelOps Report Server

Report Server Architecture: phoenixdb and reportdb

AccelOps Report Server contains two databases:

phoenixdb

This database contains the entire AccelOps CMDB and is populated via asynchronous PostgreSQL replication (slony) in near-real time.

reportdb

This database contains the results of event queries.

Topics in this section describe how to view the tables in these databases, and how those tables are organized. For viewing the tables, we recommend using the pgAdmin PostgreSQL database utility, which you can download from the pgAdmin website.

Working with CMDB Data in AccelOps Report Server

Data from the AccelOps CMDB database is populated to the AccelOps Report Server and stored in the Report Server phoenixdb. This section contains information on how to view the organization of phoenixdb, and write queries against the data it contains.

Viewing phoenixdb Organization

Querying Incident Data in AccelOps Report Server

Reference: Attribute Columns in the ph_incident_view Table Sample Incident Queries

Querying Other CMDB Tables in AccelOps Report Server

Querying Device Vendor and Model Distribution for Discovered Devices Querying Discovered Devices

Viewing phoenixdb Organization

This database contains the contents of the entire AccelOps CMDB database, including incidents.

  1. In the pgAdmin utility, go to File > Add Server.
  2. In the New Server Registration dialog, enter connection details for AccelOps Report Server.

For Maintenance DB, select phoenixdb.

For Username and Password, use the read-only user name and password that you created when you provisioned the Report Server.

  1. Click OK.

When the connection to the AccelOps Report Server is established, phoenixdb will load in the Object browser. There are approximately

197 tables in phoenixdb, which are replicated from the AccelOps cluster.

  1. Select a table to view, then right-click to open the Options
  2. In the Options menu, select View Data, and then select an option for which rows you want to view.

For example, to view the contents of the ph_device table, which contains CMDB information about discovered devices, you would select and then right click on ph_device, then select View Data > View All Rows.

You can also use this method to examine Views and other objects in the phoenixdb database.

Querying Incident Data in AccelOps Report Server

There are two ways to look at the incident data inside AccelOps Report Server:

Incident Tables (ph_incident and ph_incident_detail)

Contains the incidents

Incident View (ph_incident_view)

This is a database view that adds other context to the incident tables by joining with other tables in the database. Added information includes location and business service. Some information is parsed out for easier query, such as host names and IP address fields from incident_source, and incident_target fields in ph_incident are parsed out as separate fields in ph_incident_view.

This topic describes how to view the data contained in Incident View.

  1. Follow the instructions in Viewing phoenixdb Organization to access the phoenixdb database in AccelOps Report Server.
  2. Go to Views > ph_incident_view > Columns to view the table columns.
  3. Go to Views > ph_incident_view > View Data > View Last 100 Rows to view the incidents.

 

Reference: Attribute Columns in the ph_incident_view Table

Column Name Format Description
incident_id integer Unique id for an incident
cust_org_id integer Customer Id (for AO-SP)
first_seen_time integer The time when the incident was first seen. The format is UNIX time but with milliseconds granularity. It is defined as the number of milliseconds that have elapsed since 00:00:00 Coordinated Universal Time

(UTC), Thursday, 1 January 1970    

last_seen_time integer The time when the incident was last seen. The format is UNIX time but with milliseconds granularity. It is defined as the number of milliseconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970
incident_et string Incident event type id e.g. PH_RULE_SERVER_HW_CRITICAL
incident_status integer 0: Active 1: Auto Cleared 2: Manually Cleared 3: System Cleared
incident_count integer The number of times this exact incident (with the same parameters: source, destination etc has happened)
biz_name string  Associated business service name
severity integer  Numerical severity of the incident – range 0-10
severity_cat string  Incident severity category: 0-4: LOW, 5-8: MEDIUM and 9-10: HIGH
orig_device_ip string  IP address of the device that reported the incident
ph_incident_category  string  Category of infrastructure affected by this incident: possible valies: Network, Server, Storage, Virtualization, Application, Internal
incident_src string Incident Source string formatted as a list of <Attribute>:Value; e.g.

srcIpAddr:10.1.1.1,srcName:JoeLaptop

src_ip_addr string Source IP parsed out from incident_src field
src_name string Source Name parsed out from incident_src field
src_device_location string (Geo) Location display name string for the object specified in incident_src
src_country string (Geo) Country name string for the object specified in incident_src
src_state string (Geo) State name for the object specified in incident_src
src_building string (Geo) Building name for the object specified in incident_src
src_floor string (Geo) Floor for the object specified in incident_src
src_latitude double (Geo) Latitude for the object specified in incident_src
src_longitude double (Geo) Latitude for the object specified in incident_src
incident_target string Incident Destination string formatted as a list of <Attribute>:Value; e.g.

“destIpAddr:10.1.1.1,destName:JoeLaptop” or “hostIpAddr:10.1.1.1,hostName:JoeLaptop”

dest_ip_addr string Destination IP parsed out from incident_target field
dest_name string Destination Name parsed out from incident_target field
dest_device_location string (Geo) Location display name string for the object specified in incident_target
dest_country string (Geo) Country name string for the object specified in incident_target
dest_state string (Geo) State name for the object specified in incident_target
dest_building string (Geo) Building name for the object specified in incident_target
dest_floor string (Geo) Floor for the object specified in incident_target
dest_latitude double (Geo) Latitude for the object specified in incident_target

 

dest_longitude double (Geo) Longitude for the object specified in incident_target
host_ip_addr string Host IP address parsed out from incident_target field
host_name string Host Name parsed out from incident_target field
host_device_location string (Geo) Location display name string for the object specified in incident_target – populated if incident_target contains hostIpAddr
host_country string (Geo) Country name string for the object specified in incident_target – populated if incident_target contains hostIpAddr
host_state string (Geo) State name for the object specified in incident_target – populated if incident_target contains hostIpAddr
host_building string (Geo) Building name for the object specified in incident_target – populated if incident_target contains hostIpAddr
host_floor string (Geo) Floor for the object specified in incident_target – populated if incident_target contains hostIpAddr
host_latitude double (Geo) Latitude for the object specified in incident_target – populated if incident_target contains hostIpAddr
host_longitude double (Geo) Longitude for the object specified in incident_target – populated if incident_target contains hostIpAddr
vm_name string VM Name if incident involves a Virtual machine – populated if incident_target contains vmName
user_attr string User name if incident involves user, i.e. incident_target contains user
target_user_attr string Target user name if incident involves user, i.e. incident_target contains targetUser
ldap_domain string Domain if incident involves user, i.e. incident_target contains domain
computer string Computer name incident_target contains computer
target_computer string Target Computer name incident_target contains targetComputer
incident_details string Incident Details containing evidence on why the incident triggered e.g. Triggered Event Count = 90 or AVG(CPUUtil) = 90 etc

Sample Incident Queries

Show Incident Categories with Severity and Frequency Occurrence Show Incident Location

Show Incident Categories with Severity and Frequency Occurrence

This query will show which parts of the infrastructure are triggering events.

  1. Follow the instructions in Viewing phoenixdb Organization to access the phoenixdb in AccelOps Report Server.
  2. Under Views, select ph_incident_view.
  3. In pgAdmin, click on the SQL icon in the menu bar to open the SQL query window.
  4. Enter this SQL query:

 

SELECTph_incident category, incident_et, severity_cat, src_ip_addr, host_name, FROMph_incident_view

GROUPBYph_incident category, incident_et, severity_cat, src_ip_addr, host_name

ORDERBYCOUNT(*) DESC;

COUNT(*)

  1. When the query executes, you will see a list of matching incidents in the Output Pane. Show Incident Location
  2. Follow the instructions in Viewing phoenixdb Organization to access the phoenixdb in Accelops Report Server.
  3. Under Views, select ph_incident_view.
  4. In pgAdmin, click on the SQL icon in the menu bar to open the SQL query window.
  5. Enter this SQL query:

 

COUNT(*)

SELECT host_device_location, severity_cat, ph_incident_category, FROM ph_incident_view

GROUP BY host_device_location, ph_incident_category, severity_cat

ORDER BY host_device_location ASC, severity_cat ASC COUNT

(*) DESC;

  1. When the query executes, you will see a list of incidents and their locations in the Output Pane.

Querying Other CMDB Tables in AccelOps Report Server

Querying Device Vendor and Model Distribution for Discovered Devices Querying Discovered Devices

Querying Device Vendor and Model Distribution for Discovered Devices

Querying Discovered Devices

Working with Event Data in AccelOps Report Server

Data from the AccelOps EventDB database is populated to the AccelOps Report Server and stored in the Report Server reportdb. This section contains information on how to view the organization of reportdb, and write queries against the data it contains.

Viewing reportdb Organization

Syncing an AccelOps Report with Report Server

Deleting a Report from AccelOps Report Server

Modifying an Existing Report in AccelOps Report Server

Viewing reportdb Organization

This database contains the reports that are synched from the AccelOps cluster.

  1. In the pgAdmin utility, go to File > Add Server.
  2. In the New Server Registration dialog, enter connection details for AccelOps Report Server.

For Maintenance DB, select reportdb.

For the Port enter 30000 (default port used for the reported).

For Username and Password, use the read-only user name and password that you created when you provisioned the Report Server.

  1. Click OK.

When the connection to the Report Server is established, reports will load in the Object browser.

  1. Select a table to view, then right-click to open the Options
  2. In the Options menu, select View Data, and then select an option for which rows you want to view.

Syncing an AccelOps Report with Report Server

  1. Log in to AccelOps.
  2. Go to Analytics > Reports.
  3. Select a report.

Any reports with a Sync checkbox can be synced. Run the report to make sure it contains some data.

  1. For each report you want to sync, select the Sync
  2. Click OK.
  3. After several minutes, follow the instructions in Viewing reportdb Organization to view the reportdb database.
  4. Under Tables, you should now see the synced reports.

Table Structure for Synced Reports

When you sync an AccelOps report to AccelOps Report Server, two pairs of tables are created in reportdb, one pair for each organization in the case of AO-SP. For each organization, multiple tables are created:

  1. A parent table containing data for all months: the table name is of the form <Report Name>_<ID>_<custId>
  2. A child table for the current month: <Report Name>_<ID>_<custId>_<yYYYYmMM> where YYYY is the year and MM is the month.

Queries should be written using the parent table. To see data in the parent table, follow the instructions in  Viewing reportdb Organization . The re portdb database fields are generated from the display fields in AccelOps report definitions. Only the field report_time is added to the Report Server table definitions to capture the time when the particular report is generated. For example, if you synced the report Network Devices by CPU, Memory, you would see these fields:

Field Description
report_time UNIX time at which the report is generated. Unix time (or POSIX time or Epoch time) is a system for describing instants in time, defined as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970 not counting leap seconds.
hostName Host Name of the device for which CPU and memory are being measured
hostIpAddr Access IP of the device for which CPU and memory are being measured
AVG(cpuUtil) Average of all the CPU utilization metrics within the last 5 minutes ending with report_time
AVG(memUtil) Average of all the CPU utilization metrics within the last 5 minutes ending with report_time

Deleting a Report from AccelOps Report Server

  1. Log in to AccelOps.
  2. In Analytics > Reports > Synced Reports, select the report you want to delete.
  3. In the Sync Details dialog, clear the Sync option for the report, and then click OK.

The report will no longer be synced with Report Server. You can verify this by making sure the Sync option is not selected for the report on the Analytics > Reports > Synced Reports page. You can now delete the report from AccelOps Report Server.

  1. Log in to AccelOps Report Server via SSH and navigate to the directory /opt/phoenix/deployment/jumpbox.
  2. Run the py command, along with the table name and date as arguments, to delete the report.
  3. After you have deleted the table containing the report information, you will need to delete the parent table, which will now be empty of content, using the same py command.

Modifying an Existing Report in AccelOps Report Server

Suppose a system report is synced and exported to AccelOps Report Server. When you modify that report in AccelOps, you must rename it, at which point it becomes a user report. When you then sync that report for Accelops Report Server, a new table is created on the AccelOps Report Server.

Suppose now that you have a user-defined report that is already synced to the AccelOps Report Server, but you modify it inline in AccelOps, which means that you have changed the report conditions without changing the report name. This will cause a change in the table, but a new table will not be created. Here are some examples of inline modifications, and how they affect the structure of the table as well as the data collected in the table:

Modification Effect
GROUP BY field added The corresponding table has the new GROUP BY field, but only newer data populates the field
GROUP BY field removed There is no change in the corresponding table, and newer data does not populate the field
GROUP BY field changed For example, the field srclpAddr is changed to destlpAddr. Both fields are retained, but newer data populates d estlpAddr.
Aggregated fields added The corresponding table has the new field, but only newer data populate that field
Aggregated field removed There is no change in the corresponding table, and newer data does not populate the field
Aggregated Field

Changed

For example, AVG(cpUutil) is changed to MAX(cpuUtil). Both fields are retained, but newer data populates MA X(cpuUtil).

 

 

FortiSIEM Registering Report Server

Registering Report Server

  1. In the Admin tab, select License Management.
  2. Under Report Server Information, click Add.
  3. Enter the Report Server IP Address, and the Database Username and Password you want to use to administer Report Server.

These are also the credentials that you will use when you set up the Visual Analytics Server to read data from Report Server.

  1. Click Run in Background if you want Report Server registration to run in the background for larger installations.

When CMDB size is under 1GB, registration takes approximately 3 minutes to complete.

  1. When registration completes, click OK in the confirmation dialog.
  2. Under the Admin tab, select Cloud Health and make sure Report Server is up and running.
Syncing with the Report Server

Using AccelOps Visual Analytics involves first syncing reports contained in the primary AccelOps application to the AccelOps Report Server.

  1. Log in to your Supervisor node.
  2. Go to Analytics > Reports > Synced Reports.
  3. Select a report.

Currently only reports that contain a Group By condition can be synced. Both system and user-created reports can be synched as long as the contain a Group By condition.

  1. Select Sync.

When the sync process initiates, the Supervisor node dynamically creates a table within the Report Server reportdb database. When the sync is established, it will run every five minutes, and the last five minutes of data in the synced report will be pushed to the corresponding table. This lets you run Visual Analytics on event data stored in the Report Server reportdb database.

FortiSIEM Hypervisor Installations for Report Server

Hypervisor Installations for Report Server

These topics cover the installation of Report Server in various hypervisor enviroments.

Installing and Registering AccelOps Report Server in Amazon Web Services

Installing and Registering AccelOps Report Server in KVM

Installing and Registering AccelOps Report Server in Microsoft Hyper-V Installing and Registering AccelOps Report Server in VMware ESX

Installing and Registering AccelOps Report Server in Amazon Web Services

Follow the instructions for setting up an AccelOps virtual appliance as described in Setting Up Supervisor, Worker and Collector Nodes in AWS, and then register the Report Server to the Supervisor as described in Installing and Registering AccelOps Report Server in VMware ESX.

Turn on archive mode for Report server CMDB replication

  1. Mount a NFS shared directory on both Super and report server and make sure that this mount can survive system reboot. For example:

/data/replication/archive

  1. Make this shared directory own by postgres.postgres
  2. On Super, edit postgresql.conf under /cmdb/data to turn on archive mode by uncommenting (removing # in the first column) the following lines and make sure archive_command points to the correct directory which is created in step 1
  3. On Report Server, edit /cmdb/data/recovery.conf and uncomment the following lines and make sure restore_command and archive_cleanup_command are pointing to the directory created in step 1:
  4. On Super, restart postgresql DB ‘service postgresql-9.1 restart’
  5. On Super, restart App Server (Glassfish)
  6. On Report Server, restart postgresql DB ‘service postgresql-9.1 restart’

Registering Report Server

  1. In the Admin tab, select License Management.
  2. Under Report Server Information, click Add.
  3. Enter the Report Server IP Address, and the Database Username and Password you want to use to administer Report Server.

These are also the credentials that you will use when you set up the Visual Analytics Server to read data from Report Server.

  1. Click Run in Background if you want Report Server registration to run in the background for larger installations.

When CMDB size is under 1GB, registration takes approximately 3 minutes to complete.

  1. When registration completes, click OK in the confirmation dialog.
  2. Under the Admin tab, select Cloud Health and make sure Report Server is up and running.

Installing and Registering AccelOps Report Server in KVM

Follow the instructions for installing an AccelOps virtual appliance as described in Importing a Supervisor, Collector, or Worker Image into KVM, and then register the Report Server with the Supervisor as described in Installing and Registering a Report Server Node in ESX.

Turn on archive mode for Report server CMDB replication

  1. Mount a NFS shared directory on both Super and report server and make sure that this mount can survive system reboot. For example:

/data/replication/archive

  1. Make this shared directory own by postgres.postgres
  2. On Super, edit postgresql.conf under /cmdb/data to turn on archive mode by uncommenting (removing # in the first column) the following lines and make sure archive_command points to the correct directory which is created in step 1
  3. On Report Server, edit /cmdb/data/recovery.conf and uncomment the following lines and make sure restore_command and archive_cleanup_command are pointing to the directory created in step 1:
  4. On Super, restart postgresql DB ‘service postgresql-9.1 restart’
  5. On Super, restart App Server (Glassfish)
  6. On Report Server, restart postgresql DB ‘service postgresql-9.1 restart’

Registering Report Server

  1. In the Admin tab, select License Management.
  2. Under Report Server Information, click Add.
  3. Enter the Report Server IP Address, and the Database Username and Password you want to use to administer Report Server.

These are also the credentials that you will use when you set up the Visual Analytics Server to read data from Report Server.

  1. Click Run in Background if you want Report Server registration to run in the background for larger installations.

When CMDB size is under 1GB, registration takes approximately 3 minutes to complete.

  1. When registration completes, click OK in the confirmation dialog.
  2. Under the Admin tab, select Cloud Health and make sure Report Server is up and running.

Installing and Registering AccelOps Report Server in Microsoft Hyper-V

Follow the virtual appliance installing instructions in Installing in Microsoft Hyper-V, and then register the Report Server node with the Supervisor as described in Installing and Registering AccelOps Report Server in VMware ESX.

Turn on archive mode for Report server CMDB replication

  1. Mount a NFS shared directory on both Super and report server and make sure that this mount can survive system reboot. For example:

/data/replication/archive

  1. Make this shared directory own by postgres.postgres
  2. On Super, edit postgresql.conf under /cmdb/data to turn on archive mode by uncommenting (removing # in the first column) the following lines and make sure archive_command points to the correct directory which is created in step 1
  3. On Report Server, edit /cmdb/data/recovery.conf and uncomment the following lines and make sure restore_command and archive_cleanup_command are pointing to the directory created in step 1:
  4. On Super, restart postgresql DB ‘service postgresql-9.1 restart’
  5. On Super, restart App Server (Glassfish)
  6. On Report Server, restart postgresql DB ‘service postgresql-9.1 restart’

Registering Report Server

  1. In the Admin tab, select License Management.
  2. Under Report Server Information, click Add.
  3. Enter the Report Server IP Address, and the Database Username and Password you want to use to administer Report Server.

These are also the credentials that you will use when you set up the Visual Analytics Server to read data from Report Server.

  1. Click Run in Background if you want Report Server registration to run in the background for larger installations.

When CMDB size is under 1GB, registration takes approximately 3 minutes to complete.

  1. When registration completes, click OK in the confirmation dialog.
  2. Under the Admin tab, select Cloud Health and make sure Report Server is up and running.

Installing and Registering AccelOps Report Server in VMware ESX

These instructions are for installing Report Server on VMWare ESX, and assume that you have already installed and configured an AccelOps environment. For instructions for a complete AccelOps install, see the topics under Installation.

Installing Report Server

Follow the instructions for installing an AccelOps virtual appliance as described in the topics under Installing a Supervisor, Worker, or Collector Node in ESX and in Configuring the Supervisor, Worker, or Collector from the VM Console.

Turn on archive mode for Report server CMDB replication

  1. Mount a NFS shared directory on both Super and report server and make sure that this mount can survive system reboot. For example:

/data/replication/archive

  1. Make this shared directory own by postgres.postgres
  2. On Super, edit postgresql.conf under /cmdb/data to turn on archive mode by uncommenting (removing # in the first column) the following lines and make sure archive_command points to the correct directory which is created in step 1
  3. On Report Server, edit /cmdb/data/recovery.conf and uncomment the following lines and make sure restore_command and archive_cleanup_command are pointing to the directory created in step 1:
  4. On Super, restart postgresql DB ‘service postgresql-9.1 restart’
  5. On Super, restart App Server (Glassfish)
  6. On Report Server, restart postgresql DB ‘service postgresql-9.1 restart’

FortiSIEM Installation and Configuration of AccelOps Visual Analytics

Installation and Configuration of AccelOps Visual Analytics

Installation and configuration of AccelOps Visual Analytics involves setting up AccelOps Report Server, and then integrating it with Tableau Server and Desktop from Tableau Software. Topics in this section contain setup and configuration instructions for Report Server. For information on setting up and configuring Tableau Server and Desktop, see the online Tableau Software documentation.

Requirements for Visual Analytics Report Server

Setting Up Visual Analytics

Hypervisor Installations for Report Server

Syncing with the Report Server

 

Requirements for Visual Analytics Report Server

You install Visual Analytics Report Server as an AccelOps node, and these requirements assume that you have already set up and installed AccelOps. If you are working with a fresh install of AccelOps that includes Report Server, see the topics under Installation for complete requirements and installation instructions for the AccelOps Virtual Appliance.

Hardware Requirements for Report Server Nodes

Component Quantity Host

SW

Processor Memory OS/App Storage Reports Data Storage (1 year)
Report

Server

1 ESX 8 Core 3

GHz, 64 bit

16 GB 200GB (80GB OS/App, 60GB

CMDB, 60GB SVN)

See recommendations under Hardware Requirements for

Supervisor and Worker nodes

 

 

Setting Up Visual Analytics

There are three components to Accelops Visual Analytics:

  1. Accelops Report Server
  2. Tableau Server
  3. Tableau Desktop

Setting up Visual Analytics involves setting up each of those components in order, and establishing the relationship between them.

  1. You must first install Report Server as described in Installing and Registering AccelOps Report Server in VMware ESX.
  2. After installing Tableau Server on a Windows server, and installing Tableau Desktop on a Windows or Mac OS X device, you then connect the two systems as described in the Tableau Software product documentation.
  3. When this connection is established, it automatically triggers the remote registration and configuration of the AccelOps Report Server, including replication of the CMDB and EventDB data from the AccelOps Cluster to the AccelOps Report Server, as well as the user account required for access to the original databases.

Registration of the Report Server and replication of the AccelOps database data may take some time depending on the size of the original CMDB. Registration is complete when the replication process catches up with the latest data in the system. From that point on, replication from the CMDB to Accelops Report Server takes place in near real time, letting you run Visual Analytics queries against CMDB data that has been replicated to the Report Server’s phoenixdb.

You can find full information about setting up all components of AccelOps Visual Analytics in the section Installation and Configuration of

AccelOps Visual Analytics