Creating Event Database Archives

Online v. Offline Storage

Setting Purge and Archive Policies

Archive and Purge Alerts

Online v. Offline Storage

The FortiSIEM event database, eventDB, is for near-to-intermediate term storage and querying of events. As an online database, eventDB has fast query performance, but this performance comes with a limited storage capacity, and is expensive in terms of resource consumption. For these reasons, data needs to be periodically purged from eventDB and moved into offline storage, but still be available for querying for forensic analysis. FortiSIEM checks the capacity of the online EventDB storage every 30 minutes, and when approaches capacity, begins to move event information, in daily increments, into the offline storage location.

The FortiSIEM virtual appliance includes a data archiving function that enables you to define an offline storage location, and a policy for the number of days that events will be kept in online or offline storage. This archiving function also includes the ability for compliance auditors to validate logs to ensure that they haven’t been tampered with in the offline storage. The data is cryptographically signed (SHA256) at the point of entry, and the checksums are stored in the database. The check sums can be re-verified on demand at any point of time, and if the data has been tampered with, then the check sums will not match. The data integrity reports can be exported in PDF format. If the events in offline storage need to be queried at some point in the future, they can be restored to the FortiSIEM virtual appliance.

Setting Purge and Archive Policies

Online data is only moved to the archive location when online storage reaches capacity. When you set the archive policy as described in Managin g Event Data Archive, you are setting the amount of time that archived data will be retained before it is purged. For example, if you set the Data Management Policy for your deployment or an organization to 90 days, then maintenance will run every day to purge data that is over 90 days old. If there is not enough offline storage for 90 days, then archived events will be purged from offline storage to create more capacity. If there is enough storage for the 90 days, then events will only be purged after 90 days. For this reason it is very important that you set an archive location that has sufficient capacity to store the amount of data for the number of days that you specify.

For multi-tenant deployments, you can set archive policies for each organization. If one organization requires 30 days of storage, and another customer requires 90 days of storage, then FortiSIEM will attempt to enforce these policies in relation to the amount of storage available. For the first organization, events will be deleted from the archive storage location on the 31st day to free up capacity for the organization that has longer storage requirements.

As with the online EventDB data, every 30 minutes FortiSIEM will check the capacity of the offline archive storage, and when the remaining storage capacity reaches a 20GB threshold, it will begin to purge data from the archive location, beginning with the oldest data, and purging it in daily increments, until the remaining storage capacity is above 20GB.

Archive and Purge Alerts

There are several system alerts that are related to eventDB capacity and the archiving function:

Managing Event Data Archive

Managing Online Event Data

Restoring Archived Data Validating Log Integrity