Overview
The LogiSense Billing application has a built in Data Archival data archival tool which removes old data from the system to prevent its database from growing too large and prevents prevent certain tables from becoming bloated.
Data that is Archived archived is sent to an S3 bucket and organized by day and target (targets are explained in more detail below).
Archived Data and Targets
The table below details data archive targets and our recommended settings for them. These settings can be modified per target and customer, pending approval. Our if required. The general default is to store data in the Prod a production system for 3 months.
| Info |
|---|
Note: There are certain targets that cannot have their Archive Type archive type changed (for example, API Logs currently can’t be Archivedarchived, they must be Purged, for examplepurged). |
Release | Target | Description | Archive Type and Duration | Reason |
|---|---|---|---|---|
10.4.0 | Usage | Data relating to Usage Records (Rated, Mediated, Extensions, Aggregates). | Archive & Purge - 3 Months | Usage Records that were processed could be audited at a later date, by auditors or customer disputes. |
10.4.0 | Unprocessed Usage Exceptions | The usage exceptions that have not been resolved. | Purge - 3 Months | If the exceptions haven’t been dealt with after 3 months, they probably aren’t going to be resolved or charged. |
10.4.0 | Processed Usage Exceptions | Usage Exception records that have been sent for Rerating. | Purge - 1 Day | These would normally get deleted once sent for reprocessing. |
10.4.0 | Rerated Usage Records | Original Usage Records that have been sent in for Rerating, making the original record obsolete. | Purge - 1 Day | These would normally get deleted once sent for reprocessing. |
10.4.0 | API Logs | The request/response API logs from every API call made to the system. | Purge - 5 days | There are many API calls made every day in the system and this table fills up quickly. This data also is rarely referenced again. |
10.4.0 | Tax Logs | The Tax Logs from every tax calculation made in the system. | Archive & Purge - 3 Months | Tax calculations are audited annually and could be challenged on any invoice. |
10.4.0 | Webhook Logs | The response data from every webhook sent from the system. | Purge - 3 Months | These generally are not required once the change has been made or the action has completed. |
10.5.0 | Data Import Tracking and Batch details | Usage API and Data Import Tracking details | Purge - 3 Months | Once imported, this data is used by the system from the UDRMediated and UsageRated tables. The import details are no longer used by the system. |
...
Monitoring Data Archival Activity
...
Within the LogiSense Billing AdminPortal data archiving activity can be tracked by loading the Operations / Monitor / Data Archive screen.
...
All Archive data archive operations are by default configured to run for a max maximum of six hours, if the . If an operation takes more than six hours it will be cancelled and listed as Cancelled on this screen.
Downloading Archived Data from S3
When data is successfully archived to an S3 bucket it will be listed as Completed on the Data Archive screen and the archived files will be listed under the Files tab. From this tab you can download the archived files if you are required to keep them longer than they will be stored in the S3 bucket.