Troubleshooting Scenario

Because some clients request that we delete their Organization's historical data regularly, we have developed a script in Datadog to automatically delete data for a specified Organization at a configurable time interval. For clients that request regular data deletion, it is important that we maintain evidence that the automated task runs successfully over time.

This article describes the process of searching Datadog logs to confirm historical data has been deleted.

Confirm Data Deletion in Datadog Logs

You can confirm that an Organization's historical data has been deleted by searching on the Logs page.

Searching Datadog Logs

1. In the Datadog UI, click Logs > Search.
2. Use the environment tag to specify the Callback instance, if needed.
3. Enter "DeleteOldDataJob" in the search field.
4. Update the time/duration filter near the top-right of the page to include the relevant timeframe, if needed.
5. Export the search results to CSV.
6. Search the exported results to find the following line with the relevant Organization name: Finished deleting data from organization: '<organization name>'

You should find a log statement similar to the following, indicating the data was deleted successfully:

I, [2021-12-04T21:00:22.208614 #11636] ?INFO -- : ******* Success! ******* DeleteOldDataJob::SubJob (JID: d01d39c0dff2a237b2043886) Finished deleting data from organization: 'Load Test' with ID: 7

Additional Log Examples

You can see additional example log statements below, showing the complete process.

Performing DeleteOldDataJob::SubJob (JID: d01d39c0dff2a237b2043886) with arguments: client_id: 7

...

I, [2021-12-04T21:00:22.208614 #11636] ?INFO -- : ******* Success! ******* DeleteOldDataJob::SubJob (JID: d01d39c0dff2a237b2043886) Finished deleting data from organization: 'Load Test' with ID: 7

I, [2021-12-04T21:00:22.208666 #11636] ?INFO -- : Performed DeleteOldDataJob::SubJob (JID: d01d39c0dff2a237b2043886)