How to handle deleted records in Incremental data refresh?
When using incremental updates for your data source in Bold BI, it is essential to understand how deleted records are handled. This article will explain the limitations of incremental updates and provide an alternative solution for managing deleted records in the Bold BI database.
Incremental Updates
Incremental updates in Bold BI are designed to add or update rows based on the last modified column. This approach is efficient for large data sets, as it only processes the changes since the last update. However, incremental updates do not handle deleted records from the source data. As a result, deleted records will remain in the Bold BI database even after an incremental update.
Full Load Refresh
To ensure that deleted records are also removed from the Bold BI database, a Full Load Refresh is recommended. A Full Load Refresh will replace the entire data set in the Bold BI database with the current data from the source. This process ensures that any deleted records are removed from the Bold BI database, as they will not be included in the new data set.
While Full Load Refreshes can be more time-consuming than incremental updates, they provide a more accurate representation of the current data, including the removal of deleted records.
Conclusion
In summary, incremental updates in Bold BI do not handle deleted records. To ensure that deleted records are removed from the Bold BI database, a Full Load Refresh is recommended. By using a Full Load Refresh, you can maintain an accurate representation of your data, including the removal of deleted records.
Additional References
