Query Result Caching in a Data Warehouse & Incremental Cache Update (T-0755) (MFN 3613)

Abstract

A data warehouse separates both the decision and operation layers, which allows the analysis, planning and strategic decision-making based on current as well as the historical data. Such an implementation allows using ad-hoc queries instead of having pre-defined reports. Since a large number of users may use a data warehouse, the probability is high that the same or even different users may submit the same query repetitively. This may result in repeated evaluation of large volumes of data. The query result caching in a data warehouse environment is adopted to reduce the response time by avoiding both the query re-evaluation and repeatedly huge data scans, which improves the efficiency of the data warehouse. With this technique, the query itself and the corresponding result-set are stored in a logical cache such that if the similar query is re-submitted, the results will be obtained from the cache instead of the data warehouse itself. In situations where the original data is updated, different techniques are adopted to ensure consistency between the data warehouse and the result-set in the cache by incrementally retrieving the updated data from the data warehouse. This research work pertains to propose a technique for the incremental update of query cache in the case of data warehouse updates. The proposed technique incrementally retrieves the newly inserted detailed data from the data warehouse and merges it with the already stored cached data. The experimental results of our proposed technique demonstrate that it reduces the query evaluation time and thus achieves a substantial performance improvement, when compared to the result-set retrieval through the non-cached evaluation.