Brochures

Benefits of Data Archiving in Data Warehouses

Description
Benefits of Data Archiving in Data Warehouses
Categories
Published
of 12
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Related Documents
Share
Transcript
  IBM Software White Paper Benefits of data archiving in data warehouses  2 Benefits of data archiving in data warehouses  Contents  2 Executive summary  3 Typical reasons for rapid data growth  4 Challenges associated with data warehouse growth  5 Traditional data growth solutions that do not work  6 Understanding data archiving 9 Benefits of data archiving  10 Guiding principles and technology requirements 11 Managing data growth responsibly with data warehouse archiving Executive summary Data warehouses are the pillars of business intelligence and analytics systems, often integrating data from multiple data sources in an organization to provide historical, current or even predictive analysis of the business. Information from multiple internal or external transactional systems is extracted, transformed and loaded into data warehouses as atomic data. This cumulative data and the analytics systems that leverage it provide the technology and methodology that help organizations discover and develop meaningful insights.Due to the consolidated nature of data warehouses, these data stores often suffer from rapid growth. Typical reasons for this phenomenon include expansion of data warehouses with new subject areas or data marts, compounded data growth from organic or inorganic business growth, or a “let’s keep it all, someone might need it” attitude toward historical data.  This unchecked data growth often results in ever-increasing infrastructure and operational costs, poor data warehouse performance, and an inability to support complex data retention and legal hold requirements. A data archiving solution helps organizations address these challenges by allowing IT staff to intelligently move (and purge) historical and inactive data from production databases into a more cost-effective location while still providing the capabilities to query, search or even restore data if needed.  A tiered archiving strategy provides additional benefits in terms of managing performance and cost-effectiveness. Data archiving can also alleviate data growth issues by: ã Removing or relocating inactive and dormant data out of the database to improve data warehouse performance ã Reducing the infrastructure and operational costs typically associated with data growth ã Leveraging proven policies and processes to cost-effectively manage multi-temperature data ã Improving disaster recovery and backup/restore plans to consistently meet service-level agreements (SLAs) ã Supporting compliance with data retention, purge or hold policies This paper describes a data lifecycle management strategy for data warehouses that is designed to manage high-volume data growth cost-effectively, and avoid performance degradation.  IBM Software  3 Typical reasons for rapid data growth  The data warehouse is commonly an organization’s largest database. This is due to several factors: Big data and the explosion in data volume:  With the advent of big data technologies that help organizations generate insight from large information assets, companies are keeping unstructured and structured data that might have been thrown away in the past. Apache Hadoop and similar technologies continue to gain momentum and adoption, and will provide new ways of processing large amounts of such data, extracting intelligence from multi-structured data sources, and integrating the results into existing data warehouses for further analysis and reporting.  The “data tomb” effect:  Data warehouses may become the dumping ground for historical data from various transactional systems, with little regard to the true value of the business intelligence within this dead data. This “data tomb” effect may be caused by the lack of an optimal archiving and data retention strategy in the srcinating transactional system itself. Expansion into new subject areas:  Companies frequently expand data warehouses with new subject areas and new data sources, making them part of a central repository for the enterprise or interconnected data marts. While this expansion can provide insights for crucial business activities, it can also lead to significant data expansion.  4 Benefits of data archiving in data warehouses  Business growth:  Larger organizations are often subject to compounded data growth from mergers and acquisitions, as  well as organic business growth. Consolidation of multiple implementations into one results in a larger system. Lack of retention and disposal policies:  Unfortunately, the business side of an organization may not provide IT teams  with enough clarity on data retention and disposal policies.  Most organizations have a “let’s keep it all, someone might need it later” mentality for historical data, which prevents them from exploring cost-effective data retention, hold or purge processes.Each of these factors provides an impetus for IT organizations to adopt data lifecycle management strategies and efficiently manage categories of data according to their value in a data  warehousing architecture. Challenges associated with data warehouse growth High-volume data growth and large warehouse implementations present multiple IT challenges and business risks. While many data warehouse solutions and architecture choices exist in the market, every approach poses several common challenges (see Figure 1). Cost of ownership  The impact of exponential data growth on infrastructure and operational costs can be huge, often taking up most of an organization’s data warehousing budget. Larger amounts of data require larger capacity, resulting in more hardware and storage requirements—as well as higher costs to maintain, monitor and administer this infrastructure. Large data warehouses generally require bigger servers and appliances, which may also increase software licensing costs for the database, database tooling, integration or business intelligence (BI) tools.  Figure 󰀱. Performance and capacity challenges associated with data warehouse growth. PerformanceHardware capacity     D  a   t  a   b  a  s  e  s   i  z  e
Search
Tags
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks