 |
| Importance Of DW Modernization | hashedin.com |
DW modernization is truly happening.
Irrespective of what strategy is in play in your DW modernization approach, the real
truth is that it all requires substantial modifications to the logical and
systems architectures of the extended data warehouse environment and there are
many commercial and open-source options for that.
Overview
Existing enterprise data warehouses are now bursting at the
seam choked by the weightiness of new structured and unstructured data sources.
As with most data-driven
platforms, inadequate organizational support limit Data
Warehouse Modernization. Success in modernization depends on the team,
which may suffer if technical deficiencies due to inadequate staffing for data
warehousing and its related disciplines, insufficient proficiencies on new
technologies and procedures, or the lack of experience with big new data types
and analytics.
Funding for modernization could be restrained by the cost of
implementing the latest technologies and hardware and software upgrades.
Modernization could be threatened by the poor quality of data or metadata
whether it is focused on big new data, traditional enterprise data, or both.
Utilizing new architectures to a solution existing in the system necessitates a
generous amount of retrofitting when the current DW architecture was designed
for standard reports and OLAP only. Similarly, shifting to the complex,
multi-platform system architectures atypical of modern DWs could be stymied by
the complexity of architecting a modern, complicated environment and the
intricacy in the management of a multi-platform DW environment.
Data warehouse platform limitations under a current
warehouse can be a sizeable obstacle when the existing DW environment cannot
scale up to big data or take in data fast enough to leverage big volumes or
streaming data. Moreover, modernizing the ecosystem all around a warehouse
needs the purchase or upgrading of many tool types otherwise, the resulting
modernization is restricted by a deficiency of tools for purposes of analyzing big
new data types or for integrating and management of big new data types.
 |
| Traditional IT Transformation | trivadis-training.com |
Best DW Modernization Practices
We continually noticed that DW modernization can assume a
lot of forms at varying scopes. The possibilities boil down to the most common
categories, to wit.
System modernization –
This system modernization includes upgrades and areas for
hardware and software servers or tools, at one end, and at the other end,
organizations are adding up fresh data platforms and analytics means to their
extended data warehouse environments to accommodate the latest data types, huge
volumes of data, and workloads processing.
Arbitrary modernization-
On one hand, it is helpful that many modernization tasks are
centered on the business requirements of a particular project, thus attaining
the level of data warehouse-to-business alignment. On the other hand,
interviewed DW professionals have taken notice that random tasks tend to pop up
erratically in which some demand a prompt reaction whereas other tasks can be
doubled into the usual cycles for an uninterrupted modernization process.
Modernizations of non-DW –
Surveyed and interviewed users keep on talking about how the
warehouse hardly ever totally gets modernized. It is because DW is frequently
modernized to better collect and provide data for fresh or evolving business
processes, integrated data solutions, reports (dashboard styles), and analytics
(advanced forms such as data mining, graphs, and statistics).
Optimized modernization –
Significantly, this is an important productivity concern for
the reason that an average DW professional often spends up to 30 percent of his
or her time on performance fine-tuning and other similar optimizations. In one
scenario, modern tools from both vendor and open-source communities have turned
into an expert at automated optimization, particularly for SQL-based queries.
Conversely, the emergent figure of stand-alone platforms in users’ extended
DWEs has pushed up the amount and intricacy of cross-platform queries, which
are not accordingly easily optimized.
Continuity in modernization –
The concern here is to nurture uninterrupted enhancement in
a well-thought-out approach that guarantees respectable benchmarks and quality
– and sanity. Numerous successful DW professionals have spoken well on exactly
how they stick to a regular, trimestral cycle for applying necessary updates to
the main DW. This one is comparable to the regulated release cycles as seen in
software companies and open-source incubation projects, but more so often it is
in line with the urgent requirements of the enterprise. Surely, the trimestral
cycles work fine with small-to-midsize tasks, from fine-tuning current data
models (performance or extending a customer view) to rolling out a new area
(employees or locations). On the other hand, the trimestral cycle can also be
applied to all-inclusive modernization tasks, for example, the implementation
of a new tool or platform or rolling over a group of users from one tool or
platform to another.
Disruptive modernization –
As one respondent aptly put it, “we have started a new BI
program that will rebuild the entire DW on a new platform.” A modernization
project can be dramatic, upsetting, and disruptive to users mainly when it
involves the rip-and-replace approach of key data sets, platforms, or tools.
Business managers will have to be involved in formulating and following the modernization plan. Modernization must be carefully planned as a multiphase
and multiyear project. At this level of modernization, as mentioned above, the
modernization aspect includes not only the DW but the entire technology stack,
and the modernization of that scale will also be accompanied by changes to
business processes, and sometimes even the whole business model. Thus, the
multi-phase plan is not just for the warehouse or the technology it
encompasses, but the plan must also outline how the business users and
processes will migrate and roll over as the modernization move forwards.
Strategies in Modernization
Data is captured and managed on the data warehouse platform,
which naturally consists of a DBMS (Database Management System, or an
equivalent, such as Hadoop), an operating system, networking, server hardware,
and so on. Therefore, the warehouse and its platform are two distinctive but
related layers. Hence, the distinction is significant because DW modernization may
hit only the data, the platform, or both. Data is not something to be replaced,
instead, modernization may extend, consolidate, remodel, and improve data. But
in many cases, however, replacing the DW platform could be viable if not a
practical modernization approach when the platform is deficient or no longer a
decent fit for business and its technological goals.
Rip and replace is an inherently expensive and disruptive
modernization exercise which is why many organizations prefer to just modernize
and improve the existing platform. A growing number of organizations complement
the existing DW platform by adding up other standalone platforms to the DWE or
combined update and complement strategies.
With these concerns in mind, it is not surprising that many
users are taking into consideration DW platform replacements as they plan their
DW modernizations.
Augmenting (but do not replace) the current DW’s main platform by adding up supplementary data platforms and tools –
More survey respondents, (42%) selected this strategy.
Choosing this strategy is consistent with the movement’s multiplatform DW
environments, which is one of today’s strongest trends in data warehousing.
This is a non-disruptive task from a business perspective for it preserves
current investments in data warehousing, and when done properly, it prolongs
the life of a costly and functional system. TDWI has for years seen users
deploy DW appliances and columnar databases as part of their warehouse
augmentation and modernization policy and just recently, Hadoop has turned out
to be the leading analytics and data platform for such policies.
Replacing DW’s existing main platform –
Here, 15% of organizations describe DW modernization as
replacing the existing data warehouse’s main platform, which of course is conservative.
For a minority of users, this approach is fully appropriate despite the
disruption and expenses that may incur. Typical of this approach are those with
a deficient or outmoded DW platform.
Determining strategy on a case-to-case approach –
Arbitrary modernization, as discussed earlier, has its
rightful place in DW modernization but should be controlled to reduce its havoc
and turmoil. From a different perspective, considering that modernization hits
several strata of the overall technology stack, plus has varied business
drivers, hence it is inevitable that a certain amount of case-to-case analysis
is compulsory.
No strategy yet, although one is needed –
A few interviewed users (14%), grumbled that the continuous
modernization of the DW was one practice after the other due to business
impatience, resulting too often in a disorganized DW that endures many
architectural verrucae, has inconsistent data quality, and is difficult to
optimize and maintain.
Conclusion
 |
| Modern Data Warehouse | beyondkey.com |
Data have evolved from an important to an extremely critical
part of entire organizations in terms of predicting and tracking particularly
important business processes. A modern DW is a basic requirement to make this
happen.
DW modernization can combine streaming and other
unstructured data resources with existing data warehouse investments. It
further optimizes DW storage and provides query-able archives and rationalizes
the DW for greater simplicity and lower cost.
Additionally, DW modernization provides better query
performance to enable complex analytical applications resulting in improved
business insights into operations for real-time decision-making.
________________________
References:
IBM | Big data at the speed of business
TDWI | Data Warehouse Modernization in the Age of Big Data
Analytics
Saxon
| Challenges and Benefits Associated with Modernizing the Data warehouse