Why Digital Transformation Projects Fail; How Data Architecture Can Help

Every year, large corporations spend trillions of dollars on digital transformation projects, which they hope will help them take advantage of emerging technologies and boost their business operations.

Among these technologies are predictive analytics and decision-support systems, which have the potential to transform the traditional focus of IT from saving costs and solving technical constraints to proactively creating value and opportunities across the organization.

However, implementation challenges mean many of these digital transformation projects end in ruins, leading to huge financial losses for the companies involved.

Researchers from the Harvard Business School and Stockholm School of Economics in Sweden recently published a study in the Journal of Digital Economy (published by KeAi), which explored the origins of these implementation challenges, and examined how enterprises can improve their chances of driving business value creation from new digital technologies, especially big data and cloud computing.

Drawing on detailed survey data, the researchers identified ‘data architecture’, in other words, the way in which an organization’s logical and physical data assets and data management resources are structured, as a crucial element of successful digital transformation.

“Data architecture broadly captures the technological architecture’s capabilities around processing data streams through multiple layers and pipelines, combining data sets across various sources and locations, and developing and deploying machine learning models at scale,” explained Sam Cao, an Assistant Professor in the Department of Entrepreneurship, Innovation and Technology at Stockholm School of Economics.

“A coherent data architecture allows firms to integrate large amounts of data in an automated and timely manner. And it helps traditional corporations translate technical investments into user-centric co-inventions. Such co-inventions include machine learning applications and predictive analytics embedded across the organization in various business processes, which increase the value of work conducted by data users and decision-makers.”

The researchers also found that legacy systems negatively affect a firm’s ability to align their data architecture and commit to transformation.

“Outdated technology components within existing IT systems can encumber digital transformation efforts,” said Cao. “Complex software systems further exacerbate the problem. Mission-critical software applications are embedded in many parts of the organization, making legacy technologies even harder to remove.

“The study points to the fact that firms should carefully evaluate these challenges before committing to digital transformation initiatives, and it highlights the importance of a practical architectural framework for managing AI.”

The Methodology

The data was collected by Keystone Strategy and Microsoft via in-person interviews with senior technology executives at more than 100 Fortune 1000 corporations, with combined revenues of over $4 trillion. The survey covered a comprehensive set of technical frameworks and advanced tools developed by digital natives in data analytics and AI adoption. Each corporation surveyed was chosen for its traditional operating model, which can negatively impact the adoption of new digital technologies. These companies are often constrained by complex legacy enterprise systems that have been created and modified over long periods of time.