2024 in Review: Navigating the Shifting Data Landscape

2024 has been a pivotal year for data, marked by exponential growth, heightened security concerns and an evolving regulatory environment. This year’s data landscape is defined by continued acceleration of cloud adoption, the increasing power of AI and a growing awareness of the critical need for robust data governance. 

In this post we’ll explore the key trends that have shaped the year and what they mean for organizations as we look ahead to 2025.

The continued ascent of the cloud

The dominance of cloud computing continues to accelerate. The IDC predicted that public cloud spending worldwide would reach $805 billion in 2024, doubling by 2028, fueled largely by AI adoption. While the cloud offers greater scalability, flexibility and accessibility of data storage and processing for organizations, it also introduces new risks and complexities for data management and cybersecurity, such as the loss of visibility, control and governance of data.

As businesses seek cost-savings and operational agility through expansion to the cloud, resulting in increasingly fragmented IT environments and uncontrolled data proliferation.

The ease of access to cloud services also enables the creation of shadow stores and environments — data repositories or systems that are created or used without the knowledge or approval of the IT or security departments. These factors can leave businesses unknowingly exposed to cyber-attacks or data breaches. 

According to research by Thales, 44% of organizations experienced a cloud data breach in the past year, with a concerningly low percentage (less than 10%) encrypting 80% or more of their sensitive cloud data. This highlights the critical need for proactive cloud security strategies, effective cloud data governance and robust encryption practices.

The rise of AI: Growing power and responsibility

Artificial intelligence is revolutionizing data use, offering unprecedented opportunities for innovation, operational efficiency and business insights. While still lagging behind bleeding-edge AI technologies, the ethical and regulatory frameworks governing AI use are evolving rapidly. 

The EU AI Act, which came into force on August 1, 2024, provides a crucial framework for safe AI development, reflecting a global trend towards increased AI regulation. Other jurisdictions also invoking AI regulations and frameworks include Australia’s AI Safety Standard, Canada’s AI and Data Act, New Zealand’s Trustworthy AI in Aotearoa principles, Singapore’s Model AI Governance Framework, South Korea’s AI Act, the UK’s AI Standards Hub, and the US’ Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. 

The surge in AI governance responsibilities has fallen largely on privacy and cybersecurity teams, with no prior expertise in AI models or development. In fact, the IAPP found that 69% of surveyed chief privacy officers have acquired additional responsibility for AI governance. 

The AI revolution is well and truly underway, and it’s essential for organizations to proactively address ethical considerations in both AI development and use, and engage fully with emerging legislation while they have developmental control over their algorithms and AI models.

Synthetic data: Privacy panacea or regulatory burden

Gaining access to adequate volumes of training data for AI models is a major challenge for developers. Synthetic data, offering a privacy-preserving alternative to real data, is gaining traction for model training and validation. However, its use necessitates careful consideration of privacy regulations.

For example, in their recent guidance, the Office of the Australian Information Commissioner clarified that privacy obligations extend to both the input and output data of AI systems. Additionally, that personal information generated or inferred by AI systems, including images, is considered to be personal information and governed by relevant privacy laws. 

While synthetic data promises innovative solutions, organizations utilizing it must ensure it is created and used responsibly, complying with all ethical and legal obligations. 

Data sovereignty in an uncertain world

The world is in a precarious geopolitical position, driving organizations’ concerns for sovereignty and control over their most valuable assets regardless of location. 

The concept of data sovereignty, while initially derived from the demands of privacy regulations such as GDPR, has become a key focus for organizations in the context of organizational security and resilience. 

According to research by Thales, “future-proofing” the portability of cloud resources has become the primary driver for digital sovereignty initiatives. The trajectory of the current global political landscape remains uncertain, with many regions involved in or taking on a war footing. As such, organizations must understand and prioritize their cloud data assets and ensure they can retain autonomy and control of them under any circumstances.

The observability revolution

In today's data-driven world, organizations are moving beyond basic monitoring to embrace data observability. This involves gaining a comprehensive understanding of data health, quality and lineage, requiring robust visibility into data flows and transformations. This is not only a technological challenge but also a strategic imperative for operational efficiency. 

The significant costs associated with data storage ($3,350+ per TB annually) emphasize the critical need for data minimization and the elimination of redundant, obsolete and trivial (ROT) data. 

Furthermore, organizations’ are coming under increased regulatory scrutiny to uphold their privacy and data protection responsibilities. Data minimization and purpose limitation — retaining only data necessary for specific and defined purposes — is enshrined in privacy legislation worldwide, including the EU GDPR and several US state privacy laws.

Organizations are relying increasingly on business intelligence and analytics to drive decision-making and strategy. However, without a comprehensive view of their data landscape – a way to maintain visibility across their data estate and solutions to manage and monitor their data assets – following a structured framework such as Gartner’s Data Security Posture Management (DSPM), businesses are missing out on potential opportunities and exposing themselves to unnecessary cyber-risks.

Evolving security architectures

The rise of data mesh and zero-trust architectures is reshaping data security strategies. Data mesh architectures, adopting a decentralized approach to data storage, necessitate strong data governance and cataloging. Zero-trust, emphasizing continuous user verification and strict access control, places data at the heart of cybersecurity. 

These architectures rely on a comprehensive understanding of the pre-existing data landscape, upon which they can be built. Further, continued governance of data assets is required to manage and maintain the integrity and security of these environments. 

Breaches continuing unabated

The cost of data breaches continues to rise, reaching an average of $4.88 million in 2024 — a 10% increase over 2023. The prevalence of ransomware attacks, targeting backup repositories and demanding exorbitant ransoms, further highlights the urgency of robust data security and backup strategies. 

In the first half of 2024, the average extortion demand per ransomware attack was over $5.2 million — the highest figure to date. While most jurisdictions advise organizations against paying ransom demands, many businesses find themselves unable to restore operations by any other means. However, that a staggering 28% of organizations paying ransoms fail to recover their data emphasizes the devastating consequences of inadequate preparedness.

A new era for payments and beyond

The year has not only been shaped by technological advancements and evolving cyber-threats, but also by strategic mergers and acquisitions that have the potential to redefine industry boundaries and expectations. 

Among the more notable of these is Mastercard’s acquisition of Recorded Future, a move signaling potential seismic shifts in the cybersecurity and payments sectors. (A theme we’ll explore in a separate post.)

This acquisition highlights a growing trend where financial service giants like Mastercard are recognizing the huge value of integrating advanced threat intelligence capabilities to bolster their cybersecurity frameworks and payments services. 

Other notable mergers and acquisitions in 2024 have similarly emphasized a convergence of data analytics, threat intelligence and AI capabilities. Companies are increasingly realizing that data security, privacy and governance cannot be siloed efforts but must be integrated into their core business strategies.

Navigating the shifting data landscape

2024 has emphasized the critical importance of robust data strategies in an increasingly complex and shifting digital landscape. As organizations navigate the rapid advancements in cloud computing, AI and data governance, they must remain vigilant against emerging security threats and regulatory challenges. 

The trends highlighted this year demonstrate the pressing need for proactive measures in data management, from enhancing cloud security and AI governance to embracing data observability and protecting data sovereignty. By prioritizing these areas in the year ahead, businesses can not only safeguard their data but also leverage it for strategic advantage, ensuring both resilience and growth in an uncertain world.

To find out how Ground Labs can support your business, arrange a complimentary data workshop or book a call with one of our experts today.