The story of the Integrated Data Service (IDS) is a cautionary tale of mission creep, budget misappropriation, and the perennial challenge of escaping legacy IT systems. It's also a stark reminder that in government digital transformation, grand visions often collide with mundane realities like 21-year-old databases and departmental politics.

The Promise: A Data-Driven Government

The ONS's 2020-25 business plan promised an Integrated Data Platform Program that would "support the integration of Government data, enable inclusive analysis on national issues and facilitate enhanced dissemination of data and analysis." The platform would deliver a "strategic, secure and user-friendly environment" with integrated microdata available for research and public presentation.

This vision aligned perfectly with broader government ambitions for evidence-based policymaking. With comprehensive, linked data across departments, ministers and civil servants could understand complex social and economic challenges, design targeted interventions, and measure outcomes with unprecedented precision.

The timing seemed opportune. Cloud technologies offered scalable infrastructure, data science techniques promised new analytical insights, and political support for digital transformation appeared strong. The business case projected benefits justifying the investment, and Treasury approved a multi-year funding envelope.

The Reality: Budget Appropriation and Missed Objectives

An independent review by Sir Robert Devereux revealed what actually happened to that £240.8 million: with Treasury approval, it was diverted to fund "more general tech and data costs" as the ONS struggled with legacy IT systems. Rather than building the promised integrated platform, the money subsidised basic operational costs.

This budget appropriation—technically approved but fundamentally undermining the programme's purpose—exemplifies a common pattern in government IT: ring-fenced transformation budgets become slush funds for keeping the lights on when operational pressures mount.

The Infrastructure and Projects Authority (IPA) raised the alarm last year, upgrading the project's risk rating from amber to red after it completed only three of twelve recommended gateway review actions. The IPA also slashed nearly £500 million from the programme's projected benefits—a devastating reassessment suggesting the business case had been fundamentally optimistic or that implementation had gone badly wrong.

By March 2025, the National Infrastructure and Service Transformation Authority (NISTA)—the IPA's successor—maintained the red risk rating, citing the programme's inability to "demonstrate sufficient progress in unlocking cross-government data sharing," limited case studies of actual analysis undertaken on IDS, and lack of a replacement Programme Director.

What Went Wrong: The Technical and Political Challenges

The Devereux review identified several fundamental problems:

Legacy System Paralysis

The ONS still runs HCL Notes and Domino—the 1990s-era collaboration platform originally developed by Lotus. The organisation acknowledged that flat financial settlements forced "tough choices" including deprioritising modernisation programmes. Yet modernisation was supposedly what the IDS budget was for.

This creates a vicious cycle: operational costs for maintaining legacy systems consume resources meant for transformation, preventing the modernisation that would reduce operational costs. The IDS budget became another victim of this dynamic.

The Two-Platform Problem

The IDS was originally intended to replace the ONS's ageing Secure Research Service (SRS). Instead, both now run in parallel—doubling operational costs whilst confusing potential users about which service to use. The SRS, which recently celebrated its 21st birthday, continues serving accredited researchers with curated datasets whilst the IDS serves primarily as an internal ONS environment with limited external access.

This outcome directly contradicts the original business case and represents a fundamental programme failure. Rather than consolidation and modernisation, the ONS now maintains two separate data-sharing platforms.

Departmental Politics and the "High-Handed" Approach

The Devereux review noted that the IDS "relied heavily on securing data from other departments" but adopted what several observers characterised as a "high-handed approach, essentially demanding data, and downplaying others' concerns for example around security."

This political misstep proved costly. Government data sharing requires trust and cooperation between departments with different priorities, risk tolerances, and technical capabilities. An approach perceived as demanding rather than collaborative undermined the cross-government buy-in essential for success.

Departments protective of their data—particularly given legitimate security and privacy concerns—found it easy to resist or delay sharing with a programme that seemed to dismiss their anxieties.

Limited External Usage

Despite promises of revolutionising data access for researchers and analysts across government, few external users adopted the IDS. The platform failed to demonstrate compelling advantages over existing access routes, whilst its complexity and access restrictions deterred potential users.

This usage failure undermines the programme's entire rationale. A data-sharing platform's value lies in the insights it enables. Without users generating those insights, the platform becomes expensive infrastructure serving no purpose.

Enter the Third Platform: The National Data Library

Just as the IDS programme was being wound down, the Labour government's autumn budget confirmed creation of a National Data Library (NDL)—a third data-sharing service, this time controlled by the Department for Science, Innovation and Technology rather than the ONS.

The NDL promises to "provide simple, ethical and secure access to public data assets, giving researchers and businesses powerful insights that will drive growth and transform people's quality of life through better public services and cutting-edge innovation, including AI."

This sounds remarkably similar to the original IDS vision, raising obvious questions: Why create a third platform when existing ones haven't succeeded? How will the NDL avoid the problems that plagued the IDS? What's the relationship between these three services?

According to HMRC guidance, the division of labour appears to be:

• SRS: Curated, research-ready datasets for accredited researchers (21 years old, still functioning)
• IDS: Primarily internal ONS environment with limited external access (not currently accepting new applications)
• NDL: New platform for simple, ethical, secure access to public data (details unclear)

NISTA's report noted "ongoing uncertainty in the relationship with NDL and the future of linked data"—hardly a ringing endorsement of clear strategic planning.

The Cost of Failure

The financial waste is obvious: £240.8 million spent with limited tangible benefit.
But the broader costs extend far beyond the immediate budget:

Policy-Making in the Dark

The government faces enormous challenges: housing shortages, NHS backlogs, social care crises, education reform, immigration policy, and more. Evidence-based responses require comprehensive data linking across departments to understand causal relationships and measure intervention effectiveness.

Without functioning data integration, policymakers work with fragmented information, making it harder to design effective interventions or learn from failures.

AI Ambitions Without Data Infrastructure

The government has ambitious plans for AI to improve public sector productivity and service delivery. But AI systems require high-quality, comprehensive data for training and operation. The failure to deliver integrated data infrastructure directly undermines AI ambitions.

Erosion of Trust in Digital Transformation

Every high-profile government IT failure reinforces scepticism about digital transformation projects. The IDS debacle makes it harder to secure support and funding for future initiatives, even when genuinely needed.

Opportunity Cost

The £240 million diverted to general tech costs represents funding unavailable for other priorities. In an era of constrained public spending, this waste is particularly galling.

Lessons for Government IT

The IDS failure offers several important lessons:

Legacy Modernisation Can't Be Deferred Indefinitely

The ONS's continued reliance on systems from the 1990s isn't sustainable. Eventually, either catastrophic failure forces emergency modernisation (expensive and risky) or gradual decay makes the organisation unable to fulfil its mission. There's no comfortable middle path where legacy systems continue indefinitely whilst transformation happens around them.

Cross-Government Programmes Require Genuine Collaboration

Demanding data from other departments doesn't work. Successful cross-government initiatives require patient relationship-building, addressing legitimate concerns, demonstrating value, and earning trust. Technical solutions alone cannot overcome political and organisational resistance.

User Adoption Requires Clear Value Proposition

Building infrastructure doesn't guarantee usage. Platforms must offer compelling advantages over existing approaches, with benefits that clearly outweigh adoption costs. The IDS apparently failed this test, leaving expensive infrastructure underutilised.

Strategic Clarity Prevents Platform Proliferation

The existence of three separate data-sharing platforms suggests strategic confusion. Before launching new initiatives, government should clarify how they relate to existing services, whether consolidation makes more sense than proliferation, and how to avoid duplicative investments.

The Bigger Picture

The IDS failure sits within a broader pattern of struggling government IT programmes. From the abandoned NHS IT programme to ongoing challenges with Universal Credit, the public sector's record on large-scale digital transformation remains mixed at best.

This pattern partly reflects the genuine difficulty of these undertakings—integrating systems across large, complex organisations with diverse requirements and constraints. But it also reflects systemic issues: optimistic business cases, insufficient technical expertise, inadequate governance, and political pressure for quick wins over sustainable solutions.

As the government prepares for Rachel Reeves's autumn budget, the need for comprehensive data to inform policy has never been greater. The challenges facing the UK—from healthcare to housing to economic productivity—require evidence-based responses informed by high-quality analysis.

Yet after five years and £240 million, the government has less integrated data infrastructure than promised and more platform confusion than before. The Treasury was right to cut funding for a programme that had failed to deliver, but the underlying need hasn't disappeared—it's only become more urgent.

The National Data Library represents another attempt at the vision the IDS promised but failed to deliver. Whether it succeeds will depend on learning from predecessor failures rather than repeating them. That requires honest assessment of what went wrong, realistic planning about what's achievable, and sustained commitment to doing the hard work of modernisation properly.

Without these elements, the NDL risks becoming the fourth chapter in an ongoing saga of expensive failure—another platform added to the confusion, another budget appropriated for other purposes, another opportunity missed to build the data infrastructure modern policymaking requires.

Navigate Digital Transformation Without the Costly Mistakes

The ONS data-sharing programme demonstrates how easily digital transformation initiatives can go wrong—consuming budgets whilst delivering limited value. At Altiatech, we help organisations avoid these pitfalls through realistic planning, rigorous governance, and proven delivery methodologies.

Don't let your transformation programme become the next cautionary tale. Contact our expert consultants:

• Phone: +44 (0)330 332 5482
• Email: innovate@altiatech.com



Get your digital transformation right the first time with experienced guidance and practical solutions.