Electric vehicles, heat pumps, induction cooking, and distributed solar technologies are transforming electric power networks into volatile and bidirectional power systems, which are exerting unprecedented pressure on distribution system operators (DSOs). Traditional forecasting and stress reinforcement cycles, which work best for more stable and predictable networks, are lagging behind in predicting where and how stress will appear and how fast it will grow.

Meanwhile, regulators throughout Europe are urging the DSOs to transform into proactive, data-driven local system managers who are required to produce detailed development plans, measure the availability of flexible resources, and substantiate that investments are efficient and future-proof. Flexibility is becoming a more viable tool to leverage capital expenditure, deal with short-term constraints, and avoid acting on each constraint by itself.

This new perspective by global consultancy Arthur D. Little finds that while the vast majority of DSOs possess huge amounts of information, particularly from smart meters, GIS, SCADA, asset registers, and measures of power quality, most of this data is siloed, disjointed, and not optimally prepared for strategic planning. Data is not the problem – it's data readiness.

ADL recommends a data-as-a-product model, which is divided into three refinement layers: a bronze layer for raw data ingestion, a silver layer for cleansing, validation, and harmonisation of the data, and a gold layer for delivering business-ready intelligence, such as profiles of customer segments, substation health scores, and local flexibility potential maps. This architecture is about turning raw, inconsistent operational data into trusted inputs for investment decisions and regulatory compliance. Most importantly, ADL does not see it as an IT initiative but as a business transformation, and attributes it to the C-level, with accountability shared between planning and operations, and a change of culture towards using data to inform business decisions.

The approach is demonstrated in two case studies. In one, the leading DSO applied deep neural networks to recreate more than 74 million missing daily customer load curves for 2024, with a monthly validation error of less than 10 per cent, thereby providing bottom-up grid analytics without universal smart meter deployment. The entire data perimeter of an entire DSO was processed in about five minutes by the reconstruction engine, allowing for daily or monthly refresh cycles to be scaled and made practical.

In a second case, ADL assisted a DSO in creating a substation-level flexibility business case, which estimated that nonstructural overloads (those that can be addressed other than through infrastructure investment) would impact 122 substations, or 163 MWh of flexibility-relevant energy, for a total of 7,800 violation hours in 2029. Using this analysis, planners were able to identify structural constraints that need capital investment to overcome, and also time-bounded, localised constraints that could be adapted through flexibility and can be addressed through a more economically efficient, portfolio-based planning approach.

Data-driven DSOs that view data as a strategic asset, managed at the executive level, industrialised to build re-usable pipelines, and enriched with external data like hyperlocal weather and EV charging patterns will be best equipped to optimise investments, manage grid stress, and credibly move forward the clean energy transition.