The Regulatory Policy Institute Research Group

Institutional Innovation Initiative & the Energy Transition

The Energy Transition represents an unprecedented challenge for Policy Makers. The speed and magnitude of emissions abatement demanded by the 2008 Climate Change Act, whose 4th & 5th carbon budgets are currently off course, combined with the subsequent increased ambition signalled by the UK 2050 Net-zero commitment made in 2019, along with the recent course-correcting commitment to target a 68% fall in emissions by 2030 vs. the 1990 baseline, requires the rapid promotion and diffusion of clean energy innovation with action extending to the hard-to-abate sectors dominated by buildings, industry and transport. This is before we even explore the nation’s external consumption emissions footprint.

A technology transition might be thought of as the time taken for an innovation to displace incumbents moving from 5% to 80% market adoption or, failing that, the point at which peak adoption is achieved. Technologies go through distinct phases, typically: invention, demonstration, deployment, incremental improvement from learning-by-doing, economies of scale and diffusion. Transitions can often be characterised by periods where hybrid technologies of old and new prevail and overlap, for instance boats powered by sail & steam. New energy technology or source transitions are relatively rare and often very protracted. One recent study observed that the fastest historical sector-specific energy transition observed has been around thirty years.

Full energy transitions, involving all sectors and services, have taken much longer than sector-specific ones, possibly hampered by the path dependency of policy, the costs of overcoming entrenched incumbents and the extent of technology lock-in, but also by the sheer complexity that such change involves. Sometimes, exogenous price shocks or evolution in economic structure have played a role in tipping the balance. Understanding the key characteristics of such transitions is critical as we embark on one from fossil fuels to renewables. Innovations often start as expensive niche products with superior or additional characteristics for which consumers (or governments) are willing to pay. Every historic full energy transition, be it biomass to coal, coal to oil or oil to natural gas, has involved different sectors and services for which bespoke technological and institutional solutions have been required to achieve transition.

Currently, 80% of world energy comes from fossil fuel sources. By 2050, the global population is projected to grow to ~9.5 billion from its current ~7.6 billion, a billion people currently off-grid will be on-grid, the global economy will have at least doubled in size, with energy demand expanding by an estimated 30-50%. Given our understanding of the challenges involved in full energy transitions, moving fully from 80% reliance on fossil fuel to renewable & carbon-abated sources in the 30 years to 2050 requires an unprecedented step-change. Some liken the challenge to that of the Apollo & Manhattan projects. But these were single goaled projects involving defined technologies for a single client. This is a much more complex challenge which reaches into every sector, product and process in the economy. The more complicated the challenge, the less able the state is to deliver and the more reliant on markets for its delivery it should become. Nevertheless, it still has a key role to play.

The aspiration of distant policy targets like “net-zero 2050” is that they set an expectation that hopefully energises all the departments of state, researchers in Universities, investment focus of corporations and organs of the economy towards a particular outcome. The downside is that you can set a target either to a date or to a budget, but you can’t do both simultaneously. The outcomes created by such policy targets could be perverse and unintended in terms of the directions that they take, their impacts on the achievement of other policy objectives, and the ultimate substantial costs that they may impose. Optimistic estimates of what Net-zero may cost have ranged from 1-3% of GDP. Poorly conceived policy might raise these costs materially. Nevertheless, the costs associated with no action at all potentially dwarf both these numbers.

The problem is compounded by the fact that the ultimate successful technologies are uncertain. Consequently, with the acknowledgement of the tight timeframe, market risks of competing new-technology redundancy, and the limited resources available for research, development and deployment, policy makers are driven in the direction of picking technology winners, a role that they have historically been ill-suited to perform.

Moreover, there remain disagreements around end-uses for certain competing technologies, for instance heat pumps vs hydrogen boilers in building heating, or batteries vs. hydrogen in certain modes of transport. The ultimate outcome may favour one or a hybrid technology solution. Equally, different technology solutions may be favoured in different local, national, regional or international contexts, requiring the provision of very different kinds of support infrastructure.


If inaction is not an option, but public intervention is fraught with risks around technological direction, political or industrial capture, unintended consequences and execution, what can Government, conflicted as player and referee with limited information and foresight, usefully do to empower change?

A number of obvious questions arise: Is it simply about the government providing market signals around ultimate sector decarbonisation targets with key staging posts? Is change better prompted via standards and regulation, or via carbon taxes that guide the direction of travel in a technology neutral manner? Or does government need to make more assertive strategic decisions around which technology routes are to be pursued? Is the innovation challenge less about picking technology winners, and more about picking which races to be in? In turn, is this less about promoting radical and more about assisting incremental innovation? Is the public sector challenge materially about super-charging R&D and providing funding support for Demonstration & Deployment, or Is it more about promoting private investment and facilitating financing channels for the roll out of specific proven technologies? Where does Industrial Policy fit into this, if at all, as an intermediating factor? In all probability, there is a balance to be found between all of these approaches over differing timeframes, but the nature of the public mechanisms by which they are promoted matter.

The last decade has seen a substantial body of literature generated which explores the role and experience of innovation institutions in the promotion of industrial policy and energy technology. Examples of such institutions include the UK Catapults, DARPA & ARPA-E in the US, and the Fraunhofer Gesellschaft in Germany, with other prominent examples in Asia.  Some insights are found regarding the ideal characteristics that different types of innovation institutions might embody, although historic design has often related less to optimality and more to national legacy institutions and prevailing economic orthodoxy. Discussion around the optimal relative positioning of such institutions vs. government, industry, Universities and the third sector, tends to focus on whether they are promoting incremental or radical innovation, are policy directed or technology neutral, focused on near-to or far-from market technologies, central to or peripheral from the innovation systems framework, wide or narrow in their scope of enquiry, the extent to which they rely on public or private sector capital, and their desired framework of accountability.

In the UK, a country with relatively few legacy energy-innovation institutions, the case for an arms-length body modelled on ARPA-E has recently been advanced. But exploring this approach requires much more than just an international institutional cut-and-paste, involving a proper audit of national energy, clean growth and industrial policy objectives, existing energy-innovation initiative, a wider consideration of desirable optimal institutional characteristics and the national innovation system context. Arguably, the definition of institutional initiative should extend beyond simple organisations, embracing a set of formal bodies, initiatives and policies, including regulation, which self-reinforce and seek to assist in the promotion of change.

Addressing the UK de-carbonisation challenge then naturally pushes us towards questions around what kind of regulatory regime can assist markets in meeting these energy innovation, product and process challenges. What characteristics should publicly sponsored institutional initiatives promoting transition embody? How can policies be aligned such that they encourage change, but do not conflict, while minimising the chances of promoting costly or perverse outcomes?

For enquiries about support for and/or participation in this line of work please contact:


© Gerard Fox RPI 2021

Share on Twitter
Share on LinkedIn