The politics of policy

How we can use the Overton Index to analyse shifts in policy trends and agendas - how do they change between months and years? What drives their changes? How quickly do they change? How quickly does the appetite for evidence grow or shrink, and on what issues?

In light of recent elections across the world and the surrounding media debates, we decided to take a closer look at the whys and whats of policy development, and how this can be shaped by politics. 

The intertwining of politics and policy is a fundamental aspect of governance, shaping the direction and priorities of a nation. Recent elections  give us a compelling case study for how political dynamics influence policy decisions, reflecting the broader trends and specific agendas of the winning parties. We’ll take a look at  the different ways in which the political landscape might shape policy, why this is important for the academic-policy landscape, and how you might explore this using our data. 

Political mandates and policy direction

Elections serve as a barometer of public sentiment, with parties campaigning on platforms that reflect their policy priorities. By being elected, a party has the mandate to implement its proposed policies. As an example, if the winning party campaigned on a platform of economic reform, social welfare enhancement or environmental sustainability, these themes are likely to dominate the legislative agenda. Taking a particular government and mapping out its policy subject areas or topics by years, we can quickly get an idea of how politics has shaped policy development and implementation between administrations and/or major political events.

The 2016 BREXIT referendum is a prime example of this - the vote to leave the European Union set the legislative agenda for the next few years, with r successive governments negotiating the terms of departure, including new trade agreements and immigration laws. 

In the US, Obama’s 2008 election victory gave him a clear mandate to pursue healthcare reform, leading to the passage of the Affordable Care Act (ACA) in 2010, which reshaped the US healthcare system. These are examples of:  party wins > party passes policies which underscored their campaign. Let’s take a quick look at the UK Government policy data between years (2014 and 2024) in the Overton Index, as an illustration.

We looked at the policy data from UK government sources for 2014 and mapped the topics, so we can see which areas appear most frequently. For the purposes of clarity, we’ve limited it to any topics which appear more than 500 times in this set of the documents. 

We then repeated this exercise for 2024:

We’ve done this as a relatively quick and simple snapshot exercise for illustration purposes here, which is quite a high-level way to look at it. It’s worth noting that the 2024 set is an incomplete set - and given the recent election in the UK, it may look quite different by the end of the year!

If you were interested in taking a closer look, we’d recommend using topic distribution for a more nuanced (and normalised!) representation of how policy topics can grow or shrink over time, calculating frequencies as percentages of the net number of documents published that year in order to normalise for publication rates. Alternatively, you could simply select a topic to see how many documents have been published each year under. 

As an example, we took BREXIT and used Overton’s automatic reporting function to get a quick idea of how much has been published on the topic and when: 

Legislative majority and policy implementation

Unsurprisingly, we see a large publication volume between 2018 and 2020 relative to other years, which coincides with the 2019 general election and significant public interest. That particular election resulted in a substantial conservative majority (365/650 seats) in Parliament, allowing PM Boris Johnson to pass Brexit legislation with relatively little opposition - between that and the level of public interest in this issue at the time, it’s perhaps unsurprising to see a spike in the level of productivity on this particular topic (though, we’d recommend further analysis before drawing a firm causal conclusion here!). 

A year later, in the USA, the Democratic Party’s control of both the House of Representatives and the Senate allowed President Biden to advance key policies such as the American Rescue Plan, aimed at economic recovery from the COVID-19 pandemic more effectively. The composition of parliament, senate, congress (and any other type of legislative body) plays a significant role in determining how effectively the policies of the winning party can be implemented - a hung parliament or slim majority can lead to compromises and coalitions, whereas a strong majority enables the government to push through its agenda more decisively, as exemplified in this journal article published in Parliamentary Affairs in 2020.

Opposition influence and policy debate

Even when a party has been elected, the role of the opposition is critical in shaping policy discourse - they scrutinise, challenge and propose alternatives to the government's policies, contributing to a dynamic policy formulation process. Depending on your context, this can have varying implications. 

In the UK, the strength and cohesion of the opposition can affect the legislative process - and therefore, how efficiently the winning party can pass their policies. If the opposition is united on particular issues such as healthcare, they can exert significant pressure on the government, leading to more moderate or revised policies. An example of this in the UK is Labour’s opposition to the Conservative-led proposal to austerity measures in the 2010s, contributing to significant public debate and leading to the moderation of particular policies over time - particularly those affecting social services and public sector pay. 

Alterations to Obama’s ACA programme were similarly realised during ongoing legislative and judicial challenges; One such example is that in the initial ACA proposal included a public option, a government-run health insurance plan that would compete with private insurers to provide more affordable healthcare choices. 

It’s well documented that republican law-makers and democrats, influenced by lobbying from the insurance industry, opposed the public option. They argued it would unfairly compete with private insurers and potentially lead to a single-provider system. To secure enough votes for passage, democrats dropped the public option from the final legislation, in order to gain the required support to ensure the bill’s passage.  

So…what can we learn using Overton’s policy data? Let’s take a  look at who has influenced/is influencing the authoring organisation. 

One way to explore  this would be to identify all policy documents containing a mention of Obamacare between the dates of the initial proposal, and the date it was published into law. Looking at some basic milestones, we can quickly establish an idea of what the ‘window of influence’ was:

1. Obama hosts summit on healthcare reform, 05/03/2009
2. Initial Proposal (HR 32000 and Senate HELP Committee Draft), published 17/09/2009
3. Town hall meetings and public consultations (various dates, 08/2009)
4. Revised bill, H.R 3962, published 29/10/2010
5. Final Bill (HR3590), published 22/12/2010

So, we want to look at anything between 05/03/2009 and 22/12/2010, mentioning different names and common variations that refer to the Affordable Care Act or Obamacare, ensuring a broad search across the database. We can use the following search strategy:

• Use a boolean string to identify any documents mentioning ACA or variations thereof: ("Affordable Care Act" OR "ACA" OR "Obamacare" OR "Obama care" OR "Health Care Reform" OR "Patient Protection and Affordable Care Act" OR "PPACA" OR "Health Insurance Reform" OR "Health Reform" OR "Healthcare Law" OR "Health Care Reform")
• Filter for the following criteria:
  • Was published between 05/03/2009 and 22/12/2010
  • Was authored by a government source in the USA
  • Cites others - either scholarly or policy
• Then, review the automatic report and look at ‘cites policy from source’. 

In this timeframe, 609 documents were authored, citing policy documents from 118 different sources. 87 of these documents were authored by Senate Committees, another 80 by House Committees, and 43 by the Congressional Research Service. Amongst the cited organisations, there are also quite a few think tanks, including NBER, Kaiser Family Foundation, Urban Institute, RAND corporation, ASPE and brookings, as a non-exhaustive list of examples.

Public opinion and policy responsiveness

Interestingly, analysis by researchers at the University of California also shows that public opinion can also affect policy implementation and uptake. In the case of Obamacare, republicans were less likely than democrats to enrol in an insurance plan through state or federal exchanges - but if the role of government was de-emphasised, the partisan gap could be closed. In the context of our policy data, we’d perhaps expect to see more amendments and discussion to policy documents on politically contentious issues - when we remove the date filters from our initial search, we can see that there is a huge volume of policy documents from US government sources, and is still actively being discussed in policy documents more than 10 years on from the initial proposal. This is potentially an interesting indication as the longevity of the policy development process. 

Aside from public opinion and opposition, the economic context in which a government operates can heavily impact policy decisions - factors such as inflation, employment rates and debt will determine the outcomes in most cases. Fiscal policies, spending programs and regulatory changes will be shaped by the need to address economic challenges and opportunities. 

Let’s take a look at an example in Australia - the Renewable Energy Target (RET). This aimed to increase the proportion of renewable energy in Australia’s national supply, but concerns about the economic impact on industries and electric prices led to political and economic pressure. 

Using the topics to narrow down our initial search, we can identify relevant documents such as various submissions to the Senate inquiry on Wind Turbines, such as this example by the Institute of Public Affairs: 

This example refers to the previously mentioned concerns about the economic impact and the costs of electricity as well as the political implications for the clean energy regulator and legal rights.

The original target was revised downward in 2015 from 41,000GWh to 33,000GWh by 2020 to balance environmental goals with economic realities. You can view the report for things related to  ‘Renewable Energy Target’ authored by the Parliament of Australia here. 

Knowing the context of the political system you’re operating in will also be useful when exploring the app - for example, parliaments will often be the epicentre for public debates prior to national policies and legislation being implemented, which can be helpful for identifying key influencing bodies and mechanisms.

Understanding each of these factors can provide valuable insights into how governments prioritise and implement their agendas - and how appetite for evidence and its usage might be shaped accordingly. 

We know that basing policy on evidence  is a no-brainer, but translating research into reality in the field can often be tricky. In an environment fraught with conflicting interests, agendas and fluctuations in economic health  implementation of good evidence-based policy is by no means certain. In some cases, the definitions of a ‘good’ policy outcome and subsequent revisions to the policy might look different too!