Overton Blog
In 2023 Overton launched the Policy Impact Micro Grant - a scheme to fund research projects that explore the policy landscape, and its relationship with evidence and the academic sphere.
Some of the funding was allocated to Francesca Soldati, Scholarly Communications Advisor at the University of Aberdeen, to explore how UKRI funding influences policy making. Here Francesca details her project and its results, including the councils with the most policy citations and the type of policy documents that cite UKRI funded work the most.
As one of the world's largest public funding bodies for research and innovation, UK Research and Innovation (UKRI) plays a pivotal role in advancing knowledge and innovation. Given the substantial taxpayer investment, it's essential to demonstrate how UKRI-funded projects benefit society by shaping policies and delivering societal returns.
Measuring and assessing the use of research in policy is a powerful way to quantify the exchange of knowledge from academia into wider society, and it's arguably one of the most direct ways for scholarship to enhance quality of life.
My study focused on understanding how research funded by UKRI's seven research councils influenced policy development from 2012 to 2022.
By cross-referencing publications from Web of Science with the Overton policy database, I analysed the extent to which funded research was cited in policies, the benefiting organisations, the influenced policy areas, and whether highly cited academic papers within academia also had a greater impact on policy making.
Here is a summary of my key findings:
1. Impact varies across disciplines:The Economic and Social Research Council (ESRC) was the most influential in policy, with 39.6% of its funded publications cited in policies, followed by the Natural Environment Research Council (NERC) at 25.4% and the Medical Research Council (MRC) at 18.2%.
In contrast, the Engineering & Physical Sciences Research Council (EPSRC) and the Science and Technology Facilities Council (STFC) had the lowest impact on policy, with respectively 4.8 and 0.8% of their funded research cited in policy documents (Fig.1). This does not diminish the societal impact of these councils’ research, rather suggests that using policy citations as a metric may not adequately assess societal return for these types of disciplines.
2.Government policies as major beneficiaries:
Of the organisation ‘types’ that use research funded by UKRI councils (i.e. government, think tanks, IGOs), government is the most common beneficiary. 51.7% of all cited publications across councils were used in government policy.
This highlights the significant role of funded research in developing policies that directly impact society. It’s relatively uncommon that research is cited directly by policy makers. Much research is ‘translated’ by intermediaries such as think tanks, and repackaged into ‘digestible’ formats that policy makers can access more easily. So this statistic is interesting. Though it’s outside the scope of this study to explain why, one can speculate - it perhaps speaks to UKRI councils’ status and indicates that they are seen as trustworthy by decision makers. It also might reflect the UKRI’s status as a publicly funded entity, and suggest that their funded projects are closely aligned to government research priorities.
Overall, the data suggest a promising alignment between each council's focus and the policy areas influenced by their funded research, indicating that UKRI Research Councils maintained a defined scope during the observed period (Fig 3). Notably, three policy topics—Health, Risk, and Research—emerge as focal points drawing expertise from multiple councils.
Indeed, the recent emphasis on interdisciplinary approaches in academia - highlighted in REF2029 and the proposal of Times Higher Education (THE) to create yet another ranking based on interdisciplinarity - is prompting a reassessment of collaborative research practices within and across universities. This emerging agenda has the potential to blur the more traditional "field boundaries" identified in this study. It would be intriguing to revisit this analysis in a decade to evaluate its influence on research focus areas and to observe potential refinements in these definitions.
Figure 3. Distribution of funded publications per Research Council across the most frequent citing policy topics.
4. Preference for Highly Cited Research:
Documents cited in policy tended to be highly cited in their academic fields, with a significant portion ranking in the top 10% field citation percentile (FCP) (Fig 4).
This suggests that policymakers often rely on highly cited papers for policy development. Similar findings were also reported by Yin et al., 2021. While this could indicate aligned interests among the public, policymakers, and academics, it could also suggest that academic interests guide the selection of research for policy development. While it's challenging to determine the most plausible explanation solely from the data, citations seem to serve as indicators of the probability of being cited in policy.
This explorative analysis offers one perspective on measuring societal influence of UKRI-funded research and highlights the importance of acknowledging the nuances across disciplines.
While citation in policy appears to be a constructive indicator for impact in social and environmental sciences, it may not be true in other domains. Many questions remain open and further research in this area has the potential to enhance our understanding of the intricate relationship between research, policy, and societal impact. This could lead to a better grasp of the dynamics contributing to evidence-based policymaking and the development of more robust measures for societal benefit.
Thank you to Francesca for sharing her findings! You can read her full project here.