For the past few months, I have been working with the Overton team to understand the makeup of materials indexed in the Overton database and the nature in which they cite academic literature. I’m pleased to announce that the results of this analysis have been published on arXiv this week and also featured in Times Higher Education.
The findings are promising, showing that Overton indexes a sufficiently large quantity of data to make citation indicators practical, with especially good coverage in areas of social science such as economics, political science, sociology, and education.
As someone with a keen interest in citation data and metrics, I have been itching to get my hands on the Overton data since I first heard about it from Euan back in 2019. It promises new opportunities to understand how research is utilised outside of academia and ties in nicely to the contemporary research evaluation focus on impact.
Overton has done a great job tackling the onerous task of wrangling out references to research from a variety of publicly available reports, whitepapers, guidelines and other grey-literature content, creating a rich citation network that offers novel analytical opportunities. Overton indexes data from more than 30,000 international sources, collecting more than 5 million documents that contain over 14 million references.
Our paper Overton: A Bibliometric database of policy document citations is a high-level survey of the bibliometric potential of the policy citation database, answering basic questions relating to the variety of documents and publication sources indexed and the network of citations that is extracted. Key aspects of the analysis include:
One important aspect of the paper is a comparison with data collected via ResearchFish. Since 2014, all UK funders have asked academics to report the variety of outcomes associated with funded grants such as spin-out companies, intellectual property, public engagement, and crucially policy influence. Therefore, it is possible to test whether this self-reported data broadly agrees with that automatically collected by Overton. The agreement is good in some research areas suggesting that Overton may become a valuable resource in future research evaluation systems.
This is a really exciting development in the research impact space that I believe has the potential to enhance the visibility of the academic-policy interface.
We hope you find it interesting and we’d love to know what you think! @overtonio @martinszomsor
Electric Data Solutions provides bespoke analysis to universities, funders and publishers to help them understand their unique contribution to the global research system.
Cover photo by 🇸🇮 Janko Ferlič on Unsplash