Blog: From Numbers to Narratives: An Everyday Tale of Impactful Folk
by Ben McLeish, Director of Engagement at Altmetric and Dimensions
Wednesday 26 September
The first major session after the 5AM keynote featured three incisive talks about how narratives can be formed around the impact data we see in the altmetrics space. What stories can be told? Can these data help to inform the questions which research asks? Can causal links be drawn between online social and sharing activity and the ultimate concrete impact in citations, patents, policy and so on? And how can these data be easily visualised to tell the stories of impact for a researcher?
Brett Buttliere of TU Dresden was first up, his prior work having had the focus of looking at the sharing and discussion activity of scientists online. His work has shown that scientists tend to talk online about research which has disagreements, and about topics which bother us (such as the topic of cancer for example.) This makes intuitive sense, as this is most likely a reason why the online news media suffers from an addiction to “clickbait” and distortion of reportage when it comes to topics such as cancer prevention. So it also comes as no surprise that when we review the Altmetric Top 100 articles for a given year, the research centers around hot-button issues such as vaccines, cancer and the like.
Brett raised an interesting question early on - it’s all very well to improve any method of measurement in order to gauge something - but this is itself not particularly useful if you aren’t measuring something useful. So, can altmetrics help gauge how people are attempting to change each others’ opinions on research online, or even be vectors for helping that discussion grow? (Interestingly, the London Institute has published a paper on how to debunk in a world of online tribes who don’t listen to each other. Interestly, and perhaps directly related to this, it has an impressive Altmetric score and good general distribution.) Can the Socratic method, this move from Thesis, to antithesis, and finally synthesis of ideas, and research development, be informed by measuring online attention and distribution?
Brett’s talk culminated in a very interesting bit of data science. He took impactful papers and data mined them for negative terms and found a strong correlation. Words like “cancer”, “infectious”, “care” (which, while appearing positive is usually associated with the problems of healthcare and how to improve it). The more controversial, the more negative the research, the more likely its discussion and dissemination would be on the high end.
Patty Smith of Northwestern University (Meghan Markle’s alma mater it turns out) took up the baton of how one can best make visual use of altmetric data to apply it in supporting researchers when they are making funding applications, or spotlighting media highlights around a certain story. The NIH Biosketch was a specific example, a sort of mini CV where researcher achievements and impact are highlighted.
This sort of use of altmetrics data has become more and more important as more funders start asking for demonstrations and evidence of dissemination and public engagement around research in order to garner future funding. As such, Altmetric data is a simple additive to an existing CV structure.
Two further uses of altmetrics data were also demonstrated by Patty. For larger sets of publications (potentially hundreds of papers) there’s a desire to see rollups of data added together (how many thousands of tweets or how many dozens of stories does a group of items receive.) Unfortunately, this is quite a job to do by hand, as it involves manually counting these items up (Pro Tip - also mentioned by Patty - this is what commercial offerings such as the Altmetric Explorer do out of the box.)
Finally, Patty showed how a timeline of a piece of research, from publication, to its first tweet, to a news story covering the research is readily produced and is easy to understand. This works particularly well even when the “numbers” or score of an item is perceived as low. Patty even showed an example of an item which had 1 news story, 4 tweets and a facebook post. However, it turned out that the news story was a spot on the CBS Newshour, meaning that small numbers nevertheless showed a decent impact when you look at the context itself.
Mike Taylor of Digital Science was the last of our speakers, who began by pointing out that, to date, it looks like almost 500 academic papers have been written and published on the subject of altmetrics. Mike demonstrated via a brief literature review that one can draw an extremely reliable path from pageviews of a publication (which are encouraged by social sharing and dissemination) to downloads (which are encouraged by pageviews) to ultimate citations (which famously correlate with downloads or Mendeley data.) Mike even used an example of a 3 week old bone resorption article which has featured heavily on Twitter, and has since garnered 3 citations; ironically two of those citations were on the subject of a paper’s potential early impact.
Mike also made the important point that other than citation attention, altmetrics can show societal impact (in policy documents - something which Patty Smith had also mentioned was key to demonstrating translational research of medical publications), technical transfer (via tracking when papers are referred to in patent literature) or clinical impact (when clinical trials on the same research are shared.)
All in all an interesting session which evenly covered the purpose, insights and usage of altmetrics data to draw concrete conclusions and desired goals of research dissemination.
The Politics of Altmetrics
Annotations: The Next Altmetric?
Altmetrics Experiences From the Front Line: Publisher and Institutional Use Cases
Research Roundup: A Look at Studies into Altmetrics
Developing Meaningful Metrics - Day 2 of 5AM
Driving User Engagement - 5:AM break out session
Tech Focus - 5:AM break out session
Metrics, Funding and The REF: 5:AM Panel Session
New thinking in altmetrics
Blog: From Numbers to Narratives: An Everyday Tale of Impactful Folk
5:AM keynote: What’s wrong with Science journalism, and how can we fix it?
5 Years of Altmetrics
The Journal of Altmetrics
Media Attention for Your Research
Evaluating Societal Impact of Research
Glossing Over the Truth
Fake Science and Political Online Attention
Looking ahead to 6:AM
Altmetrics for Journal Publishers: Practical tips for demonstrating impact and engaging with Open Research
Metrics beyond Impact: New Approaches for the Novice Researcher
Doing Research Assessment: Implications for the Future of Social Science
When it comes to the UN’s Sustainable Development Goals, how are researchers measuring their impact?
Political instability, Lingua Francas, and Engaging the Local Communities - exploring the challenges facing Africa's Scholarly Communications ecosystem
6:AM – Themes and reflections