Altmetrics Experiences From the Front Line: Publisher and Institutional Use Cases

by Josh Clark, Marketing Executive at Altmetric
Thursday 27 September

One of the most insightful presentations during the 5:AM conference this year was the ‘Using altmetrics: Experiences from the front line’ panel session which focused on how institutions and publishers are employing the data in their day to day workflows to achieve their targets.
Altmetrics Experiences

The panel was chaired by Martijn Roelandse, Head of Publishing Innovation at Springer Nature, with presentations from Paul Naish, Publisher at Taylor & Francis and Dr. Juan Gorraiz from the Department of Bibliometrics at the University of Vienna.

First to speak was Paul Naish. Paul used three case studies about how the Taylor & Francis journals editorial team used altmetrics post article publication to broaden the reach and influence of their publications.

The first case study was about an article published in the Annals of Science in 1962 called ‘Scientist: The story of a word’ about defining the term ‘scientist’. The article recently received new attention on Twitter. Following this new spike in attention, the editorial team decided to make the article free to read which was supported by a social media promotional campaign. Following this campaign they commissioned a new article within the journal about the usage and attention of the original article called ‘The Story of ‘Scientist: The story of a word’. By reacting to the new attention and employing new promotional tactics they were able to further boost attention and usage for the article.

The second case study was about an article about ‘chess robots and the AI uprising’ from the Journal of Experimental & Theoretical Artificial Intelligence. After the paper was published Paul and his team monitored the altmetrics attention it was receiving and found that it was attracting a very high amount of mentions on Twitter. Much of the attention was from users excited about the subject of the paper and comparing it to films like ‘The Terminator’. In response to this, the Editorial team made a comic strip about the subject of the paper and promoted it with a social media strategy which they found boosted both usage of the article and further increased the online attention.

The final case study was around a paper published in ‘The American Statistician’ journal which featured a new and seminal statistics model. The editorial team at Taylor & Francis were aware of the potential impact and online attention that the article would receive prior to its publication. The altmetrics for the paper supported the editorial team’s prediction, and to further boost the attention for the work the paper the team published an interview and did a Twitter Q & A with the Journal’s editor.

Juan was next to speak and started by introducing the variety of bibliometric services the University of Vienna provide for their academics. Juan explained the University believe that if their scientists are going to be evaluated they need to give them feedback on how they can improve their work.

Recently Juan produced an analysis of the metrics the University was providing for their researchers. To do this he did a study to assess the level of adoption of metrics and altmetrics in the scientific community. Juan started by looking into which academic disciplines were most actively using metrics and altmetrics during the years 2014 to 2016. He then analyzed whether there was a correlation between a number of different factors including usage and citation and the intensity of ‘signals’ retrieved for a number of categories including mentions, captures, citations and usage data. The final factor the Juan explored was whether there is a correlation between co-authors and co-affiliations with the intensity of all metrics and measures tracked as part of the study.

To analyze this Juan retrieved a data sample of research containing ‘Austria’ in the affiliation data during 2014, 2015 and 2016. He then drilled down into each discipline by splitting them up by their subject area and recorded the attention in five categories: usage, captures, mentions, social media, and citations. The results showed a diverse spread in the metrics and altmetrics attracted by each subject area.

When looking at the trends in coverage and intensity of attention the percentage of publications with social media attention increased from 28% in 2014 to 33% in 2016. The highest amount of attention across all subjects was from Mendeley readers. Health and Life sciences received the most amount of coverage on Twitter, closely followed by social sciences. Physical sciences and Arts & humanities received the lowest amount of social media coverage. The results of Juan's analysis into the density of attention showed a widespread decrease over time whilst the intensity of tweets increased.

In conclusion, Juan’s analysis provided informative insights that can be used to inform the assessment of research within the University of Vienna. In particular, Juan was interested to see the increased uptake of social media within the research community. However, the research showed the further need to clarify the stability and reproducibility of altmetrics data before they can be implemented in research assessment.