Metrics beyond Impact:
New Approaches for the Novice Researcher

Wednesday 4 September 

by Rachel Miles and Amanda MacDonald, Virginia Tech

Over the past year, in my current position as the Research Impact Librarian at Virginia Tech, much of my position has focused on the more traditional uses of research impact indicators in research evaluation and benchmarking. For the past four years, I have focused my research on how librarians, faculty, and researchers use and perceive research impact indicators. In addition, in my previous position at Kansas State University, I sometimes tracked research impact indicators, such as altmetrics, for publications authored by K-State researchers. I was always surprised and delighted when I found mainstream news media attention to these publications.

I soon realized I was also using altmetrics to understand the research I was tracking. Although I was frequently working with digital scholarly objects, I didn’t necessarily always understand the titles of the outputs, let alone the articles themselves. Even when I did understand the titles, I didn’t always understand their significance. After all, I was busy, like most academic librarians.

Despite my busy professional life, a new window had opened for me in the world of altmetrics. The possibilities for enhancing instruction began to spark in my brain like fireworks, but the idea of teaching this concept was put on the backburner for the time being.

When I came into my current position about a year ago at Virginia Tech, I began teaching workshops on researcher profiles, research impact, and altmetrics. Amanda MacDonald collaborated with me to teach workshops on altmetrics to faculty. After I learned she primarily worked with and instructed undergraduate students and researchers, I eagerly told her my idea: let’s teach students information evaluation and information literacy skills with altmetrics.

At first, Amanda didn’t understand what I was frantically trying to explain. I showed her one of the research output examples that got me so excited in the first place. An article on the evolution of multicellularity which was funded by the K-State Open Access Publishing Fund in 2016. I had used altmetrics to demonstrate the publication’s immediate, short-term engagement with the public and its potential long-term impact in an annual report.

I asked her, “Do you understand the significance of this article? Of the evolution of multicellularity?”

“No,” she answered, still a bit perplexed.

“Neither did I, if I’m being honest. But look.” I opened the Altmetric Details page and navigated to the News tab. From there, I selected one of the major news articles so that we could skim it together. “Now, if I’m being really honest, until I read this Washington Post article about it, I really didn’t understand how significant of a discovery this was. I’ve been following up on this research ever since, just out of fascination. If this can help me, imagine how it can help students.”

“Ohhh,” she said, her face going a bit blank with awe, “This is a game changer.”

I continued to show her ways to explore and evaluate scholarly outputs, their attention, and the context behind that attention in Altmetric Explorer (AE), which Virginia Tech subscribes to. (If your institution does not subscribe to an altmetrics database like AE or Plum Analytics, you can use Dimensions to search for scholarly publications and sort by the Altmetric Attention Score.)

I gave Amanda the key, the concept, and the idea, and she ran with it in her own instruction with undergraduate students. Amanda is the Undergraduate Research Services Librarian at Virginia Tech, and our unique interests and fields created one of the strongest collaborations I’ve had the pleasure of participating in.

Our presentation, “Rumor Has It: How Exploring Research Engagement through Metrics Transforms Student Learning” will be presented at the 6:AM Altmetrics Conference in Stirling, Scotland this October. We will discuss how to effectively and carefully incorporate the ACRL Information Literacy Framework when using altmetrics in the classroom and how to use altmetrics to teach undergraduates to disengage from and explore outside their own echo chambers and information bubbles.