3:AM – Status quo, standards, and outlook
This post is contributed by Stacy Konkiel, Outreach and Engagement Manager at Altmetric.
The final session of 3:AM began on a fun note, with Jennifer Kamp and Joe Wass of CrossRef role-playing to explore how complex a seemingly simple conversation on “getting started with altmetrics” becomes when you start asking the tough questions. The pair staged an altmetrics play in several acts, exploring issues such as the use of metrics versus qualitative data for evaluation, differences in data coverage and collection between providers, and the need for responsible interpretation.
The play concluded with Joe highlighting how the forthcoming CrossRef Event Data service will address many of the above issues. The service is being designed to aggregate data from many different sources, accounting for variances in data sharing practices, data normalization, and more in an auditable, transparently documented data stream.
Next, Todd Carpenter from NISO gave an update on the recently completed NISO Altmetrics Initiative. The project brought together stakeholders from academia, industry, and libraries to articulate a definition of altmetrics, personas for potential altmetrics users and consumers, how altmetrics might help raise the profiles for non-traditional research products (specifically, research data), and to design a “code of conduct” for altmetrics data providers that supports the tenets of transparency, replicability, and accuracy for altmetrics data.
Todd emphasized that the culmination of the Initiative did not mean the end of altmetrics work for NISO. The organization seems keen to stay involved in the community by working to foster adoption of altmetrics, and by leading continued maintenance on the standards they’ve developed to date.
Mike Taylor of Digital Science then offered his final thoughts for the meeting, summarizing many of the recurring themes and offering his opinion of the future of altmetrics. He observed that:
- The Journal Impact Factor still reigns supreme in evaluation schemes around the world, meaning that altmetrics advocates have a hard road ahead if we wish to unseat it as the primary measure of research excellence;
- The altmetrics community must articulate the value of our metrics if we wish to be considered alongside the JIF in evaluation;
- We do complicated, nuanced work and analysis using altmetrics, but we can certainly make that work easier to understand to the general public and researchers–and indeed we must, if we wish to drive adoption of altmetrics by “rank-and-file” researchers!
Then a panel of experts joined Mike onstage to share their final thoughts on the future of the field.
Dan Penny of Springer Nature suggested that the future should include metrics that are fun, simple, and interoperable if we wish to drive adoption among researchers. Heather Piwowar of Impactstory opined that innovation will soon come in the form of machine learning and network analysis, moving beyond simple metrics, and that as altmetrics become more widely used by administrators, we should expect to see a pushback from researchers follow. Samantha Gan of NC3Rs predicted that the near future will be driven by funders like her own, who want to incentivize researchers into doing more, especially more activities that might drive true impact. Isabella Peters of ZBW and member of the European Commission’s “Expert Group on Altmetrics” suggested that altmetrics researchers should begin to look more at the data surrounding the impacts of non-traditional research outputs (software, data, and so on), and she also posed a provocative question: is it ethical to continue to develop altmetrics if we don’t know what we’re measuring or why we want to measure it? Lauren Cadwallader of Cambridge University predicted that advances in altmetrics will help researchers better develop public outreach strategies. And Todd Carpenter of NISO surmised that the complexity of the field of altmetrics–apparent throughout 3:AM, held six short years after the field began in earnest–may lead to a period of instability and challenge for the community, one that can be overcome with time and dedication–something clearly in great supply at a meeting that’s now got a dedicated following and robust community of supporters.