Only Five Years Old
This is a guest blog post from Jennifer Lin.
In this blog post, I adopt a naturalistic approach to describing altmetrics both for rhetorical amusement, but also to foreground the temporal dimension of altmetrics development. The simple heuristic has been employed to offer a quick glance back and a quick glance ahead, not to ascribe a teleological trajectory that predetermines an altmetrics end state.Only five years old
At this age, a fly has outlived its life expectancy by 4 years & 11 months. A whale has moved out of its mother’s pod and is readying itself to mate. A female panda is fully mature and is already bearing cubs. And as for the slowest of all species, humans are just beginning to learn calculus, build houses, and write symphonies. Almost. But take a counterexample, with a more rapididly growing organism, altmetrics. How does it stack up? What stage of development is it?
I’ve been pondering this question as myself and fellow 2:AM organisers plan for this year’s altmetrics conference. Altmetrics is no longer an infant: early indications already show increased complexity of function and skill progression. It has acquired certain competencies and a rudimentary set of modes of expression. After a long gestation and infancy, altmetrics has reached early childhood at only five years old.
The release of the altmetrics manifesto is often considered the “birth moment” in 2010, alongside the ALM project in its early days at PLOS. Since then, a small group of four visionaries (Paul Groth, Cameron Neylon, Jason Priem, & Dario Taraborelli), and a single publisher, has grown into an established framework where a proliferating web of tools and services adopted across the publishing sector are transforming way the ways in which we think & talk about the dissemination of scholarly research.
At the start, altmetrics was written off as silly bean mash. Turns out it’s far easier to scour the web and track the footprint of research articles than change entrenched views that the existing norm, the journal impact factor – already old and enfeebled – was in dire need of help.
Altmetrics is five now, and it has gained considerable ground, even some legitimacy. The most convincing indicator of progress (and potential) in my view is the dramatic shift in the nature of the conversation at the highest levels. Decision makers have begun to regard altmetrics in a genuine and serious manner. Multiple affirmative dimensions to this discourse have taken shape. With authentic consideration comes real engagement. And subsequent to that will come more accelerated progress.
The Metric Tide report is one such example. It found considerable scepticism of the broader use of metrics in research assessment and management in its survey of researchers, universities, representative bodies, and learned societies. But the process itself and the ensuing discussions of the positive and negative or unintended effects of metrics on various aspects of research culture have all been part of the broader development of altmetrics. This is a critical milestone. That such conversations have been increasingly more inclusive, conscientious and substantive is also noteworthy. The diversity of opinions, approaches, and methodologies in the early childhood stage is not only an expected phenomenon but a healthy one.
Legitimation goes hand in hand with its formalization for altmetrics. We have witnessed this evolving in established spaces:
- across publishers and the infrastructure organizations (CrossRef DOI Event Tracker, NISO)
- across mainstream technology providers (Highwire, EBSCO, Scopus)
- across funders, policy makers, and governmental bodies (HEFCE and Research Excellence Framework)
- across university re-evaluations of their assessment framework and management systems to accommodate a broader set of quantitative indicators
Indeed, the number of local systems impacted is rapidly increasing as altmetrics heads towards the next stage.
What can we expect for altmetrics once it reaches adolescence? Data integrity is an issue inherent in all information systems. But it will become a standard component to data provisioning if we proactively grow altmetrics according to best scholarly practice (i.e., the practice of science itself) and make underlying data transparent, monitorable, and reproducible. Such data infrastructure will need mindful planning and construction and done so in a way that fairly and equally supports the entire scholarly community.
And as trust in the validity and reliability of the data grows, greater functional diversity will become possible. The application of altmetrics for literature discovery is relatively untapped. We will see it integrated into tooling, but more importantly in the actual daily practices of researchers and others seeking to navigate the overwhelming mounds of publications.
The science of altmetrics will also make far greater headway as we see bibliometricians further test the hypotheses and assumptions that undergird this enterprise. Subject matter experts across research areas will plumb the ethical issues surrounding performance management, equality, diveOnly Five Years Oldrsity, and interdisciplinarity. As it comes of age, we will develop complex statistical models to provide context to the measurements along with more sophisticated modes of interpretation, no longer boiling down complex judgements to simple scores and numbers. Additionally, the fruits of adolescence will lead to applications of measurement that extend to other outputs such as data, software, etc. As that unfolds in a substantial manner, we will obtain a far more complete portrait of activity when altmetrics data are linked together across associated research products.
To lay the groundwork for the next stage, we have organized the 2:AM conference program to reflect these very areas. High-level highlights include:
- continued work in standards development for altmetrics data gathering and presentation
- impact of altmetrics on existing roles within the research enterprise from library services to open science movements to those measuring economic and social impacts of research and the new opportunities for them
- ethics, transparency, and cultural issues surrounding altmetrics applications
- altmetrics for research outputs beyond the paper: data, software, posters, simulations, models, etc.
We hope to encourage cross-pollination and deep engagement on the very topics already recognized as fundamental to altmetrics adulthood by convening representatives across research sectors together at the conference.
Adulthood and beyond?
Altmetrics’ approach to adulthood might be a steep and awkward ascent. But it is also one that leads to a broad set of attributes fully expressing its capacities in the native environment. For some time, the failings of our existing system to adequately validate research outputs have been well understood. Altmetrics will play an integral role here in the future. We cannot create a normative framework sufficient for the values intrinsic to the research enterprise without the incorporation of expert judgment, carefully selected quantitative indicators, and qualitative measures (James Wilsdon’s ‘variable geometry’). Altmetrics will then need to figure out how to “talk to” peer review (and vice versa).
Rather than striving for a single definition or measure of quality illiberally meted out across all contexts, altmetrics will be part of a larger discussion on the ‘qualities of quality’, introjecting dimensionality to a fundamentally heterogeneous concept. We will more fruitfully engage with a broader discussion on what quality, value, impact actually means with a meaningful corpus of data at our disposal.
There seem to be few though critical risk factors that will impact continued altmetrics development: a) lack of access and validation of the raw altmetrics data and b) lack of integration with a broader scholarly infrastructure comprised of unique identifiers for people, institutions, and research outputs (or worse yet, lack of infrastructure itself). Effects that hinder community trust will most certainly lead to delays in maturation. That said, I am encouraged to see altmetrics already materially contributing to a broader reevaluation of our current practices of research assessment. It is also heartening that the broader research enterprise has arrived a juncture of productive reflexivity, recognizing the systemic and potential effects of metrics and then improving such indicators (measurements and modes of use) in response.
These are propitious days for all of us. Five years ago, the altmetrics manifesto closed with a statement that still rings true:
Altmetrics are in their early stages; many questions are unanswered. But given the crisis facing existing filters and the rapid evolution of scholarly communication, the speed, richness, and breadth of altmetrics make them worth investing in.
Today, we have a better handle on some of the fundamental questions, though many questions remain unanswered. This work is only accomplished over time through wholesale engagement across the research ecosystem and as with almost all real things (as do the fly, panda, whale, and child), learning by living. The first five years were a doozy. These are even headier times, and I very much look forward to the next ones ahead. Come join us at 2:AM with our birthday party hats on, and let’s spur on the next stage of altmetrics together! (NB: There WILL be cake.)