The psychology in Altmetrics. Brett Buttliere

 

The psychology in Altmetrics. Brett Buttliere

Altmetrics provides unprecedented access into the scholarly process and outcomes of papers. Here we will argue that these data can be beneficially utilized for understanding the scientific endeavor, but also the psychological and sociological principles that lay behind the data.

Understanding the psychology of the individual and the sociological forces behind the data will be important if we want to understand, for instance, why a particular paper was discussed so much, or how science comes to be considered good (and thus what science to fund). It is also important if we want to understand what scientists and funders consider as great science, in general. Human biases will affect what we pay attention to, as scientists are human.

The psychology of what is good science?

Some of the work going on in our lab is looking at this question directly, what do scientists and people generally think is good science. One project analyzed together many of the available Altmetrics, to see if the metrics were generally measuring the same things about the paper (e.g., Quality, Impact) or what they measured and how those things relate to each other (Buttliere, & Buder, 2017). Our analysis suggests that there are three metrics which can best be labeled agnostically as types of attention (without labeling good or bad yet).

Figure 1: Top five most discussed articles across all areas of science during 2015. One can see perfect antibodies, whether vaccinations cause autism, human extinction, why cancer occurs, and whether psychology is reproducible or not. Controversial Indeed! https://www.altmetric.com/top100/2015/

Another way we are examining what scientists and laypeople think is good science is by developing a game where players attempt to identify which papers had the best outcomes on these papers. This game has two benefits, where we can ask what types of papers do people think are good science, and we can see how accurate this is to what is actually considered good science (at least in the Altmetric scores). This game will be coming out in the next weeks, but a Beta can be seen at  https://i2.mnf.uni-tuebingen.de/.All feedback would be great and can be left at the site or emailed to Brettbuttliere@gmail.com

How conflict drives science.

More than asking what types of science people think are good and fund-worthy, the psychological and sociological forces themselves can be studied within the context of Altmetrics. One of the most important forces, in our opinion, both in psychology and sociology, is cognitive conflict, and its related drive toward a unified and consistent understanding of the world. Already in 1950, Leon Festinger and other social psychologists were exploring what scientists and people pay attention to, utilizing a generalized dissonance reduction model, meaning that people will talk more about things they disagree than they agree, especially more of the topics are relevant for group outcomes (Figure 1). This general dissonance reduction mechanism, we argue, is the fundamental motivation behind the entire scientific endeavor.

Table 1: The top 10 most utilized sentiment laden keywords from those papers published by PLoS in 2014. Note that most of them are negative.
Keyword Popularity Hu & Liu SentiStrength AverageScore
1. diseases 8,790 0 -2 -1
2. cancer 5,764 -1 -3 -2
3. disease 4,446 0 -2 -1
4. infectious 4,137 0 -1 -0.5
5. care 3,001 0 1 0.5
6. cancers 2,019 0 -3 -1.5
7. disorders 1,829 0 -1 -0.5
8. risk 1,439 -1 -1 -1
9. infection 1,419 -1 -1 -1
10. stress 1,272 -1 -1 -1

 

One of the major goals of the work my colleagues and I will present at the 4AM Altmetric conference tests these ideas by examining whether how often something is discussed is related to the amount of disagreement in that discussion (Buttliere, Buder, & Costas, 2017).

Festinger’s (1950) hypothesis is that the more we disagree about something, the more we will discuss it (with the goal of reducing uncertainty), especially if that thing is relevant for the group’s outcomes or wellbeing. To test this hypothesis we gathered 162,232 Tweets about 32,870 papers PLoS published in 2014, about 1,961 Microtopics. Aggregating the Tweets to the Microtopic level, we examined whether the number of tweets about the Microtopic was related to the amount of contradictions (e.g., ‘not’, ‘but’, ‘however’) in those Tweets. Overall we found a significant positive correlation (r = .13) and much ‘anecdotal’ evidence in e.g., the most discussed papers from Altmetrics overall.

Another project we are currently working on to better understand the role of cognitive conflict in science utilizes the keywords scientists assign to papers. Utilizing several independent datasets and sentiment analyzers, we basically find that scientists study more negative keywords, more often. Table 1 is a result among 257,728 keywords on 23,385 papers published in Psychology during 2013.

These projects and topics are part of a larger overall project examining the role of cognitive conflict in science. If you would like to hear more, you can come to the presentation on September 27th, between 11:15 and 11:30, at the 4th Altmetrics Conference or email me at BrettButtliere@gmail.com. Thank you!

Works Cited

Buttliere, B. & Buder, J., (2017). Personalizing papers using Altmetrics: approaching Quality or Impact as we would Personality or Intelligence. Scientometrics.

Buttliere, B., Buder, J., & Costas, R., (2017). More discussed scientific topics are more contradicted. 4AM Almetrics Conference, Toronto, Canada.

Festinger, L. (1950). Informal social communication. Psychological Review57(5), 271.