Tuesday, December 18, 2018
New paper: the accuracy of FiveThirtyEight's 2018 election predictions: an exploratory analysis
I submitted a paper with this title to SocArXiv, which you can read here in the unlikely event that you want to. (The content of that paper was originally posted here but it has since been removed, 'cause there's no need for it to be in 2 places at once.)
Friday, December 7, 2018
Stereotype accuracy part II
(Introductory author's note: all quotes in this post that I did not write will be in Courier font, but everything else will be in Times New Roman.)
In a previous post, I looked at the obviously fishy claims that Rutgers social psychology professor Lee Jussim and his colleagues (but especially Jussim himself) have been making regarding the purported accuracy of stereotypes. Before broadening this post to look at the many questionable arguments Jussim has made about many other topics, I will point out that Jussim's claim that stereotypes are usually very accurate (despite being false) has crept its way into a number of recent peer-reviewed papers that cite it as evidence of a supposedly widespread, structural bias in favor of liberal views in social psychology. Consequently, in this post I will critique some recent articles not written by Jussim or his colleagues, but which cite their research on stereotypes and portray it in a favorable light.
So first I need to sum up the argument being made by those advancing Jussim et al.'s claims about stereotype accuracy: ostensibly, there is overwhelming evidence that stereotypes are moderately to highly accurate, but liberal social psychologists (i.e. almost all social psychologists), blinded by their ideological preconceptions, refused to even approach or consider this evidence. Martin (2016), for instance, claims,
In the field of behavior genetics (BG), this style of argumentation often manifests as behavior genetics researchers calling their critics "blank slatists", or saying they have some sworn ideological allegiance to total environmental determinism/the standard social science model when explaining human behavior. This lets BGists portray themselves as offering the reasonable idea of maybe letting genes be part of the equation that leads to human behavioral traits, as an alternative to those nutjobs who want to pretend that human genes and evolution don't even exist. Here we see Martin similarly using this approach to avoid addressing specific points made by one's critics, and instead trying to elicit sympathy from readers by portraying the author as under attack by the PC brigade that supposedly controls the vast majority of academia.
Where was I? Oh yeah, Martin's article. Martin was saying that the idea of stereotypes being accurate a) could've been researched empirically for a long time, but b) wasn't researched empirically nearly as often as it could have been, because c) almost all social psychologists were blinded by the taboo against such research by their supposedly all-encompassing liberal ideologies. Further, he claims that d) when a handful of brave, Galileo-like mavericks finally stood up to the leftist cabal that rules almost all of the social psychology field with an iron fist, e) they proved that stereotypes are actually very accurate, on average, which f) proves that stereotypes arise from accurate perceptions of reality, not prejudice.
Before addressing these arguments I want to point out another fundamental issue with the "stereotypes are accurate" argument I did not mention in my previous post on Jussim's work in this area. Specifically, as Jussim himself acknowledges, there is not a single dimension of "accuracy" on which a perception or belief can be assessed, but rather several possible "scales" on which one may attempt to do so. In a 2015 journal article, for example, Jussim et al. note that there are two distinct ways that stereotype accuracy can be assessed. These two ways are discrepancy scores and correspondence. As Jussim et al. further explain,
Though the above points I made seem troubling (though I'm hardly an impartial judge of how compelling my own arguments are), I think that the biggest problem with this research is that, ironically, it stereotypes stereotypes themselves by referring to them in blanket terms as "accurate". This ignores not only the highly problematic nature of referring to entire groups as all possessing a characteristic without acknowledging variation within groups on that characteristic, but also the fact that even by these researchers' own criteria, some stereotypes are decidedly inaccurate. Political stereotypes, for instance, were said to "exaggerate group differences" by Jussim et al. (2015). In addition, these authors note that "Empirical reports based on independent samples from around the world (e.g., McCrae et al., 2013) have consistently found little national-character stereotype accuracy". Consequently, blanket statements about the "accuracy" of stereotypes serves to commit the very fallacy of generalization that psychologists have been criticizing stereotypes for for decades now: it ignores that not all members of group x (in this case, stereotypes) have characteristic y (in this case, accuracy).
Sources
Jussim et al. 2015
Lockenhoff et al. 2014
Panofsky 2014
Martin 2016
In a previous post, I looked at the obviously fishy claims that Rutgers social psychology professor Lee Jussim and his colleagues (but especially Jussim himself) have been making regarding the purported accuracy of stereotypes. Before broadening this post to look at the many questionable arguments Jussim has made about many other topics, I will point out that Jussim's claim that stereotypes are usually very accurate (despite being false) has crept its way into a number of recent peer-reviewed papers that cite it as evidence of a supposedly widespread, structural bias in favor of liberal views in social psychology. Consequently, in this post I will critique some recent articles not written by Jussim or his colleagues, but which cite their research on stereotypes and portray it in a favorable light.
So first I need to sum up the argument being made by those advancing Jussim et al.'s claims about stereotype accuracy: ostensibly, there is overwhelming evidence that stereotypes are moderately to highly accurate, but liberal social psychologists (i.e. almost all social psychologists), blinded by their ideological preconceptions, refused to even approach or consider this evidence. Martin (2016), for instance, claims,
"...stereotype accuracy has been considered a taboo topic, and only a small number of researchers have investigated if stereotypes are accurate (e.g., Jussim 2012b). Much of this research has shown that stereotypes are indeed accurate (on average), particularly in direction. These findings contradict the assertion by some scholars that stereotypes primarily arise from intergroup envy or scorn (e.g., Fiske 2010). Rather, they develop from valid observations of the social world. Far from being the foolish mistake-makers that social psychologists have made them out to be (Baumeister 2010), humans are mostly perceptive observers. Were it not for the taboo against accuracy research, this scientific discovery might have occurred earlier."Hoo boy, there's a lot of bullshit there! Firstly, we see the repeated victim mentality of anyone pushing a controversial claim that they claim is supported by strong scientific evidence: they are attacking their critics as motivated by political correctness and reluctant to even touch certain oh-so-controversial topics with a ten-foot pole because of their fear of "taboos". This is reminiscent of the argument style behavior geneticists often use, which also involves accusing their critics of political, rather than scientific, motivations. Aaron Panofsky's 2014 book Misbehaving Science refers to this style of (ad hominem) argumentation as "hitting-them-over-the-head" style. Panofsky states that the goal of this discursive style "...was not to seek synthesis, integration, or sober rational persuasion but to engage in polemical scientific attack, declaring themselves as crusaders who would rout the antigenetics heresy gripping behavioral science" (Panofsky 2014, p. 142).
In the field of behavior genetics (BG), this style of argumentation often manifests as behavior genetics researchers calling their critics "blank slatists", or saying they have some sworn ideological allegiance to total environmental determinism/the standard social science model when explaining human behavior. This lets BGists portray themselves as offering the reasonable idea of maybe letting genes be part of the equation that leads to human behavioral traits, as an alternative to those nutjobs who want to pretend that human genes and evolution don't even exist. Here we see Martin similarly using this approach to avoid addressing specific points made by one's critics, and instead trying to elicit sympathy from readers by portraying the author as under attack by the PC brigade that supposedly controls the vast majority of academia.
Where was I? Oh yeah, Martin's article. Martin was saying that the idea of stereotypes being accurate a) could've been researched empirically for a long time, but b) wasn't researched empirically nearly as often as it could have been, because c) almost all social psychologists were blinded by the taboo against such research by their supposedly all-encompassing liberal ideologies. Further, he claims that d) when a handful of brave, Galileo-like mavericks finally stood up to the leftist cabal that rules almost all of the social psychology field with an iron fist, e) they proved that stereotypes are actually very accurate, on average, which f) proves that stereotypes arise from accurate perceptions of reality, not prejudice.
Before addressing these arguments I want to point out another fundamental issue with the "stereotypes are accurate" argument I did not mention in my previous post on Jussim's work in this area. Specifically, as Jussim himself acknowledges, there is not a single dimension of "accuracy" on which a perception or belief can be assessed, but rather several possible "scales" on which one may attempt to do so. In a 2015 journal article, for example, Jussim et al. note that there are two distinct ways that stereotype accuracy can be assessed. These two ways are discrepancy scores and correspondence. As Jussim et al. further explain,
"One method of assessing accuracy is not “better” than the other; each contributes unique information (Jussim, 2012; Ryan, 2002). Discrepancy scores indicate how close perceivers’ stereotypes come to being perfectly accurate (scores of 0 reflect perfect accuracy). Correspondence indicates how well people’s beliefs covary with criteria" (Jussim et al. 2015, p. 492).So it would behoove those who want to make confident claims about the "accuracy" of stereotypes to make sure that they use both methods (or take into account studies that do so) before concluding that stereotypes are either accurate or inaccurate. So surely, when Jussim et al. (2015) claim in their paper's abstract that the accuracy of stereotypes is "one of the largest and most replicable findings in social psychology", they are basing this on both types of studies, right? This is not at all the impression you get from their table 2, which claims to present, and I am not making this up, "Stereotype Accuracy Correlations From Over 50 Studies Showing That Stereotypes Are More Accurate Than Social-Psychological Hypotheses". They appear to only be paying attention to correlations between perceived and actual group characteristics, without paying attention to discrepancy scores, in coming to the obviously provocative conclusion that stereotype accuracy is actually greater than that of social psychological hypotheses collectively. That being said, they do acknowledge the existence and results of discrepancy-score studies bearing on this topic, e.g. when they say "Although not every study examined discrepancy scores, when they did, a plurality or majority of all consensual stereotype judgments were accurate. For example, an international study of accuracy in consensual gender stereotypes about the Big Five personality characteristics found that discrepancy scores for all five reflected accuracy (Lockenhoff et al., 2014)." However, it should be pointed out that it is more difficult to assess the relative "accuracy" of psychological hypotheses and stereotypes when the latter are assessed based on discrepancy scores (e.g. 1 SD) rather than correlation coefficients. Further, the statement that Lockenhoff et al. (2014) "found that discrepancy scores for all five reflected accuracy", referring to the Big Five model of personality traits (Neuroticism (N), Extraversion (E), Openness to Experience (O), Agreeableness (A), and Conscientiousness (C)), seems to be rather at odds with the following quote from that very paper (Lockenhoff et al. 2014, p. 685): "Across all facets of N (and for N1: Anxiety in particular), assessed sex differences appeared to be more pronounced than GSDs [gender stereotype differences], and this was true for both self-reports and observer-ratings."
Though the above points I made seem troubling (though I'm hardly an impartial judge of how compelling my own arguments are), I think that the biggest problem with this research is that, ironically, it stereotypes stereotypes themselves by referring to them in blanket terms as "accurate". This ignores not only the highly problematic nature of referring to entire groups as all possessing a characteristic without acknowledging variation within groups on that characteristic, but also the fact that even by these researchers' own criteria, some stereotypes are decidedly inaccurate. Political stereotypes, for instance, were said to "exaggerate group differences" by Jussim et al. (2015). In addition, these authors note that "Empirical reports based on independent samples from around the world (e.g., McCrae et al., 2013) have consistently found little national-character stereotype accuracy". Consequently, blanket statements about the "accuracy" of stereotypes serves to commit the very fallacy of generalization that psychologists have been criticizing stereotypes for for decades now: it ignores that not all members of group x (in this case, stereotypes) have characteristic y (in this case, accuracy).
Sources
Jussim et al. 2015
Lockenhoff et al. 2014
Panofsky 2014
Martin 2016
Subscribe to:
Posts (Atom)