Tuesday, May 28, 2019

Relationship between poll quality and President Trump's (dis)approval rating

Background: President Trump often touts Rasmussen approval rating polls that give him relatively favorable ratings (e.g. [1], [2]). Critics have responded by arguing, among other things, that "Rasmussen's polling has consistently been higher than other polls in evaluating approval for Trump" [3]. This hypothesis was systematically evaluated in the current analysis, as was the hypothesis that higher-quality polls (as measured by FiveThirtyEight's rating system) tended to give the president higher/lower (dis)approval ratings, on average. 

Methods: Trump presidential approval rating polling data was downloaded from this link (by clicking on the link labeled "presidential approval polls" at the bottom). Data were then analyzed to calculate average approval and disapproval ratings for polls with a) a specific letter grade from FiveThirtyEight, b) Rasmussen, and c) just all polls overall, to test for relationships of poll characteristics with the result.

Results: The average approval rating and disapproval rating across all polls was 42% and 54%, respectively. Letter grades for polls by agencies that had such grades assigned to them ranged from a minimum of D- to a maximum of A+. When grades were converted into a numerical score (D- = 1, then increasing by 1 more point for each letter-grade tier (D = 2, D+ = 3, C- = 4, etc.)), there was a strong negative correlation between letter grade and approval rating (r = -0.74), while the correlation between letter grade and disapproval rating proved to be very weakly positive (r = 0.06). There was also a fairly strong negative correlation between letter grade and net approval rating (the latter being % approval - % disapproval; r = -0.60). In addition, Rasmussen did indeed tend to produce higher average approval ratings for Trump than the average of all polls combined (47% vs. 42%), and it also produced lower average disapproval ratings (52% vs. 54%). Trump's average approval rating based only on the highest quality polls (A+ grade) is 41% and his disapproval rating based on such polls is 53%. Notably, this is also based on polls conducted during his entire presidency (so far); as such, it is unsurprising, but reassuring, that the average approval rating of 41% matches closely with the average of 40% estimated by Gallup.

Conclusions: I found strong evidence that higher quality polls tend to produce lower approval rating and net approval rating estimates, but no such effect was found for disapproval ratings, which tended not to systematically vary with FiveThirtyEight quality rating. On average, a 1-point increase in poll quality rating is associated with a 0.48% point decrease in Trump approval rating. Furthermore, I corroborated previous reports indicating that, on average, Rasmussen polls tend to lead to higher approval ratings for Trump (as well as lower disapproval ratings).

No comments:

Post a Comment