The timing was most fortunate. I was reading “The Opinion Makers: An Insider Exposes The Truth Behind The Polls” during the final weeks of the 2012 presidential campaign. Almost daily, there were new polls revealing the present standing of many candidates and the attitudes of citizens about a dozen social issues. The art of polling, now 73 years of age, had become the accepted tool to measure everything from causes of the fall of Rome to who had last seen Elvis Presley.
One can but ponder why this book carries the title “Opinion Makers” rather than “Opinion Finders” since the book is filled with graphs and studies of what pollsters have uncovered about the opinions and habits of people the world over. Author David W. Moore, based on a career of professional polling, gives his opinion as a clue to the answer. “I wish I could say that the examples cited here are isolated. Unfortunately, they are part of a routine process by which all media pollsters create a public opinion that serves the news media but does not serve the public or its elected representatives.” The rest of the book gives vivid evidence of the causal factors that make a pollster’s job a risky profession.
A quick, preliminary list of obstacles that stand in the way of pollsters ever producing an accurate report of public opinion will equip the mind to understand why Moore gives such a harsh assessment of media pollsters. Foremost are sampling errors such as an unrepresentative base, poor selection of words used in polling questions, public ignorance about the subject being polled, misinterpretation by pundits using the findings, public apathy about events under study, inability to measure the intensity of opinions given and the timing of the poll. No wonder, then, that the results get garbled.
The aforementioned and other obstacles to accuracy resulted in unreliable single polls and great variation when multiple polls took place. One example will illustrate these variations. In 2007, three veteran pollsters — CNN, Gallup (two polls) and CBS — wanted to determine the public support (or lack thereof) for a Bush-recommended “surge” of armed forces to Iraq.
The results are as follows:
• CNN — 64 percent for; 32 percent opposed
• Gallup 1 — 61 percent for; 37 percent opposed
• Gallup 2 — 51 percent for; 46 percent opposed
• CBS — 44 percent; 45 percent opposed
Here we see a move from a 32 percent gap to an almost dead-even reading. The public might never be aware of the huge gaps in these polls when they are separated in time by only a week.
It was noted earlier that the wording of a question could make a significant difference in the way respondents replied to questions. This became obvious in cases where sensitive issues were measured. Was the treatment of prisoners in Abu Ghraib prison, reported in April of 2004, “torture,” “apparent abuse” or just “harsh?” At stake in this need for an accurate response were the demands for the resignation of Defense Secretary Donald Rumsfeld.
Considerable attention was given to such special issues such as how to interpret the number of nonresponders to calls from pollsters, the advent of new technologies such as cell phones and email. Reduced to its essence, it appears that pollsters are able to adapt to these innovations and move on.
During the weeks leading up to the day of the election, my wife and I began following the predictions of pollsters with keen interest. We were especially interested in the predictions of a lone statistician writing for the New York Times. Nate Silver, using a formula of his own creation, began forecasting figures that were quite unique. His forecast called for President Obama to win the electoral vote by a 314 to 224 margin, with a 79.3 percent chance (to 20.7 percent for Romney) of winning the election. Even more amazing was his future prediction which expanded his chance of winning.
Other pundits treated this turn of events as an absurdity. On Nov. 7, an story appeared in the New York Times declaring that Silver “… perfectly predicted all 50 states last night.” There can be little doubt that the methodology used by Silver will have an impact on the future approach to polling by professional pollsters. It is a point worthy of note that the name of Nate Silver does not appear in Moore’s book. Another, even more significant, point is that Republicans couldn’t buy the election.
Allan Powell is a professor emeritus of philosophy at Hagerstown Community College.