In this political season, the results of polling have proven as problematic as they have consequential. Questions of validity increasingly plague polls at the very moment that the body politic most bends to their will (or to that of the media outlets reporting on them).
Moreover, media increasingly adopt a strategy of first framing and then reporting public attitudes, at times appearing to shape polling outcomes while claiming to be merely mirroring them.
A case in point is the latest polling news on public attitudes towards legal marijuana, as reported by the Washington Post. Support has reached its highest level ever, according to the latest survey conducted by the Associated Press in conjunction with the National Opinion Research Center (NORC), reaching 61 percent of 1,042 respondents in favor.
But there is less here than meets the eye. For instance, the previous high point of approval ratings, 58 percent as detected by Gallup, is a finding within this poll's margin of sample error (3.9 percent). That is, the result represents, technically, "no change" from a statistical point of view.
But that was not the perspective of the media, whose enthusiasm approached "agenda journalism," the kind that celebrates every outcome favoring the journalist's preferred stance, while denigrating or ignoring contrary evidence.
They should have been more cautious. Consider that of the 639 people who supported legal marijuana, only 33 percent of them thought access should be "unrestricted," while for 24 percent, their support was contingent on there being "a medical prescription."
The framing of this last response is itself misleading, in that the question illegitimately implies that marijuana is an "approved" medication, able to be prescribed, which is true nowhere in the country.
Further, the poll's demographics are suspect, such as in their assignment of political affiliation. In this poll, only 35 percent of respondents are either "Republican or Lean Republican." The comparable Democratic percentage is set at 48 percent.
Given the finding in Quinnipiac polling that Democratic support in Colorado for legal marijuana (74 percent) is higher than that of Republicans (36 percent), this 13-point lean towards Democrats may shade results.
But the most surprising feature of the sample can be found with regards to employment. Fully 43 percent of respondents are not employed (school-aged 18-24 year olds are only 12 percent of the total).
This distribution may be a function of the real Achilles heel of the survey – the response rates, which attained a "cumulative response rate" of only 10.5 percent. That is, the survey had to winnow through a hundred contacted persons in order to measure the actual attitude of fewer than eleven. A similar, largely hidden fault now threatens nearly all political polls.
The lesson is that a survey meant to represent overall American attitudes has non-response rates that may yield a non-representative group of people – here, it is those not in the workforce, and hence, available for surveys such as this. It is possible that people not in the workforce have different attitudes regarding the acceptability of drug use than do the employed, a conjecture buttressed by the strong association between actual drug use and non-employment status.
There is finally the selective reporting by the media on the contents of the poll. Only the positive marijuana attitudes were reported, while the actual survey contained considerably more news, and much of it negative regarding legal drugs.
First, a remarkable 43 percent reported "a relative or close friend" with a substance use problem. Moreover, several Administration positions regarding the current epidemic of opiate dependency and overdose deaths did not receive support.
Fully 67 percent thought that doctors were not "a cause or only a minor cause" for the current epidemic of opiate dependency and overdoses, notwithstanding Administration arguments to the contrary. And while distributing the overdose antidote Naloxone was supported, 57 percent believed that it should require a doctor's prescription. (Regrettably, a much-needed policy to reduce the drug supply was not a provided survey question option.)
Nearly 70 percent agreed that drug use treatment needs improvement, yet 61 percent wanted more effort to "crack down on drug dealers" (and 53 percent supported "cracking down on users"), running contrary to Administration sentiments that "we can't arrest our way out of the drug problem."
Finally, even though there are now multiple advocacy calls for the government to establish "safe" injection facilities or even the provision of heroin itself for heroin treatment, only 7 percent (69 people) in this poll thought that heroin or cocaine should be legal, while 73 percent of even those supporters thought that there should be "restrictions."
None of these issues or responses was mentioned in the Post story. Instead, we find the typical self-confirming spiral regarding marijuana legalization. In Gallup polls, support for legal marijuana hovered in the 30 percent range until 2009, when it finally rose higher than 40 percent. Since then, a steady drumbeat of news accounts has relentlessly driven higher the attitudes that are conducive to Obama Administration positions, followed by "news" chronicling this higher public support, as ostensibly shown by polls.
This polling/reporting news cycle, one realizes, owes more to politics than to science.