Slanting the Tea Party

I know I could break you down
But what good would that do?
I could surely never know
That what you say is true
--Information Society

This article proposes biased reporting of recent Gallup survey data that gauged support of and opposition to the Tea Party. The Gallup headlines read to the effect that Tea Party support has recently dropped to near all time lows. Is this an accurate message or does it reflect bias?

Here is the historical series associated with the study:

A few things can be observed. When data are plotted over time, the resulting series is well viewed as output of a process. In the case of psychometric data, the process can be viewed as thinking about a subject (whether that thinking is shaped by reason, emotion, or some other driver) and the output as the rendering of some form of judgment, attitude, or perception about the subject over time. The above series can therefore be seen as the output of a thought process that renders judgment concerning the Tea Party in either a positive (% Supporter) or negative (% Opponent) manner.

Of course, there is also a group of people who have rendered neither positive or negative affect about the Tea Party. In fact, this 'neutral' group constitutes a significant majority in this case. That this group is not plotted on the graph or considered in the analysis is a hint of bias. As suggested by the article, why shouldn't the headlines read, "Survey continues to show that over 70% of Americans do not oppose Tea Party?"

When data series are viewed as a process, then the focus of analysis is rarely on a new data point itself. Instead, the focus is on how the data point fits in with previous data points and whether any patterns can be observed. In quality control, which employs such process-oriented analysis routinely, the objective is on whether the process is 'in control' or 'out of control.' Usually, this means assessing whether the process mean has significantly shifted up or down over time.

It does appear the % Supporter mean has shifted down over time. One way to test this is via a simple t-test. There are 16 data points in both of the above series, and the eye is attracted to the decline between the 10th and 11th data points in the % Supporter series, which shows a drop from 27 to 21 as measured in Fall 2011. If we compare the mean of data points 1 thru 10 with the mean of data points 11 thru 16, then we get the following results:

The mean of the % Supporter series is about 5 1/2 points lower since Fall of 2011 with that difference being highly significant. The mean of the % Opponent series is also calculated to be lower since that time, but the drop is smaller and not statistically significant.

The point here is that nothing materially new was revealed by the recent round of data. The new data point of 22 merely adds to confidence that a downward shift in Tea Party support as measured by the Gallup approach has occurred. This shift may have happened two years ago. Since then, it is difficult to construe anything other than that support for the Tea Party has been stationary (i.e., not trending).

The salient question to ask is what caused the drop in measured support for the Tea Party toward the end of 2011? Was there a material change in attitude toward the Tea Party? Another possibility, one that we never learn from Gallup, is that there was a material change in survey methodology that made the step change lower in late 2011 an artifact of the measurement system.

In fact, one change in methodology is readily apparent from the above graph. Frequency of data collection has decreased dramatically since late 2011. According the graph above, only 3 of the 16 surveys have been conducted since the beginning of 2012, and the current data point represents the first one obtained in about a year. Decreasing data collection frequency reduces confidence in the meaning of new data points. There is more unmeasured space between points thus making it more difficult to differentiate between natural ebb and flow in the series versus something that is different from usual.

Stated differently, when examining the above graph it should be readily apparent that we know more about the shape of the % Supporter and % Opponents series prior to 2012 than since then.

Yet, the Gallup report suggests no caution with respect to interpreting the recent data. Instead, it suggests that Tea Party support is indeed trending lower. And near the end of the piece it attempts to tie this 'low ebb' of Tea Party support to recent activities of Tea Party activists to restrain the debt ceiling and Obamacare.

This is the stuff of slant. An objective analyst would not make these assertions because they are not well supported by the data.

One other note is that if we compare the means of % Supporter and % Opponent over the entire series, then they are in a statistical dead heat (26.69 vs 27.75 respectively, p = .938). The fraction of people supporting the Tea Party is the same as the fraction opposing it.

Furthermore, as mentioned by the article, the fraction of Tea Party supporters remains higher than the fraction of Americans who call themselves liberals.

That none of this additional perspective was provided in the Gallup report suggests classic liberal slant.