cbs storm tracker
As Moore points out, "Obama's average lead each week varies only slightly over the whole month of October," yet the "substantial" variability of the polls drops off significantly in the final week. "Why," Moore asks, "do different polls show such variability over the month of October, and then suddenly converge in the last week of the campaign?" He continues:
Of course, it's true that opinions "crystallize" in the final weeks, but why should that make polls so relatively unreliable during the campaign? Shouldn't polls conducted at the same time produce the same results, even if many people are still mulling over their decisions? Shouldn't different polls find the same proportion of indecisive people?
Moore adds that the editors of Survey Practice will be asking media pollsters for their explanation and analysis of the "convergence phenomenon," reactions that they plan to publish.
I wondered whether we would see the same pattern in statewide surveys. So I replicated Moore's calculation within the twelve battleground states in which we logged 20 or more polls during October and November (Colorado, Florida, Georgia, Minnesota, Missouri, Nevada, New Hampshire, North Carolina, Ohio, Pennsylvania, Virginia and Wisconsin). Then it calculated the mean and median variance across all 12 states, along with the average Obama margin over McCain in each.
As the table below shows, the same general pattern occurs: Even though Obama's average lead remains roughly constant throughout October, the variance in that lead is even larger at the state level before a marked convergence over the last week of the campaign [I have also copied a table showing the values for each state individually, including counts of polls in each, after the jump].
So what's the explanation for this convergence phenomenon? I too look forward to reading the response of the media pollsters in Survey Practice, but I have my own theory. I know that many will wonder, as reader DTM does, "if there was some deliberate convergence." A deliberate "thumb on the scale" may have happened in some cases. But I want to suggest a more benign explanation.
Just about every political pollster that I know cares deeply about the accuracy of that last poll before the election. It is, as the cliche goes, the pollster's "final exam" every two years. Our clients use that final poll as a test of our accuracy, and our credibility and business are at stake every election.
So now put yourself in the position of a pollster who is producing results in any given race that are out of line with other public polls. Trust me, everyone notices, and your paying clients will ask you why your survey is so different from the rest. Every pollster in that position will check everything they can about their own survey and methods -- the sample, the interviewing procedures, the weighting, the screen questions and so on. Keep in mind that even the most by-the-book media pollsters leave themselves a fair amount of room for subjective judgment calls about their likely voter screen questions, models and weighting procedures.
Now consider this basic observation: Pollsters that look hard enough for something amiss will likely find and "correct" it. Those that don't -