There are two aspects of a presidential election that can be predicted: the national popular vote and the relative positions of the states. The national popular vote can be forecasted months ahead of time given the economy and other predictors. for example using Doug Hibbs’s model:

.

(As I wrote a few months ago, “the incumbent party sometimes loses but they never have gotten really slaughtered. In periods of low economic growth, the incumbent party can lose, but a 53-47 margin would be typical; you wouldn’t expect the challenger to get much more than that.”)

The relative positions of the states don’t actually change much from election to election:

You can do slightly better by using polls. As Matthew Yglesias puts it, “the large number of public polls on something like a presidential election makes the outcomes quite easy to forecast based on crude measures. What’s more, even absent polling, Presidential election outcomes seem to be pretty predictable based on nothing more than macroeconomic variables.”

Actually, even the February polls turn out to be pretty good–when combined with previous election results–to pin down the relative positions of the states.

Bayesian combination of state polls and election forecasts

Here’s the revised version of my article with Kari Lock in which we forecast the election using Hibbs for the national popular vote, and a weighted average of last election (corrected for incumbency) and the February polls to get the relative positions of the states.

Lots fo fun stuff there, including this prediction (based on February Clinton-McCain and Obama-McCain polls) of which states Clinton or Obama were expected to win in November:

There’s an idea going around that the Democrats turned in a disappointing performance in Congressional races this year. For example, a politically-minded friend of mine of the liberal persuasion wrote: “The election was good news, although the Democrats did not do quite as well in the Senate and House as I expected. Obama did not have very long coattails–given how anti-Republican Americans are these days.”

Some of the pros say this too; for example, Charlie Cook writes, “given the strength of the top of the ticket nationally, one might have thought that the victory would have been more vertically integrated. . . . what happened down-ballot was not proportional to what happened at the top.”

And Mickey Kaus attributes this to moderate ticket-splitters who, expecting that Obama would win, decided to support Republicans in Congress: “swing voters compensated for the bold, hopeful risk they took on Obama (including for overcoming any race prejudice) by gravitating back toward Republicans in their local Senate and House races.”

The only trouble with this theory is that it’s not supported by the data. Obama won 53% of the two-party vote, congressional Democrats averaged 56%. The average swing of 5.7% from Democratic congressional candidates in 2004 to Dems in 2008 was actually greater than the popular vote swing of 4.5% from Kerry to Obama.

Let’s look at what happened state by state. Here I’m plotting the swing in average district vote in each state, comparing the congressional elections of 2004 to those of 2008, ordering the states by Kerry’s share in 2004:

The horizontal blue line shows the average swing of 5.7%. The Democrats gained in nearly every state, with, unsurprisingly, some big swings in some of the small states that have only one or two congressional districts. Now let’s compare this to the state-by-state swing in the presidential vote:

Obama beat Kerry nearly everywhere, fairly uniformly with only a few exceptions–we knew that–but my point here is that Obama’s swings weren’t quite as large, on average, as the state congressional delegations’.

If you want, you can look at both swings at once:

In the states in the upper left of this graph, the Democrats improved more in the congressional than in the presidential vote; the states in the lower right are those where the Obama-Kerry swing was greater than the Democrats’ swing in House races.

There are a lot more states in the upper left than in the lower right. Each state has its own story–for example, I wouldn’t attribute Don Young’s squeaker in Alaska to Barack Obama’s coattails–but given the graphs above, I think it’s hard to make the case that, overall, the voters were saying No to the Democrats in Congress. On the contrary, congressional Democrats averaged 56% of the vote–their best showing since 1976 (and far more than the Republicans’ 52% in 1994).

Here’s the story in a map:

For some historical perspective, here are the Democrats’ two-party vote share in presidential elections and average two-party vote in congressional elections since 1946:

Presidential voting has been much more volatile than congressional voting (incumbency and all that). This makes the Democrats’ 5.7-point gain over two elections even more impressive.

Summary

I think Charlie Cook was closer to the mark when he wrote, “The political environment and momentum that Democrats seemed to have in recent months may have led to an unrealistic set of expectations. In this, perhaps we pundits share some blame.” I don’t think it makes a lot of sense to consider Obama’s 53% “enormously impressive” and congressional Democrats’ 56% a disappointment.

The data demolish the idea that voters in 2008 were pulling the lever for Barack but not for the Dems overall (not for “Nancy Pelosi,” if you will).

Notes

1. I thank John Kastellec and Jared Lander for gathering the data and sharing their thoughts.

2. I’m counting uncontested House candidates at 75% of the vote (see our earlier article for discussion of this and similar technical issues).

3. We use average district vote rather than total vote because congressional vote totals vary a lot, and we’re trying to assess national public opinion (as judged, for example, in Kaus’s quote above).

4. The Democrats won resoundingly; this means that the voters preferred them to the alternative; it does not necessarily mean the voters want the specific policies proposed by the Democrats. Recall the Democrats’ surprising lack of popular success after 1976 and the Republicans’ struggles after their 1994 sweep. 5. I’m talking about public opinion here, not campaign strategy. I’m sure that Democratic leaders were disappointed in their party’s performance in key congressional races, especially given their immense financial resources this year. At the level of public opinion, though, the Democrats in Congress outperformed Obama overall and in 38 states–and their swing beat Obama’s overall and in 32 states–so I think you’d be hard pressed to argue that the voters were balancing toward the Republicans in congressional voting. This is not to say that the voters have given the Democrats a blank check, but it really was a Democratic swing, not an Obama swing.

P.S. More graphs here.

P.P.S. Kaus replies (via blog):

I don’t understand Andrew Gelman and Matt Yglesias’ point. You don’t win the House of Representatives when you rack up a large percentage of the national “two party “Congressional vote, or when you win a large “average swing” vote on a “state-by-state” basis. You win when you win lots of actual House seats. That’s what can pass or defeat legislation. And measured by actual House seats the Democratic gains (of about 22) were a little less than expected. There is a reason for this.

My response:

1. As noted here, I think the appropriate comparison is 2004 to 2008. Obama did 4.5 percentage points better than Kerry; congressional Democrats averaged 5.7 percentage points more of the vote than their counterparts in 2004. And the Democrats gained many more than 22 House seats since 2004.

2. See my point 5 above. I’m willing to believe that the Democrats’ campaign strategy had problems, or that they underperformed in marginal districts. But, to return to Kaus’s original point about ticket splitters: “maybe there was a determined effort to apply checks and balances. By deciding to elect Obama president, more than a few voters may have opted to keep the Republican incumbent in place, just to prevent Democrats from getting carried away.” I don’t see it. If you want to talk about motivations of voters, I think it makes the most sense to look at vote shares, not just winners.

3. I don’t understand why Kaus puts “two party” and “state-by-state” in quotes. I mean, I guess I do understand, since he’s quoting me (which I appreciate), but I feel like he’s trying to say there’s something fishy about these ideas. But there’s not. “Two party” vote share just means that we exclude third parties and focus on the competition between the Democrats and Republicans. (That’s why, for example, I don’t think it makes sense to compare Obama’s 52% of the total vote to Reagan’s 50%-ish of the total vote in 1980. Reagan competed in a three-candidate race and he did much better than his main opponent, Carter.) I did my “state-by-state” analysis in order to compare Obama’s swing to congressional swings in different places.

To summarize: if the question is campaign strategy–did the Democrats do all they could’ve, or did the Republicans play a poor hand suprisingly well–then, yes, by all means, compare the election outcome to Stu Rothenberg’s and Charlie Cook’s pre-election forecasts. But if you are interested in public opinion–for example, were the voters trying to balance Obama with a more Republican congress–then I think vote swings are more informative.

I think Kaus and I could probably agree that there are two separate questions: (1) Did the voting public favor congressional Democrats (as compared to how they voted for Obama), and (2) Did the Democrats do worse in 2008 than they should have, given their lead in public opinion? I think the answer to (1) is pretty clear: the voters swung toward the Democrats in congress as well as (actually slightly more than) in presidential voting. I have no idea about (2), and I’d defer to Charlie Cook and others on this question. I’m less interested in question (2) but I agree with Kaus that such questions are important, as they affect the size of the Democrats’ majority in both houses.

P.P.P.S. Election outcome compared to anticipated seats-votes curve here.

Update: See here from Steve Kass. Apparently the researchers below made a mistake in reading the data. That sort of thing happens sometimes–that’s one reason it can be helpful to post things on a blog, so that people can catch mistakes.

My bad in not screening this more carefully before posting. In defense of Subramanian and Perkins, they sent me the paper and it was my idea to blog it. They were planning all along to do more systematic analysis of the raw data (which they haven’t yet received).

S. V. Subramanian and Jessica Perkins write:

The prevalence of poor self-rated health was substantially higher among Democrats (25.8%) as compared to Republicans (8.5%). Lower prevalence of poor health among Republicans was also observed in analyses stratified by poverty, education or race. The prevalence of self-rated poor health among poor Democrats or Black Democrats or Democrats with less than high school education were 3 – 14 times higher than comparable Republicans, suggesting that the observed differences in health status by political ideology may not be due to confounding by race or socioeconomic status:

These patterns remain in conditional linear regression models where self-rated health is modeled as a function of political ideological bent (ranging from strongly Democrat to strongly Republican) adjusted for age, sex, race, marital status, income, poverty, education, smoking and religious affiliation.

We also found that more than twice as many Democrats smoke (16.7%) compared to Republicans (7.1%). The prevalence of smoking among poor, less than high school education, or Black Democrats is 3 to 20 times the smoking prevalence among Republicans (see above).

The observation that Republicans enjoy better health status may reflect the core Republican value of individual responsibility, which could translate into increased adherence to health-promoting behaviors. It is also likely that Republicans tend to exhibit greater religiosity compared to Democrats.5 This could lead to health promoting social conditions, such as enhanced social ties and networks, in part facilitated through increased attendance in places of worship.

Our observations should not be interpreted to suggest that a Republican government necessarily equals better health. It would be erroneous to draw macro inferences based on individual-level correlations. Indeed, the question of which type of political ideology, reflected at the government-level, is better for population health or for reducing health disparities are entirely different queries. At the same time, our findings suggest that certain aspects of Republican ideology may implicitly be health enhancing for the individual. The healthier status of Republicans might also explain, perhaps, why they place lower priority on health care.

Interesting. S.V. said they’re waiting to get the raw data so they can run logistic regressions.

P.S. Let me add a few thoughts based on the comments below and at my other blog:

1. To the extent that difference in smoking rates are that large, this would be a big part of the story. But are smoking rates really so low among those groups of Republicans? Perhaps some mistake? I’d really want to look into this before going further.

2. To the extent that the above (or similar) patterns are actually occurring, I think they’re interesting, even if they’re explained by reporting bias or by experiences or by differences between people in big cities and elsewhere, or whatever. As S.V. noted in his article, differences in perceived health–even if explainable by biases–could result in differences in attitudes toward health policy in general.

Len Adleman and Mark Schilling sent along this paper in which they found systematic differences in results from different pollsters:

We compared polls produced by major television networks with those produced by Gallup and Rasmussen. We found that, taken as a whole, polls produced by the networks were significantly to the left of those produced by Gallup and Rasmussen.

We used the available data to provide a tentative ordering of the major television networks’ polls from right to left. Our order (right to left) was: FOX, CNN, NBC (which partners with the Wall Street Journal), ABC (which partners with the Washington Post), CBS (which partners with the New York Times). These results appear to comport well with the commonly held informal perceptions of the political leanings of these agencies.

I guess this makes sense, given that these different news outlets want to make their readers happy. It still surprises me a bit–I thought all these pollsters were pros. It’s not that polling and poll adjustment are easy or automatic–a lot of subjective decisions still need to be made–but I’d think it would be possible to do this without being influenced by your political predilections or those of your audience.

P.S. Some quick comments on the presentation of the results:
- I’d combine tables 2 and 3, and tables 4 and 5.
- I’d remove the second decimal place in Tables 6, 7, 9. Anything less than 0.1 percentage point is both unmeasurable (realistically speaking) and unimportant.
- I’d recommend doing all comparisons relative to the avg of all polls rather than relative to Gallup or Rasmussen. It’s clearer to have a single comparison.
- The x-axis in Figures 2 and 3 are hard to read.
- Table 10 would be clearer as a time-series graph.

Here are two things I kept saying a couple of months ago that I think don’t need to be said any more:

Evidence that the state of the economy is crucial to voters

The nonpuzzle of the close election polls

Drew Linzer writes:

I read your paper with some interest, as within the last week or so I’ve started analyzing the state tracking polls available on pollster.com using a simple Bayesian mean model, and posting my results here.

The model updates the Obama share of the Obama-McCain vote as new polls come in, and then calculates the posterior probability that that proportion is greater than 0.5.

I’m doing this as more of a hobby than anything else…was frustrated with analyses I’ve seen that strike me as overly complex, unstable, totally opaque, and frankly, fairly unbelievable — there’s a real chance Obama could get EVs in the 400s? come on. The R code I use to generate my graphs and predictions are also posted if you click the Data tab.

My goal was to create a model that was very simple, but made reasonable predictions. So, for example, I don’t have any time component — the model assumes the true state level proportion for Obama is constant. Maybe this is a good assumption, probably not, but also probably the actual within-state support numbers are not actually fluctuating as much as some of the predictions out there make it seem. The first time I set up the model, I just used the posterior from one poll as the prior on the next. Problem was that the variance of the priors got so small after a while that there didn’t seem to be enough flexibility to capture trends when they did seem to arise. So then I added a multiplier to increase the variance of the prior in proportion to how many days old the last poll was. I tinkered around with it a bit and came up with this that seemed reasonable.

sd.prior.flex <- sd.prior.flex * (1+(0.05*log(dat$daysold[i]+1)))

Changing the 0.05 makes the trend line more or less sensitive to new polls. It's really just kind of acting as a smoother as the polls appear.

The other thing the model doesn't have is any sort of cross-state correlation structure built in. Every state is treated as its own independent entity. This probably isn't very realistic either, but in these battleground states there also seems to be enough state-level polling going on to get decent enough within state estimates. Where I would like to take account of cross-state correlation is when I simulate election results. Doesn't seem like that would be too hard to estimate in a second stage after the trendlines have been calculated (or simultaneously in a more complicated model), I just haven't gotten around to it.

Anyway, so that's basically it. the "predicted electoral vote" adds up the EVs for each candidate who my trend line has above 50%. And the simulation that produces the "probability of winning" and the histogram just draws from each state's most current posterior distribution 1 million times and adds up the number of Obama EVs. As I said, I don't know if the model is "good" but it is clean and transparent, relatively stable, and, to my mind, produces predictions that accord better with the available polling data.

My thoughts: First, I don’t think it’s impossible that Obama could get EV’s in the 400s, given the uncertainties in national forecasts. It’s not likely but it’s possible. Second, I think poll aggregation is fine (whether it be Drew’s method or Realclearpolitics or 538.com or whatever), but when it comes to forecasts, I think the best thing is a weighted average of polls and model-based predictions, with the model having two parts: (1) the national popular vote and (2) the states relative to each other.

National elections are predictable from fundamentals (see, for example, the research of Steven Rosenstone, James Campbell, Robert Erikson, and Chris Wlezien, along with many others), but this doesn’t stop political scientists, let alone journalists, from obsessively tracking swings in the polls. The next level of sophistication–afforded us by the combination of ubiquitous telephone polling and internet dissemination of results–is to track the trends in state polls, a practice which was led in 2004 by Republican-leaning realclearpolitics.com and now in 2008 at fivethirtyeight.com, a website maintained by Democrat (and professional baseball statistician) Nate Silver.

Presidential elections are decided in swing states, and so it makes sense to look at state by state polls. On the other hand, the relative positions of the states are highly predictable from previous elections. So what is to be done? Is there a point of balance between the frenzy of daily or weekly polling on one hand, and the supine acceptance of forecasts on the other? The answer is Yes, a Bayesian analysis can do partial pooling between these extremes. We use historical election results by state and campaign-season polls from 2000 and 2004 to estimate the appropriate weighting to use when combining surveys and forecasts.

CLICK HERE for more on what we did (research article by Kari Lock and myself).

And here’s an illustration of the method, based on the February SurveyUSA polls of the Clinton vs. McCain and Obama vs. McCain matchups in each of the 50 states:

The short answer is that the polls in individual states–even those large Survey USA polls–have a lot less information than you might think. In some cases the polls are probably telling you something real–for example, in Arkansas, Clinton would do better against McCain than Obama would. But those maps people were making showing which states Clinton or Obama would win–those were drastic overinterpretations of transient poll data.

You can’t just take the state polls, slap on standard errors, and think you’re capturing the uncertainty about the election outcome.

The key idea is to separate the forecasting information at the national level from the information about the relative positions of the states. These are really two different things. The forecasts (and, to a lesser extent, the polls) tell you about Obama and McCain’s strength nationally, and about each candidate’s strength in Ohio (say) relative to his national strength. It’s not statistically efficient to look at Ohio, or any other state, in isolation.

There’s been some discussion of the idea that poll results have to be adjusted for party identification. In general, party ID is pretty stable over time. Vote preference varies a lot more. For a small amount of data on this, see Figures 1 and 10 of our 1993 paper.

You can adjust surveys by smoothing out party ID to get more stable
estimates over time, as we showed in this 2001 paper.

But I don’t know anything in particular about what’s going on in 2008.

I can see why political operatives want to look at these daily polls
but these horse-race numbers seem pretty irrelevant to me. Wherever
the polls stand, both parties have to go all out between now and the
election, both parties have to campaign in the swing states, to take
their best shots, etc. This is all true whether McCain is up by 5 or
Obama is up by 5.

There’s been a lot of discussion recently about Sarah Palin’s broad-based job approval as governor of Alaska. I wondered, though, whether this kind of popularity is so exceptional given the small population of the state. Here are some state popularity data I pulled off the web (here and here):

Alaska is on the left side of both graphs. That’s Frank Murkowski with the sub-25% approval in 2006 (edged out only by Ohio’s Bob Taft), and Sarah Palin with the high rating in 2008.

It seems to be easier to maintain high approval in a small state. What’s going on? Some theories: in a large state, there will be more ambitious politicians on the other side, eager to knock off the incumbent governor; small states often have part-time legislatures and thus the governor is involved in less political conflict; small states (notably Alaska) tend to get more funds per capita from the federal government, and it’s easier to be popular when you can disburse more funds; large states tend to be more heterogeneous and so it’s harder to keep all the voters happy. As the graphs show, the pattern isn’t perfect, but it looks real to me. Next step is to get data from other years.

P.S. A similar point applies to Montana Gov. Brian Schweitzer on the Democratic side. His popularity is impressive but nothing super-special considering he’s in a small state.

I wonder if this pattern could be another source of bias in favor of small states, to the extent that being a highly popular governor is considered to make you considered for national office.

Political conventions may not be as exciting as they once were, but they but they still give nominees a jump of about 5 percentage points. The bounce varies from election to election but it is nearly always positive, typically between 0 and 10% in the polls:

Let’s break down the bounce to see what’s happening among self-declared Democrats, Independents, and Republicans in each year. Gallup has data before and after every convention since 1984 broken down by the partisanship of the people polled.

On average, the bounce is positive among all groups, but the largest changes occur among Independents. This makes sense: the 20% or so of voters who identify with neither party are least likely to have already made up their mind and most likely to change their opinions. What is perhaps more of a surprise is that the bounces are, on average, of comparable size for both conventions: we do not see Democratic voters moving more to their party during the Democratic convention, or the Republicans shifting more during their party’s. The conventions are part of a larger campaigning process by which voters learn about the ideological and policy positions of the candidates. What is relevant is the information conveyed rather than the momentary positions of the polls.