There are two aspects of a presidential election that can be predicted: the national popular vote and the relative positions of the states. The national popular vote can be forecasted months ahead of time given the economy and other predictors. for example using Doug Hibbs’s model:

.

(As I wrote a few months ago, “the incumbent party sometimes loses but they never have gotten really slaughtered. In periods of low economic growth, the incumbent party can lose, but a 53-47 margin would be typical; you wouldn’t expect the challenger to get much more than that.”)

The relative positions of the states don’t actually change much from election to election:

You can do slightly better by using polls. As Matthew Yglesias puts it, “the large number of public polls on something like a presidential election makes the outcomes quite easy to forecast based on crude measures. What’s more, even absent polling, Presidential election outcomes seem to be pretty predictable based on nothing more than macroeconomic variables.”

Actually, even the February polls turn out to be pretty good–when combined with previous election results–to pin down the relative positions of the states.

Bayesian combination of state polls and election forecasts

Here’s the revised version of my article with Kari Lock in which we forecast the election using Hibbs for the national popular vote, and a weighted average of last election (corrected for incumbency) and the February polls to get the relative positions of the states.

Lots fo fun stuff there, including this prediction (based on February Clinton-McCain and Obama-McCain polls) of which states Clinton or Obama were expected to win in November:

Ben Lauderdale writes:

I [Ben] had this map [see below] on my door for the last week. Based on exactly the same calculation using constant 95% black support and census-proportional representation. The white counties are the ones whose census names didn’t match properly with the names used in the library(maps) package in R, I was too lazy to fix them.

Cool. I’d only suggest using light gray rather than heavy black lines between counties; the map as it is overemphasizes the county borders, I think. But I respect his laziness; there’s always time later to fix the details.

Ben continues:

[Below are] the state-by-state county share plots for the lower 49, Obama vote share as a function of black population share. V.O. Key’s observation that whites who live near blacks in southern states are less positively inclined towards them is *still* visible in several states.

The circle areas are proportional to county voter turnout. (The biggest circle is L.A. county in California, and so forth.)

Ben also had this comment about his map:

It reminded me of something Bob Putnam would say every time someone presented an empirical talk in our Center for the Study of Democratic Politics series during the year he was a fellow here at Princeton: “You should include miles to the Canadian border as a variable in your regression, it is the most important proxy for political culture in America!” At least in the eastern half of the country, he has a point.

Except for New Hampshire and Vermont, I think.

P.S. Further discussion here of graphing possibilities.

Jeremy Teigen (link from John Sides) says the answer is pretty much No. Here’s Teigen:

I [Teigen] examined contested House races from 2000-2006 to see if candidates’ previous military service helped them garner a higher share of the vote. . . . the effect of being a Democratic veteran or a Republican veteran on vote share, controlling for presidential vote share in the district, gender, incumbency, campaign spending, and other relevant factors. . . .

In general, veteran status has small effects that are not statistically distinguishable from 0. Democratic vets did better than their nonveteran peers in 2002, but did no better in 2006. That election was the year that Joe Sestak, Tammy Duckworth, and others constituted the “Fighting Dems,” a year when you would expect Democratic vets to do well, but instead Republican veterans were helped by a martial past. . . . In 2006 Democratic veterans actually did a little bit worse than Democrats without a service record. This result may have occurred because Democrats were overzealous in their attempts to attract veterans as candidates, leading them to select veterans over higher quality challengers (14% of Democratic challengers were vets in 2002, compared to 28% in 2006). Republican vets running that year performed a little better than nonveteran Republican candidates, as they had been doing in the previous three elections, but the advantage just slipped above statistical significance. Overall, the effect of veteran status is very small.

It would be interesting to see data from other years and comparisons to other occupations (community organizers? small-town mayors? POW’s?).

Teigen continues:

Nevertheless, this does not seem to deter veterans from running for office and promoting their service during their campaign. Despite the allegedly “tinny ring” of military values, some candidates make their military service the key element of their campaign narrative (e.g., Craig Williams). To me, this behavior and the continued emergence of veteran candidacies says that our candidate selection mechanisms still value military service even if the general election yield is inconstant and small.

I actually disagree with this bit, or at least I’m not yet convinced by the evidence. Every candidate has some strength. If you’re a veteran, you can make a big deal about that. If you’re a trial lawyer and won cases for sick kids, you can advertise that. If you’ve been a community organizer, ditto. The lack of any special advantage for veterans does not at all imply that it’s silly for veterans to promote their service during their campaigns.

P.S.

Some technical comments: I don’t think it’s a good idea to control for campaign spending. That’s an intermediate outcome (see, for example, chapter 9).

On the plus side, I agree that it makes sense to analyze House races (larger N than just analyzing presidential contests), and I’m glad to see vote share rather than win/loss used as an outcome.

Michael Herron sent me this article-in-progress by Jonathan Chapman, Jeffrey Lewis, and himself on residual votes in the 2008 Minnesota Senate race. They conclude:

In the Minnesota Senate case there is no doubt that the number of residual votes dwarfs the margin that separates Coleman from Franken. We show using a combination of precinct voting returns from the 2006 and 2008 General Elections that patterns in Senate race residual votes are consistent with, one, the presence of a large number of Democratic-leaning voters, in particular African-American voters, who appear to have deliberately skipped voting in the Coleman-Franken Senate contest and, two, the presence of a smaller number of Democratic-leaning voters who almost certainly intended to vote validly in the Senate race but for some reason did not do so. . . . At present, though, the data available suggest that the recount will uncover many of the former and that, of the latter, a majority will likely prove to be supportive of Franken.

In a discussion of the historic nature of Barack Obama’s election, Christopher Hitchens writes, “there were not a few elected black American representatives 40 years ago.”

This claim surprised me, so I looked it up. In 1968, there were 5 African Americans in the House of Representatives and 1 in the Senate. This sounds like only “a few” to me! Was Hitchens just confused here, or am I missing something?

P.S. Somebody pointed out that there were black state and local officeholders as well. I guess it all turns on what is meant by “not a few.” Blacks were certainly a very low percentage of all U.S. elected officials back then.

In yesterday’s blog entry I looked that the swing in congressional voting nationally (House Democrats gained 5.7%, on average, compared to 2004) and by state (compared to 2004, House Democrats gained in nearly every state). My graphs elicited several interesting comments including this from Steve Sailer:

Perhaps the reason that the GOP House losses of seats were considered not so bad compared to 2006 was because in 2008 the Democrats ran up huge turnouts in black-represented Congressional districts, which were already all Democratic?

Let’s look at some district-by-district swings, starting in 2002:

Here, I’m excluding uncontested elections and those in which the challenger got less than 10% of the vote; dots indicate incumbents running for reelection, circles are open seats, and red points are those with black representatives as of 2008. (I just pulled the names off the Congressional Black Caucus website and didn’t try to go back to earlier years on this.)

What happened? Overall, the Democrats gained a bit in 2004, a lot in 2006, and some in 2008. But we knew that (see the time series plot in the blog entry linked above). We also see a bit of scatter. Beyond this, yes, there are some patterns. In 2006, the Democrats particularly gained in Republican areas–see how those dots in the lower left of the second graph are way above the 45-degree line? In 2008, the swing is more uniform. (In addition, the black Democrats did pretty well in 2008 compared to 2006, but it doesn’t seem like a big part of the story.)

Returning to the “How well did the Democrats actually do in 2008″ question, I think that one problem is that people are comparing Obama’s vote to Kerry’s vote but then comparing the congressional Democrats in 2008 to the congressional Democrats in 2006. I think it’s more appropriate to compare 2008 to 2004 in both cases. As Paul Krugman put it, “Maybe the reason people don’t see this is that the Democratic House gains were spread over two elections.”

Andrew posted earlier about how wealthier voters again voted disproportionally Republican. What about states? Did rich states vote Democratic in 2008, as they did in 2004? Did poor states vote Republican? In short, yes, they did.

The two raw data scatterplots show the average per capita income of states in the election year (in 2006 dollars) on the horizontal axis, and the Republican share of the vote on the vertical axis. A best-fit line is drawn in each year. The income-vote slope decreases a small amount in 2008. Overall, though, rich states are still blue, and poor states are still red.

So how can rich states vote Democratic, while rich individuals vote Republican? And is this pattern new? Buy our book to find out!

Next step: multilevel models of individuals nested in counties and states. That’s going to have to wait until we get the exit poll data.

We’ll have to wait until we’ve had a chance to analyze all the election returns, but so far the Democrats appear to have won 59% of the House seats, which according to the recent article by Kastellec, Chandler, and myself, suggests that Democratic House candidates averaged about 54% of the two-party vote.

Historically, both these numbers are pretty strong, bringing the Democrats to something near their position in the 1980s, and doing quite a bit better than the Republicans in 1994:

In particular, the Democrats did about as well, on average, in the House, as Obama did in the presidential tally. I haven’t yet looked at the numbers but I suspect that something similar happened with the Senate: even after just missing a few Senate seats at 51/49 and 52/48 margins, the Democrats still picked up a few.

The next thing to see is where the vote was changing; based on what’s happened before, my guess is approximate uniform swing.

After a quick look at the election results and exit polls (from www.cnn.com), some thoughts:

1. The election was pretty close. Obama won by about 5% of the vote, consistent with the latest polls and consistent with his forecast vote based on forecasts based on the economy.

2. As with previous Republican candidates, McCain did better among the rich than the poor:

But the pattern has changed among the highest-income categories:

3. The gap between young and old has increased–a lot:

But there was no massive turnout among young voters. According to the exit polls, 18% of the voters this time were under 30, as compared to 17% of voters in 2004. (By comparison, 22% of voting-age Americans are under 30.)

4. By ethnicity: Barack Obama won 96% of African Americans, 68% of Latinos, 64% of Asians, and 44% of whites. In 2004, Kerry won 89% of African Americans, 55% of Latinos, 56% of Asians, and 41% of whites. So Obama gained the most among ethnic minorities.

5. The red/blue map was not redrawn; it was more of a national partisan swing. See this state-by-state scatterplot of Obama vote in 2008 vs. Kerry vote in 2004:

The standard deviation of the state swings (excluding D.C. and the unusual case of Hawaii) was 3.3%. That is, after accounting for the national swing in Obama’s favor, most of the states were within 3% of where they were, compared to their relative positions in 2004.

By comparison, here’s the 2000/2004 graph:

The standard deviation of these state swings was 2.4%. This was even less variation–2004 was basically a replay of 2000–still, the relative state swings of 3.3% in 2008 were not large by historical standards.

Again, Obama didn’t redraw the map; he shifted the map over in his favor. (Or, to put it more precisely, the economy shifted the map over in the Democrats’ favor and Obama took advantage of this.)

Here’s the map showing where Obama and McCain did better or worse than expected based on 2004:

6. Finally, how did the pre-election polls do? Unsurprisingly, they pretty much nailed the national vote. And what about the relative positions of the states? The pre-election polls did well there too, at least using Nate Silver’s aggregations. Here’s the scatterplot:

Pretty damn good. The standard deviation of the discrepancies, again excluding D.C. and Hawaii, is 2.5%, which is a big improvement on the 3.3% using Kerry04 alone.

I see some systematic patterns: Obama underperformed where the polls had him way down, and he outperformed where the polls had him up. We should go back and look at these patterns from earlier elections and see if this is consistent. If so, it suggests a way to improve forecasts for next time.

P.S. Age graph fixed from first posting; thanks to Andy Guess for pointing out the error.

On the evening of November 8, 1988, I was working with my colleague Gary King in his Harvard office. Gary somehow had gotten his hands on a pair of tickets to Michael Dukakis’s victory party in Boston, and we were trying to decide whether to go. Dukakis was expected to lose, but . . . who could say, right? We had the TV on, and the first state to report, at 7pm, was Kentucky, which Bush had won by over 10 points. Gary informed me that the election was over: Kentucky, at the time, was near the political center of America, and there was no way that Dukakis would do much better nationally than he did in Kentucky. So we saved ourselves a subway ride and kept on working.

What about this year’s election night? Can you play along at home with the election and decide at 7pm what is happening? We will perform some calculations using vote margin (as we learned of Kentucky in 1988) and some using only the tally of states won or lost.

I did all the calculations using 10,000 simulations from Nate Silver’s latest election forecast.

At 7pm, we’ll hear from Virginia, Indiana, Georgia, South Carolina, and Kentucky (and also Vermont, which I’ll ignore because of its atypicality). Based on current poll aggregates, we expect the average vote margin in these 5 states to be 5.7 percentage points in favor of McCain.

The top row shows the uncertainty distributions for the popular and electoral vote (Obama wins are shaded in each case). The bottom row shows the corresponding distributions, conditional on the above five states going exactly as expected, with an average McCain victory margin of 5.7 points.

So, if things go as expected at 7pm, it’s all over. But what if things don’t go as expected? The simulations show that the average margin in the five 7pm-closing states could be anywhere from 12 points in favor of McCain to 1 point in favor of Obama. The next graph shows Obama’s expected popular vote margin, electoral vote, and probability of winning under a range of 7pm outcomes.

Finally, we did some calculations based on the scenario in which we don’t know the vote margins but we do know who wins each state. Here’s the story of the key 7pm-closing states:

And then, if McCain wins Virginia, it goes on from there. . . .

See here for the full paper.