Doug Hibbs’s model:

(As I wrote a few months ago, “the incumbent party sometimes loses but they never have gotten really slaughtered. In periods of low economic growth, the incumbent party can lose, but a 53-47 margin would be typical; you wouldn’t expect the challenger to get much more than that.”)

The relative positions of the states don’t actually change much from election to election:

You can do slightly better by using polls. As Matthew Yglesias puts it, “the large number of public polls on something like a presidential election makes the outcomes quite easy to forecast based on crude measures. What’s more, even absent polling, Presidential election outcomes seem to be pretty predictable based on nothing more than macroeconomic variables.”

Actually, even the February polls turn out to be pretty good–when combined with previous election results–to pin down the relative positions of the states.

Bayesian combination of state polls and election forecasts

Here’s the revised version of my article with Kari Lock in which we forecast the election using Hibbs for the national popular vote, and a weighted average of last election (corrected for incumbency) and the February polls to get the relative positions of the states.

Lots fo fun stuff there, including this prediction (based on February Clinton-McCain and Obama-McCain polls) of which states Clinton or Obama were expected to win in November:

Bill Clinton in 1992 and 1996 did well among young voters–like Barack Obama, he was a young Democrat facing older Republican opponents–but not so well as Obama in 2008.

As in many political settings, the largest gains in the graph come from incorporating additional data–in this case, the comparison of 2008 with earlier years, the comparison on young voters with those of other ages, and the comparison of the three other age groups with each other (with the lack of variation in this last comparison being a motivation to focus on trends among young voters in particular).

Difference in McCain vote share, comparing people in each state with family incomes over and under $50,000 (thus, states that are high on this graph are those where richer people were much more Republican than poorer people):

The same graph, but for whites only (following Larry Bartels’s suggestion):

As before, the states are colored as red or blue where McCain or Obama won by more than 10% of the two-party vote, and purple for the states in between.

Lots of interesting patterns here.

I had a question for you about the youth vote. What are its ethnic and red/blue composition? The reason I ask is that I was trying to integrate the apparently growing Democratic dominance in this segment with various other beliefs I have seen expressed, e.g

a) that red states have larger fertility (affordable family formation or whatever)

b) that families have an impact on the political beliefs of children (more than educators, as educators insist – at least at the college level, I haven’t really seen a discussion of school teachers) which would then provide a mechanism for (a) to affect voting share to the right of the spectrum

c) that the minorities form a growing share of the young which would tilt the playing field to the left.

My reply:

1. I don’t yet have raw survey data. The exit polls on the web do break down the vote by age and race. Among blacks, Obama won about the same among all age groups. Among Hispanics, Obama did 8% better among the young than the old, and among whites, Obama did 14% better among the young than the old.

But . . . if you believe the exit polls (which I don’t, completely), there was an interaction between age and race: many more of the young voters were ethnic minorities. Among blacks and Hispanics, there were three times as many under-30′s as over-65′s. (By comparison, among whites, there were more old voters than young voters.)

So the age effect partly arose from lots of young ethnic minorities coming out to vote.

2. People do tend to vote like their parents–children of Republicans are, on average, more likely to vote Republican–but cohort effects go on top of this. The recent economy and George W. Bush’s approval ratings aren’t likely to make the Republican Party popular with young people–especially those who are ethnic minorities. Any differences in birth rates between states are small compared to these big political swings, which are not just about Obama; see this graph from 2006:

I [Ben] had this map [see below] on my door for the last week. Based on exactly the same calculation using constant 95% black support and census-proportional representation. The white counties are the ones whose census names didn’t match properly with the names used in the library(maps) package in R, I was too lazy to fix them.

Cool. I’d only suggest using light gray rather than heavy black lines between counties; the map as it is overemphasizes the county borders, I think. But I respect his laziness; there’s always time later to fix the details.

Ben continues:

[Below are] the state-by-state county share plots for the lower 49, Obama vote share as a function of black population share. V.O. Key’s observation that whites who live near blacks in southern states are less positively inclined towards them is *still* visible in several states.

The circle areas are proportional to county voter turnout. (The biggest circle is L.A. county in California, and so forth.)

Ben also had this comment about his map:

It reminded me of something Bob Putnam would say every time someone presented an empirical talk in our Center for the Study of Democratic Politics series during the year he was a fellow here at Princeton: “You should include miles to the Canadian border as a variable in your regression, it is the most important proxy for political culture in America!” At least in the eastern half of the country, he has a point.

Except for New Hampshire and Vermont, I think.

P.S. Further discussion here of graphing possibilities.

I [Teigen] examined contested House races from 2000-2006 to see if candidates’ previous military service helped them garner a higher share of the vote. . . . the effect of being a Democratic veteran or a Republican veteran on vote share, controlling for presidential vote share in the district, gender, incumbency, campaign spending, and other relevant factors. . . .

In general, veteran status has small effects that are not statistically distinguishable from 0. Democratic vets did better than their nonveteran peers in 2002, but did no better in 2006. That election was the year that Joe Sestak, Tammy Duckworth, and others constituted the “Fighting Dems,” a year when you would expect Democratic vets to do well, but instead Republican veterans were helped by a martial past. . . . In 2006 Democratic veterans actually did a little bit worse than Democrats without a service record. This result may have occurred because Democrats were overzealous in their attempts to attract veterans as candidates, leading them to select veterans over higher quality challengers (14% of Democratic challengers were vets in 2002, compared to 28% in 2006). Republican vets running that year performed a little better than nonveteran Republican candidates, as they had been doing in the previous three elections, but the advantage just slipped above statistical significance. Overall, the effect of veteran status is very small.

It would be interesting to see data from other years and comparisons to other occupations (community organizers? small-town mayors? POW’s?).

Teigen continues:

Nevertheless, this does not seem to deter veterans from running for office and promoting their service during their campaign. Despite the allegedly “tinny ring” of military values, some candidates make their military service the key element of their campaign narrative (e.g., Craig Williams). To me, this behavior and the continued emergence of veteran candidacies says that our candidate selection mechanisms still value military service even if the general election yield is inconstant and small.

I actually disagree with this bit, or at least I’m not yet convinced by the evidence. Every candidate has some strength. If you’re a veteran, you can make a big deal about that. If you’re a trial lawyer and won cases for sick kids, you can advertise that. If you’ve been a community organizer, ditto. The lack of any special advantage for veterans does not at all imply that it’s silly for veterans to promote their service during their campaigns.

P.S.

Some technical comments: I don’t think it’s a good idea to control for campaign spending. That’s an intermediate outcome (see, for example, chapter 9).

On the plus side, I agree that it makes sense to analyze House races (larger N than just analyzing presidential contests), and I’m glad to see vote share rather than win/loss used as an outcome.

Percent black acts as a floor on Obama’s vote share; beyond that, it predicts his vote better in some regions than others.

But really there are two things going on. First, Obama’s getting nearly all the black vote; second, depending on the region, whites are voting differently in places with more or fewer African Americans.

Then I had a thought. Obama got 96% of the black vote. If he got 96% in every county–which can’t be far from the truth–then we can use simple algebra to figure out his share of the non-black vote in every county. If B is the proportion black in the county and X is the (unknown) Obama vote share among non-blacks, then, for each county,

obama.vote = 0.96*B + X*(1-B)

And so

X = (obama.vote – .96*B) / (1 – B)

This is only an approximation–for one thing, it assumes turnout rates are the same among blacks and others–but it can’t be too far off, I think. And it leads to the following graph:

(Lowess lines are shown in blue.) None of this is a huge surprise: outside the south, places with more African Americans tend to be liberal urban areas where people of other ethnicities also vote for Democrats; in the south, many African Americans live in counties where the whites are very conservative.

Notes:
1. These graphs are non-blacks, not whites. Some of the variation has to be explainable by the presence of other minority groups.
2. For a few of the southern counties, our estimates of X are negative; that just means that Obama got less than 96% of the black vote there, or there was differential turnout, or some combination of these.

P.S. More graphs here (from Ben Lauderdale).

Some of the pros say this too; for example, Charlie Cook writes, “given the strength of the top of the ticket nationally, one might have thought that the victory would have been more vertically integrated. . . . what happened down-ballot was not proportional to what happened at the top.”

And Mickey Kaus attributes this to moderate ticket-splitters who, expecting that Obama would win, decided to support Republicans in Congress: “swing voters compensated for the bold, hopeful risk they took on Obama (including for overcoming any race prejudice) by gravitating back toward Republicans in their local Senate and House races.”

The only trouble with this theory is that it’s not supported by the data. Obama won 53% of the two-party vote, congressional Democrats averaged 56%. The average swing of 5.7% from Democratic congressional candidates in 2004 to Dems in 2008 was actually greater than the popular vote swing of 4.5% from Kerry to Obama.

Let’s look at what happened state by state. Here I’m plotting the swing in average district vote in each state, comparing the congressional elections of 2004 to those of 2008, ordering the states by Kerry’s share in 2004:

The horizontal blue line shows the average swing of 5.7%. The Democrats gained in nearly every state, with, unsurprisingly, some big swings in some of the small states that have only one or two congressional districts. Now let’s compare this to the state-by-state swing in the presidential vote:

Obama beat Kerry nearly everywhere, fairly uniformly with only a few exceptions–we knew that–but my point here is that Obama’s swings weren’t quite as large, on average, as the state congressional delegations’.

If you want, you can look at both swings at once:

In the states in the upper left of this graph, the Democrats improved more in the congressional than in the presidential vote; the states in the lower right are those where the Obama-Kerry swing was greater than the Democrats’ swing in House races.

There are a lot more states in the upper left than in the lower right. Each state has its own story–for example, I wouldn’t attribute Don Young’s squeaker in Alaska to Barack Obama’s coattails–but given the graphs above, I think it’s hard to make the case that, overall, the voters were saying No to the Democrats in Congress. On the contrary, congressional Democrats averaged 56% of the vote–their best showing since 1976 (and far more than the Republicans’ 52% in 1994).

Here’s the story in a map:

For some historical perspective, here are the Democrats’ two-party vote share in presidential elections and average two-party vote in congressional elections since 1946:

Presidential voting has been much more volatile than congressional voting (incumbency and all that). This makes the Democrats’ 5.7-point gain over two elections even more impressive.

Summary

I think Charlie Cook was closer to the mark when he wrote, “The political environment and momentum that Democrats seemed to have in recent months may have led to an unrealistic set of expectations. In this, perhaps we pundits share some blame.” I don’t think it makes a lot of sense to consider Obama’s 53% “enormously impressive” and congressional Democrats’ 56% a disappointment.

The data demolish the idea that voters in 2008 were pulling the lever for Barack but not for the Dems overall (not for “Nancy Pelosi,” if you will).

Notes

1. I thank John Kastellec and Jared Lander for gathering the data and sharing their thoughts.

2. I’m counting uncontested House candidates at 75% of the vote (see our earlier article for discussion of this and similar technical issues).

3. We use average district vote rather than total vote because congressional vote totals vary a lot, and we’re trying to assess national public opinion (as judged, for example, in Kaus’s quote above).

4. The Democrats won resoundingly; this means that the voters preferred them to the alternative; it does not necessarily mean the voters want the specific policies proposed by the Democrats. Recall the Democrats’ surprising lack of popular success after 1976 and the Republicans’ struggles after their 1994 sweep. 5. I’m talking about public opinion here, not campaign strategy. I’m sure that Democratic leaders were disappointed in their party’s performance in key congressional races, especially given their immense financial resources this year. At the level of public opinion, though, the Democrats in Congress outperformed Obama overall and in 38 states–and their swing beat Obama’s overall and in 32 states–so I think you’d be hard pressed to argue that the voters were balancing toward the Republicans in congressional voting. This is not to say that the voters have given the Democrats a blank check, but it really was a Democratic swing, not an Obama swing.

P.S. More graphs here.

P.P.S. Kaus replies (via blog):

I don’t understand Andrew Gelman and Matt Yglesias’ point. You don’t win the House of Representatives when you rack up a large percentage of the national “two party “Congressional vote, or when you win a large “average swing” vote on a “state-by-state” basis. You win when you win lots of actual House seats. That’s what can pass or defeat legislation. And measured by actual House seats the Democratic gains (of about 22) were a little less than expected. There is a reason for this.

My response:

1. As noted here, I think the appropriate comparison is 2004 to 2008. Obama did 4.5 percentage points better than Kerry; congressional Democrats averaged 5.7 percentage points more of the vote than their counterparts in 2004. And the Democrats gained many more than 22 House seats since 2004.

2. See my point 5 above. I’m willing to believe that the Democrats’ campaign strategy had problems, or that they underperformed in marginal districts. But, to return to Kaus’s original point about ticket splitters: “maybe there was a determined effort to apply checks and balances. By deciding to elect Obama president, more than a few voters may have opted to keep the Republican incumbent in place, just to prevent Democrats from getting carried away.” I don’t see it. If you want to talk about motivations of voters, I think it makes the most sense to look at vote shares, not just winners.

3. I don’t understand why Kaus puts “two party” and “state-by-state” in quotes. I mean, I guess I do understand, since he’s quoting me (which I appreciate), but I feel like he’s trying to say there’s something fishy about these ideas. But there’s not. “Two party” vote share just means that we exclude third parties and focus on the competition between the Democrats and Republicans. (That’s why, for example, I don’t think it makes sense to compare Obama’s 52% of the total vote to Reagan’s 50%-ish of the total vote in 1980. Reagan competed in a three-candidate race and he did much better than his main opponent, Carter.) I did my “state-by-state” analysis in order to compare Obama’s swing to congressional swings in different places.

To summarize: if the question is campaign strategy–did the Democrats do all they could’ve, or did the Republicans play a poor hand suprisingly well–then, yes, by all means, compare the election outcome to Stu Rothenberg’s and Charlie Cook’s pre-election forecasts. But if you are interested in public opinion–for example, were the voters trying to balance Obama with a more Republican congress–then I think vote swings are more informative.

I think Kaus and I could probably agree that there are two separate questions: (1) Did the voting public favor congressional Democrats (as compared to how they voted for Obama), and (2) Did the Democrats do worse in 2008 than they should have, given their lead in public opinion? I think the answer to (1) is pretty clear: the voters swung toward the Democrats in congress as well as (actually slightly more than) in presidential voting. I have no idea about (2), and I’d defer to Charlie Cook and others on this question. I’m less interested in question (2) but I agree with Kaus that such questions are important, as they affect the size of the Democrats’ majority in both houses.

P.P.P.S. Election outcome compared to anticipated seats-votes curve here.

Breaking this up by region:

See here for further discussion and more graphs.

Here’s Ronald Reagan’s share of the two-party vote in 1980 and 1984, by state:

And here’s the swing from 1976 to 1980:

The average swing in each of these years was about the same as from 2004 to 2008, but, comparing the states, it was a lot less uniform.

1952/1956 here.

Sure, young people voted heavily for Mr. Obama, but they voted heavily for John Kerry . . .

The exit polls:

Yes, maybe the exit polls were wrong here, but considering Penn’s article was otherwise peppered with exit poll statistics, so I don’t think that’s the issue.

So what happened?

My guess is that Penn had a storyline he wanted to focus on–the trends among minorities and upper-income voters–and so he wanted to dismiss any alternative explanations. But I think this is a misconception–the idea that a story has to be simple. Penn’s big story can be important but there’s no need for him to try to artificially strengthen it by dismissing the big changes in the under-30 vote. Lots of things are going on at once here.

P.S. Somebody pointed out to me that the only demographic group that Kerry won in 2004 was 18-29. I don’t think this gets Penn off the hook, though. Obama’s victory among the under-30s was so much bigger than Kerry’s, that I thought it was highly misleading for Penn to write, “Sure, young people voted heavily for Mr. Obama, but they voted heavily for John Kerry . . .” His first “heavily” is over a 30-point gap; his second “heavily” is a 10-point gap. He’s using English words to equate two numbers that are much much difference. I’m not saying Penn did this on purpose, but I’d still classify it as a mistake.

The two raw data scatterplots show the average per capita income of states in the election year (in 2006 dollars) on the horizontal axis, and the Republican share of the vote on the vertical axis. A best-fit line is drawn in each year. The income-vote slope decreases a small amount in 2008. Overall, though, rich states are still blue, and poor states are still red.

So how can rich states vote Democratic, while rich individuals vote Republican? And is this pattern new? Buy our book to find out!

Next step: multilevel models of individuals nested in counties and states. That’s going to have to wait until we get the exit poll data.

Up through the middle class, there was little change:

But at the high end there was a flattening out, even a turn toward the Democrats, that we didn’t see before:

Is this a real change, perhaps a move to a new era in which, at a national level, the upper middle class and rich are divided evenly between the two parties?

Or maybe it’s coming from the economic crisis, which in financial terms is hitting higher-income voters harder. (After all, who has money in the stock market?)

Or maybe it has something to do with the candidates themselves. What would the breakdown of voting by income looked like had it been Clinton vs. Romney? (There might be some poll numbers on this from the primary election season, but I don’t know that I’d trust them much.) Maybe it would’ve looked more like 2004.

P.S. regarding the first line above: Boris and I didn’t have tickets. We just observed the crowd from a distance.

Given the virtual certainty that there would be a recount in a decisive state where the election was very close (not to mention decided by a single vote), and given the virtual certainly that the recount would not yield a second identical result, wouldn’t the true probability that one’s vote would make a difference in both the original count and the recount be many times larger than your estimates? I’m not sure how to calculate that higher probability — but I suspect it would range between 2 X and X-squared where X is the Gelman probability that your vote would be decisive.

My reply: No, that’s not right. See footnote 6 of the article, which points you to p.674 of my BJPS paper with Katz and Bafumi which addresses this issue. The key is that, at some point, the election is determined, and each vote slightly changes the probability of McCain or Obama is winning. Before the election, the exact vote margin is uncertain, and the probability that your vote is decisive must be calculated by averaging over all possible vote margins that might occur. The result is that our calculation works out.

- Why the polls don’t matter while the elections are ‘predictable’? Is there too much confidence in a supposely ‘Obama landslide’?

Based on the economy, Obama was predicted to get something like 53% of the two-party vote.  Due to the U.S. electoral system, if you win 53%, you will win many many states and so the Electoral College will look more like a landslide.

- Do you give any credit to this Bradley Effect?

My guess is that this pattern is less of an issue in a highly contested national election.

- Your book ends with the myth that the rich elite votes Democrat and the poor vote Republican, but does it mean that American politics are in a way a battle of elites (those in the poorest and those in the wealthiest States)?

Yes, U.S. politics is in many ways a battle of elites:  the liberal elites in the Northeast, West Coast, and Upper Midwest, versus the conservative elites in the South and the rest of the country.

- If the image of the poor, religious, gun-loving Republican is a myth, why Republicans keep using the ‘real America’ theme and talk about values of small towns in the middle of the poorest part of the country? Is more a battle of rural vs. urban?

I am not sure here, but I suspect that rural images are popular even with many Americans who live in cities and suburbs.

- And what exactly makes the upper middle class in the wealthier States vote Democrats while their peers vote Republican in other States?

The upper middle class in wealthier states are more likely to be socially liberal (even if they are economically conservative).  In poorer states, the upper middle class tends to be both economically and socially conservative.

- Why the economic slowdown benefits Democrats? Do the richest not share the view that Democrats can handle better economic issues, as the polls generally show, or do they vote against their economic interests? (sort of the opposite that What’s the Matter with Kansas said)

A bad economy makes the current administration less popular among all groups.

- Are race or inmigration the main diving lines? And if so isn’t gonna affect disproportionatelly the Obama result?

Race is a big dividing line, but the line is already there.  At this point, people of all races are reacting to the positions of the candidates and new information.

- I was intrigued by your argument that religion matters nearly as much in European election than in American ones. How could it be when here religion is out of the political campaigns and discourse and there is no question whatsoever about the faith of candidates?

I’m not sure why it’s happened, but the two parties in the U.S. differ much more on social issues than they used to.

- I’d really appreciate your comments. Actually, I liked What’s the Matter with Kansas, but I always felt there was something wrong with the picture, as in Europe the richest tend to be more conservative (althought I also attributed to the fact that, in Europe, Democrats will be probably center-right and Republicans far on the right…). Anyway, thanks to you and your book for the clarification!

I hope this is helpful to you and your readers.  I liked “What’s the Matter with Kansas” also.  That book doesn’t claim to be a statistical analysis.  What it has is analysis of the rhetoric of the Republican party and lots of discussion of the battles within different Republican factions in Kansas.  It’s important stuff.  I think that quantitative studies such as ours and qualitative studies such as Frank’s are both important.