Future of the polls – lessons from the 2012 race

>>> with us now creator of the blog are for "the new york times," nate silver . nate is the author of the book

"the signal and the noise: why so many predictions fail but some don't."

>> well, well, well.

>> and it's very good to have you on the show this morning.

>> it's great to have you here.

>> thank you, joe.

>> very good to have you here. by the way, we didn't bet because we love humanity too much, and we made a contribution, mika.

>> that's nice.

>> in nate's name.

>> when i talk to friends i have down in new jersey, people, i think, don't realize how devastating the storm was.

>> the scope. i still think, looking at pictures last night on facebook , my husband facebooks with channel 7 , and i don't facebook , but he was showing me the pictures he was posting on facebook . it is unbelievable still the damage and destruction and the people who were left out in the cold, the people who are hungry from the effects.

>> americares does great work. we have a lot to talk about.

>> sure.

>> the first thing i want to do, before your apology to me --

>> right. it's okay.

>> let's sell books.

>> let's not even bother. don't worry about it.

>> let's sell books. what's it about?

>> we live in a very information rich world. you think of all the polls we get now and we need better strategies how we sort through the information to find meaning or truth or make good predictions. the book is about that. we talk to people in every field from finance to earthquake prediction about who is kind of taking and making sense of it all.

>> so let's talk about the last election. you know, your numbers turned out to be right because the state polls if you waited turned out to be right. i want to ask you about it and we talk about this in our speeches when we go to college campuses or wherever and it seems to me what happened on the right this year was, there's a certain pollster who would come out with a poll.

>> sure.

>> and then there was a certain website that would put it at the top of the poll. and then there was a certain network that would run that -- so what we -- what we kept asking chuck todd , chuck, what's going on? you sorted through the polls for the 2010 cycle. you talk about which ones worked, which ones didn't. let's talk about the best polls, first of all, and secondly, why some of the automated polls turned in it great results.

>> well, look, most of the major network polls are good. the nbc news/" wall street journal " poll of course. i think that high quality polling really differentiates itself because if you take on a poll, you miss people who have cell phones . you have a third of the population now. and they're mostly younger urban demographic, democrats, so you will under sample democrats if you don't call people who have cell phones and lo and behold, those polls l had a republican bias this year. not because they are evil partisan. if you miss a big chunk of the population that's democratic leaning, you're going to have problems.

>> there are assumptions you have to make. the thing i kept hearing from republican pollsters, from the romney people, i'm sure you heard it all the time, too, you guys are going plus eight, plus nine --

>> but how do you make those assumptions because gallup just absolutely humiliated themselves because i think they went plus three or plus four instead of plus seven plus eight.

>> here is the issue. the best pollsters like nbc, cbs and "the new york times," let the sample tell you by themselves. they're not making any assumptions how many democrats or republicans that didn't put their finger on the scale. just said, well, we seem to be hearing from more hispanics than we did four years ago. a lot of people who are democratic leaning whereas republicans might identify as independents now. instead of making assumptions, it's the best practice when you're doing scientific survey and that worked again this year.

>> so tell us what you do and tell us what you did this past year that made you right and a lot of other people -- and for the record, i said that obama was going to win. i think it's like a week and a half out you are putting it at like 74%.

>> sure.

>> and the politician says you never know what will happen just like with baseball. the numbers may look right but certain teams, ballplayers may eat fried chicken and drink beer in the locker room. anything can happen. but talk about what you do.

>> what we're really doing is taking the polls. you take an average. that part is simple. translate it into probability so it's the same in a football game . if a team has a field goal lead with five minutes left to play, how often will they win? we're doing it the same way. obama was up by three points in ohio , virginia, iowa, other swing states . and that's usually pretty solid. a chance like a big mistake but it was a 10% chance and not a 50% chance. of course things did solidify for him towards the end. towards the end where you were seeing consistently in almost every poll --

>> he bumped back up.

>> it looked like people said toward the income, toward the incumbent with sandy, you did see him have more of a cushion toward the end of the campaign.

>> you tried to put an impact that sandy had. did you put in one, one and a half points?

>> a point, a point and a half. now that we see obama has won by more than three points in the national popular vote and more in some of the states, i think you can say he won the election because of the storm. it made things more comfortable certainly.

>> mark?

>> so anyone who writes or talks about politics gets criticized these days in one form or another. one thing about politics, we like issues of substance and the importance of the election but we all like who is going to win and look at a lot of polls. there's one strand of criticism that says too much horse race coverage in politics. too much focus on who will win rather than the underlying issues. what do you say to that?

>> i'm more interested in diagnosing 2016 than benghazi. that's where my bread is buttered.

>> thank you for being honest on that.

>> yeah. but if you're going to do a horse race , then do it the right way. one reason i like the horse race , it can be more data driven . even now we had a post on 2014 , for example, how the history will work against democrats. the house in 2014 because the president's party doesn't do well in midterms. so making all the coverage more driven. we saw, i think, in baseball, where things have come full circle now where the scouts -- you still have scouts, but they know the stats and that makes them better scouts. they know where to add value on top of that. if we can make the baseline for horse race coverage 20%, 50% more data driven then we'll have improvement all around.

>> john?

>> you follow politics in a pretty thorough way. one of the things to go from what mark was talking about that i find sort of curious is people who often attack horse race journalism are also huge fans of your work. how is it that you can reconcile that notion? i mean, to my mind what you do is the purest distillation. it seems people love nate silver at the same time.

>> it's like one of these terms that can be used in a pejorative way. everything that's not what i do. horse race journalism . it's bad.

>> that's what i said, by the way. what i said. i'm sorry, go ahead.

>> i do think you need to get to the essence of the question which is who is going to win and there's truth in advertising. we have probablies up on the side. who will win the house and the presidency? down to three numbers, right? in some ways it's more efficient, i think. and of course you explain how up get there and talking great de detail about why we think what we do and why the models we have are coming up with the numbers that they do. but in some ways you stop beating around the bush so much and say here is the distilled essence, right. 72% democrats and people in the senate or whatever or 89% odds of winning ohio , et cetera . it makes things easier in a way when people are trying to figure out how to get information.

>> and in your book here you have what's called the prediction paradox. the more humility we have about our ability to make predictions the more we can accept we don't know everything, the more successful we can be in planning the future.

>> well, and again, i used to play a lot of poker. and as a sports fan, you know as a sports fan you're going to be humbled sooner or later .

>> it happens.

>> sometimes your team will be upset by number 15 in the ncaa tournament . you get a sense for what the probabilities are and what the odds are and you know you have to make the best bets you can with the information that you have knowing that you're going to be wrong sometimes. a senate race in north dakota we had the democrat with only an 8% chance of winning and she won. a low-profile race. that was the same chance we had romney at a 10% chance.

>> and, again, it's so fluid. it's like whether it's poker or whether it's baseball or whether it's politics. it's so fluid. if romney had found his voice two weeks out and started becoming the candidate of all candidates, the state polls moved, your probablies mov probable abilit ys happened. you go into the final week and the president is sitting at plus one or two in a lot of swing states . plus four in ohio . this was one of the more static presidential races we've had.

>> it really was. people forget about races like '92. perot going in and out. polls all over the place. in 49 of the 50 states we had the right candidate winning in june. in florida for romney back in june based on the polls.

>> isn't that crazy?

>> that's how stable it was. we know about 45 states are going to vote so credit for predicting vermont or utah of course, right? but it was basically a two-point campaign. if you had gotten really great jobs news, if you had more bad jobs.

>> were you surprised by how off the romney people were? mitt romney on election night thought he was going to win based on what his pollsters told him. his wife thought even late into election night they were going to win. are you surprised?

>> look, sometimes campaigns put on a certain spin for the press, of course. when i talked to the romney campaign, i thought they had a realistic view of where they stood. maybe they talked to me in a way that's more direct because i'm not going to go out and do a lot of reporting on it, right? i think the pollster there, those guys are pretty smart guys who took a b-minus, c-plus candidate and ran a tough campaign. obama got some of the jobs numbers he wanted late in the cycle. i think the campaign was adequate. was fine. it's tough to beat an incumbent as we found out this year and people very often forget even very vulnerable incumbents, harry reid won re-election.

>> the book is "the signal