Nate Silver Was Right (and it worries me)

I'm a number cruncher.  I'm a fan of Nate Silver and Sam Wang and all the other nerdgeeks who correctly called the presidential election.  I applaud their systematic approach to data, their methodological rigor, and their opportunity to put in place a bunch of pundits who basically make shit up and call it analysis.

That said, this worries me.

This worries me because of the reductive nature of these aggregators.  This worries me because public opinion is not best defined as what public opinion polls measure.  This worries me because, as, the Obama camp demonstrated, politics has truly drilled down to micro marketing.

I have lots of time to elaborate on these concerns, once I (yes, you guessed it) crunch some numbers. 

Let me throw one out there in a hurry, though, and that's the nature of public opinion.  I've written about this in the past (here and here, about Twitter, for example) and how the circular definition above fails to capture what is truly meant by the fluid nature of public opinion.  This gets PhDweeby.  I'll write more later, but before I sign off for lunch lemme say my main fear isn't about the accuracy of aggregators.  They demonstrate the predictive power of big data.  My fear is more a case of how the aggregators influence how we perceive public opinion.   

Our opinions are more than numbers.  And this comes from a guy who basically takes opinions and translates them into numbers for analysis. 

I leave you with my favorite definition of public opinion:

Public Opinion is no more than this,
what people think other people think

That's another post for another day, but the words were uttered by none other than Prince Lucifer in his self-titled play.  They say a lot about what public opinion really is -- not just a snapshot of an attitude crystallized into an opinion by a standardized questionnaire, but something more fascinating and fluid and in many ways, all about communication.

As I said, more later.

Aggregating the Aggregators

As we all know, Nate Silver at 538, Sam Wang at Princeton, and Drew Linzer at Votamatic have made quite the names for themselves this election cycle with their number-crunching predictions.  All three call today's election for Obama with following electoral votes (at the moment I write this):

Wang:  303
Silver: 313
Linzer: 332

So to be fair, as they all suck data from polls paid for and conducted by others, that I suck data off their work and aggregate their aggregation, thus inserting even more random error in a messy political process rife with false precision.  If we do some simple calculations on the back of a stained cocktail napkin, Obama wins with 316 electoral votes.

Where do they disagree?  Florida, mostly.  Silver has it leaning Obama, Wang (map to left, weighted by each state's respective electoral vote size) has it leaning Romney, and Linzer has it leaning, er, nowhere.  It's his only swing state bathed in white.  All give Obama such key states as Ohio and Pennsylvania.

Obviously we won't know if the nerds were right until late tonight or perhaps Wednesday.

There's an Election Today

I'm sure you've noticed there's an election today.  I voted early.  Strolled downtown on a warm, sunny morning, voted in five minutes, then had lunch somewhere other than the usual places near my building.

Today it's chilly, rainy.  Grey.  (notice I used the "e" and not the "a" because I think it looks cooler)

But I'm wearing my "I'm a Georgia Voter" sticker today so people think I suffered the long lines and the weather and the grey (with an "e") skies.

So, what's my prediction?  Hell, even college kids are writing prediction columns in the Red & Black and I can tell from reading them they don't really know what they're talking about.  Why should I provide more hot (though more informed) air?  If you want predictions, my favorites are the nerds and geeks:  538 or Princeton Election Consortium.  Nate Silver and Sam Wang are terrific number crunchers using somewhat different methods and models to come to the same conclusion -- Obama wins, probably with as many as 310 electoral votes.  There are a couple of other really good ones out there, but you can find them on your own.

IQ and Political Knowledge

If we're getting smarter, at least as measured by IQ scores, then why aren't we getting smarter, at least as measured by political knowledge scores?

The Flynn effect is the steady improvement in our IQ scores over time.  Why?  Lotsa theories out there and you can read the article yourself and choose your favorite.  But while this suggests we're getting smarter, at least at fooling IQ testers, a host of studies find that the public's political knowledge has remained relatively steady since the 1950s.

How can this be?  Well, in part we're talking about two very different kinds of tests.  Plus IQ is non-motivational.  Huh?  Simply put, people are either motivated or not motivated to keep up with public affairs, and that can play a huge role in how well they answer the traditional survey items used to measure political knowledge.

So, we're getting smarter, but not more knowledgeable.  Take comfort in that as best you can.

Oh, and for fun, you can check out IQ scores by state.  Yes, Mississippi is last.

Welcome to My (research) World

In breathless prose, an article in today's The New York Times points to research about trying to predict elections not by asking who people favor but instead asking them who is going to win (pdf of study here).

Oh jeez.

I'm happy to see someone doing this.  Why?  Because, dammit, I've written on this topic quite a few times.  Most recently here.  Also here, and here, and here, and here, and, oh hell, you get the idea.  And those are just my blog posts.  I'm also in a newspaper column on this topic and, dammit, I've published research on the topic -- including a big study this summer presented at a conference that looked at elections since 1952 and how well people predicted the winner.  Punch line:  people suck in close elections, are better when there's a bigger margin.

Oh jeez.  And does the paper cite me in any way?  Sigh.  Of course not.  Bitter much?  Ah well, what would ya expect from Microsoft and economics guys?  Basically sophisticated number crunching, and not a bit of theory.

Who Wins? A Coffee-Based Analysis

Who’s gonna win the election?

Sure, you can turn to the pundits and politicians, or you can turn to the mathematicians and statisticians.  Or like me you can seek out absolutely meaningless correlations to fill your time.

Yep, I’m going with the last one.

For those of you who bemoan the effete, cappuccino-sipping liberal types, here’s an explanation you’re going to love.  I grabbed data for how many Starbucks arelocated in each state and compared it to the state-by-state electoral predictions found on Nate Silver’s excellent 538 blog.

Below are the top 10 Starbucks states, per 10,000 population, followed by the candidate predicted by Silver to win that state.  Sorry about the lousy formatting.

D.C.                1.181   Obama
Washington      0.889   Obama
Nevada            0.799   Obama
Colorado          0.690   Obama
Oregon            0.667   Obama
California         0.556   Obama
Hawaii             0.463   Obama
Arizona            0.418    Romney
Alaska             0.337   Romney
Illinois             0.323   Obama

As you can see, Obama’s coffee cup runneth over.  The incumbent is predicted to win eight of the 10 top Starbucks states.  For Romney, you’ve got to drain your cup down to #8 (Arizona) before you find one of his states.  A decaffeinated campaign, perhaps?  Well yeah.  He is, after all, Mormon.

Want more?  Of the bottom 10 Starbucks states, seven are predicted for Romney.  Wow.  

Okay, let’s get a few issues out of the way.  D.C. isn’t a state, plus I’m relying heavily on Silver’s numbers.  Then again, he called 49 of 50 states correctly in 2008 and his more or less flow nicely with another great predictive site.

More mind-numbing numbers?  The Obama states average .34 stores per 10,000 people, while the Romney states manage a meager .17 per 10,000.  The national average is .26 per 10,000 (.30 if you weight the data, but let’s skip the math).

What’s this all mean?  Nothing much, but for my next trick I’m going to add Wal-Marts to the data and do some correlations and multiple regressions and really dazzle you with complete bullshit.  Stay tuned, because on top of everything else there’s gonna be maps better than the one below (which is a bit flakey sometimes, sorry).  Oh, click below on a state in the map and see how its Starbucks rank and predicted electoral outcome.  Darker colors means more Starbucks per 10,000 people.  You can grab and move it around some, too.  Have fun.  Suggestions welcome, the sillier the better.

What Teens Know ... About the 2012 Election

Yes, I know, the headline above reads like the setup to a punch line.  What do teens know?  Not a helluva lot.  Still, some enterprising journalists set out to discover what teens know about the 2012 election.

The most likely response?

“I support whatever my parents do because they understand more about politics than I do.”

Their "survey" was of students enrolled in four advanced English high school classes in the Council Rock School District of Pennsylvania (sad note, school is out due to damage from Sandy).  While it's hardly a generalizable sample, they asked students 13 questions that ran the "gamut of knowledge" from basic to the advanced.

Students did well on throwaway questions on identifying the president, the vice president, the Republican challenger, and (less well) his VP running mate.  Only half knew which party controlled the House, which is about right -- even if you guess, there are only two parties, so half is unsurprising.

My favorite -- they presented students with a picture of John Boehner, the Speaker of the House.  Only 14 percent gave his name, party and title.  Only 15 percent could name one of their state's two U.S. senators.

Okay, are we picking on the brats?  A little.  To be honest, they didn't do so poorly compared to the levels of knowledge demonstrated by the general U.S. public, especially given most if not all of the students aren't even voters yet, and so they have strong motivation to follow politics at a high level. 

In other words, the kids did okay.