Comeback Time for Social Sciences
The limits of quantitative data are exposed while our biases remain buried deep.
The most disappointing aspect of this election might not be what our President-elect has said about The Underrepresented.
It’s what millions of voters secretly believe but aren’t willing, or comfortable, to admit: that they have some level of tolerance, if not full embrace, of discriminatory practices with gender, race, and citizenry. For the record, this applies to all education levels (e.g. 45% of college-educated white women voted Trump, while 62% of non-college-educated white women did the same).
Why were the election projections so, so far off? It could be that the scientific underpinnings and methodology of polling is afoul. Or, it could also be that people simply lied in their responses to protect themselves — from their friends/family’s “politically correct” ire, and from their own id.
But wait. Polls are anonymous. Why would someone lie on an anonymous poll?
Oh, That Nutella?
Years ago I worked at IDEO, a firm known for human-centered design rooted in ethnographic research. One of Margaret Mead’s famous points — “What people say, what people do, and what they say they do are entirely different things” — came to life with surprising frequency.
When designing a snack product for a consumer food client, IDEO researchers asked a number of participants to keep daily journals of their food consumption. They followed up with a select few, visiting research participants at their home and work. While one subject walked the team through her work space — family pictures, ergonomic set-up, etc — they saw a jar of Nutella in one of her desk drawers. She hadn’t made any mention of it in her journal.
“Oh that? [Laughing] Yeah, I sneak a spoonful every now and again,” she said.
So why didn’t she just put that highly relevant fact in her journal? Perhaps she didn’t want to admit it to herself, and writing it down would be definitive step in that direction? Maybe it just helped her, personally, to hide the fact that she behaves in a way she’s not proud of, or in theoretical agreement with. Possibly just a case of cognitive dissonance.
The tale of our election mis-forecasting just might be that: a cognitive dissonance so deep and sophisticated that raw numbers simply don’t tell the whole story, and worse, pervert the story itself.
Complex issues — like grappling with the historic, tortured choice some voters faced this November — might need equally sophisticated research techniques next time around. We might need our data scientists and pollsters to partner up with anthropologists and ethnographers to provide a contextual view for the data. It might also be helpful for quantitative folks to get a feel for the human numbers they’re crunching. [Update: In doing so they might have also realized that an overwhelming number of older, first-time voters were going to turn out — and could have changed their models accordingly.]
While pollsters aren’t designers aiming to create things, they might learn something from hybrid research, which combines the quantitative and the qualitative. Johannes Seemann sums it up below.
I’ve been reading reflections and solutions to some of the reasons we’re here today — Anil Dash, Dave Pell, danah boyd, Jennifer Pahlka, Jeff Jarvis — and while I don’t agree with all of them, I applaud the effort to move forward constructively. So I’m chipping in my own with this piece for pollsters.
However, and this is a big one, it’s hard to deny the horrific, massive undercurrent of racism, misogyny, and xenophobia that is being born out in the data. The truly terrifying part is that many Trump-voters don’t know that they’re likely racist, sexist, and afraid of the other. And they can’t take the steps to move past it if they don’t confront it.
Unlike many of the pundits I’ve been reading the past 48 hours, I actually do have friends who supported Trump for a wide range of reasons. All of them disavowed his “diarrhea of the mouth” and classified his terrible words/actions with the following: “I’m willing to overlook that in order to [fill in the blank: drain the swamp, boost the economy, secure a conservative SCOTUS, etc.].”
I just don’t know how you can overlook those statements without making an unknown, deep-down assessment and complicit acceptance of them. [If Trump supporters had outright rejected and denounced his musings, perhaps I wouldn’t have written this post in the first place. But many of us are still waiting for Trump supporters to do this one simple thing h/t jessicashortall.]
Maybe next time when pollsters include ethnographers in their research, we’ll get a better picture of what’s happening — and start to unearth, and meaningfully grapple with, the demons inside.
