Dealing with data

In two recent columns in the New York Times, David Brooks explores “data-ism,” which he describes as the “rising philosophy of the day.” In the February 4 column, he writes: “We now have the ability to gather huge amounts of data. This ability seems to carry with it certain cultural assumptions – that everything that can be measured should be measured; that data is a transparent and reliable lens that allows us to filter out emotionalism and ideology; that data will help us do remarkable things – like foretell the future.” Brooks has an open mind about the benefits and limitations of “data-ism,” and while acknowledging that “the data revolution is giving us wonderful ways to understand the present and the past,” he concludes that the jury is still out on whether the data revolution will “transform our ability to predict and make decisions about the future.”

That “ability to predict and make decisions about the future” is at the heart of the college admissions process, revealed in questions such as: how should I choose which colleges to apply to, what college should I attend, what factors influence a student’s choice to enroll at a particular institution, how many admitted applicants will enroll this year. The field seems to be in a state of flux now, with all parties – prospects and applicants, parents, school counselors, admissions staffs, and the high school and college personnel these latter two groups report to – struggling to come to terms with the flood of data available. Beyond the question of which data to collect lie other, equally important, questions. For example, how does one evaluate the reliability and validity of the data selected? Does our capacity to collect more data mean that we can make better predictions about outcomes? Is it always better to collect more information, or does there come a point at which more information hinders our ability to function effectively?

Brooks’s second column, “What Data Can’t Do,” offers responses to a few of those questions.  He rightly points out that “as we acquire more data, we have the ability to find many, many more statistically significant correlations. Most of these correlations are spurious and deceive us when we’re trying to understand a situation. Falsity grows exponentially the more data we collect.” (Less elegantly, one might say: “Mo’ data, mo’ problems.” ) Brooks also points out that “data obscures values…it’s always structured according to one’s predispositions…” This is an excellent point which should incline us to look closely at the sources of the data we use.

I like the way in which Brooks calls out central concerns in our use of Big Data. As I look at these concerns in the context of college admissions, though, I think it’s important to take a step beyond the data itself. Yes, one should question the underlying “predispositions” or agendas of the individuals or agencies supplying the data we and our students use. However, it is equally important to consider the impact of the filters each of us uses to evaluate that data. All the data in the world won’t necessarily help one make a good decision (or prediction about the future) if one focuses only on those data points which validate a pre-existing conclusion.

Given the masses of information out there, it’s very easy to pick and choose among data available to support one’s desired choice or outcome. Consider the situation Karl Rove found himself in on election night 2012, when he disputed the Fox Network’s pronouncement that Obama would carry Ohio, and thus win the presidency. In what will surely be a classic TV moment, Fox anchorperson Megyn Kelly called him on his selective use of data, asking: “Is this just math you do as a Republican to make yourself feel better?”

It’s a tricky thing: as we collect and use data (however imperfectly) to make decisions, we believe ourselves to be acting rationally. However, as behavioral economist Dan Ariely points out, the way in which we tend to make decisions is not rational at all, but in fact, “predictably irrational.” Ariely’s work demonstrates the ways in which the sub-conscious filters we apply, and the circumstances that surround us when we’re looking at the data, can skew our decisions.

photo 2-001

Ariely has studied the way in which people make decisions about a variety of topics, and his research gives us a useful framework within which to view the ways in which students make choices and decisions about which colleges to apply to, and which to attend.

For example, Ariely highlights the impact of “relativity” on decision making, pointing out that our minds are wired so that “we’re always looking at the things around is in relation to others.” (p. 7) To paraphrase, we need “comparables” in order to make judgments about objects or opportunities. (You can’t, for example, definitively say how good a particular brand of peanut butter is unless you have sampled more than one brand. ) So far, so good.

But here’s the rub: not only do “we look at decisions in a relative way,” but, as Ariely’s work demonstrates, we tend to compare our options “locally.” That is, we make comparisons to other objects or opportunities that are within our immediate sphere of attention. This accounts for the “herding behavior” college counselors so often see as high school students draw up their lists of colleges to apply to. Successive groups of students use the colleges popular with their peers as “local comparators,” and thus have a hard time evaluating a college that none of their friends has heard of. (The “illusion of attention” I wrote about in a previous post comes into play here, as well, further complicating the process.)

At the other end of the college choice timeline, Ariely’s research helps us understand why students with several college options have such difficulty making up their minds. His studies have shown that “we cannot stand the idea of closing the doors on our alternatives.” We dislike the idea of loss so much that we will devote a lot of energy to keeping options open – and that we often overlook the cost (in time and energy, among other things) of trying to maintain multiple opportunities. The cost of pursuing multiple opportunities is something worth discussing with seniors fortunate enough to have several offers of admission (along with a few wait list opportunities they can’t bear to part with) this spring.

As Brooks’s columns and Ariely’s research show, we (and the students with whom we work) will benefit from cultivating more awareness about the data we collect, and more self-awareness about the ways our own “filters” influence our use of that data as we make decisions. Neither we nor our students will be able to avoid completely the psychological cul-de-sacs of irrationality as we make decisions, but if we develop a general understanding of the factors that can lead us astray, we can decrease the number of post-decision “d’oh!” moments we experience.