“Juking the stats”

It’s happened again.  Another well-known educational institution has owned up to errors in the data it supplied to the US News & World Report for the purposes of computing college rankings.  This time, it’s George Washington University, which, for more than a decade, according to an article in the Washington Post, inflated a “key measure of the academic credentials of its incoming freshmen.”

As a fan of the TV series “The Wire,” this latest incident of erroneous data reporting put me in mind of the episode in Season 4 (“Know Your Place”) where Pryzbylewski (a police officer turned middle-school teacher) has a memorable exchange about “juking the stats.”

The news about GW also caused me to reflect on a recent article in Slate by Seth Stevenson, who wrote about experimenting with the purchase of “zombie” followers for his Twitter account.

Whether it’s for the purpose of improving an institution’s position in the USNWR rankings, posting a positive change in test results for middle-schoolers, or increasing one’s visibility among the “Twitterati,” we’re talking about the same thing: the misuse and misapplication of data to enhance recognition and popularity, or to give the appearance of success — substituting dubious short-cuts for bona fide progress toward a goal.

Seth Stevenson explained why Tweeters might be moved to purchase zombie followers.  “So why do people do this?  I assume it’s in part to create an illusion of success that people hope will be self-perpetuating.  It’s like showing up to a date in a rented Mercedes drop-top when in real life you drive a dinged-up Kia.  To the casual observer, your numerous fake Twitter followers suggest you’re a social media powerhouse — a person of influence not to be ignored.  It also seems like fake followers might beget more real followers.  I noticed that after I’d bought my zombie followers, the rate at which new, nonzombie people followed me seemed to rapidly accelerate.”

I think this explanation, though specific to life in the Twitter-verse, has equal resonance for colleges and universities that manipulate data to increase visibility in the rankings sweepstakes.  And so does Stevenson’s warning:  “you might be embarrassingly busted.”

I don’t want to get into the issues of who, how, when, and which erroneous data has been reported to the USNWR — really, this is nothing new.

There are two points I want to make, though.  First, a sad aspect of data manipulation for the USNWR isn’t just that it happens at all.  The bigger loss is that colleges and universities consistently fail to use their “bully pulpits” to educate prospective students about how to evaluate the glut of objective and subjective information available about the institutions they are considering.  Admissions officers are missing teachable moments, when guidance about how to assess the sources, reliability, validity and personal relevance of information could be offered to prospects and their parents.  One might respond that this task is the province of, or better left to, teachers and college counselors on the high school side of the admissions desk.  In my experience, though, guidance of this type, were it to be offered by college admissions officers, would be a valuable and well-received supplement to the guidance provided in high schools. (For now, we’ll leave aside the issue that fewer and fewer high schools have the resources and staff to provide this guidance in the first place.)   We all know that information provided by those holding the prizes (in this case, offers of admission) will be taken more seriously than those on the “supplicant’s” side of the admissions process.

Second, as a culture, we have arrived at a point where indicators of popularity and visibility are widely interpreted as hallmarks of quality.  Just as the Tweeter with more followers is seen as someone whose opinion is worth noting, a university with high name recognition, more applicants, and more applicants turned away, is seen as outstanding and desirable.  Popularity and quality are not mutually exclusive, by any means — but neither are they completely synonymous.

I understand the circumstances that give rise to this confusion.  I’ve written before about the difficulty that too much information (TMI) and too many choices (TMC) create for students.  Faced with TMI and TMC, it’s not surprising that students (and their parents) look for shortcuts as they research college options.  It’s much easier to find and use the opinions of others (via the number of Twitter followers, “likes” on a Facebook page, or a spot in a college ranking) than to do the research and self-examination necessary to develop one’s own opinion.  It’s a harder and more time-consuming process to evaluate a specific institution’s appropriateness for a specific individual than to assess an institution’s visibility and popularity.

Still, we have to acknowledge that we reap what we sow.  Institutions that decry the rankings on one hand but then turn around and promote the heck out of a top-ten appearance in one ranking or another teach prospective students that this measure has value.  Prospective students and parents who focus on measures of visibility and popularity when assessing colleges encourage institutions to seek and promote those measures to an extent which is undesirable.  What we’re left with is the college admissions twist on Newton’s third law of motion.  In the interactions between colleges and prospective students, we don’t find an action in one camp causing an equal and opposite reaction in the other camp.  Instead, we find that every action causes an equal and amplified reaction from the opposite cohort.