It's a technological truism: Create a system of measurement, and somebody will try to game it. Google search algorithms give way to search-engine optimization. Crime-tracking statistics lead police departments to reclassify or whitewash incident reports to show a good trend line. Systems to rank employees end up encouraging coworkers to undermine and sabotage one another. In each of these cases, the signifier overwhelms that which it was meant to signify. The map becomes more important than the territory.
For decades, that dynamic has powered the $860 million SAT prep industry. For a hefty fee, companies like Kaplan and Princeton Review promise to juice your score, not necessarily by helping you grapple with the underlying math and reading comprehension but by teaching you hacks specific to the test itself. (When guessing, answer B or C instead of A or D.) They're quite up-front about this. "Our philosophy has never been content-oriented," a Princeton Review vice president told the Washington Post. "We teach people the tricks."
That's why the College Board---the nonprofit that writes and administers the SAT---keeps adjusting the test, attempting to refocus the exam on the underlying material. Its newest version will debut next year, and it's an attempt, per the company's own executive summary, to create a "robust and durable bond between assessment and instruction" that favors true mastery over "short-term test preparation divorced from real learning."
About a year ago, the College Board announced a new partner in this effort: Khan Academy, the beloved online warehouse of free instructional videos and interactive exercises, which worked directly with the College Board to create an extensive library of free, online prep materials. It's a radically democratic reboot of the test-prep industry, which up until now reserved its advantages for the relatively well-heeled who could afford their pricey tutorials.
The tutorials themselves, which went live this morning, are equally radical. They focused relentlessly on real learning. After completing a detailed assessment, students are ranked on a scale of 1 to 4 on skills such as "interpreting linear functions" or "quadratic exponential word problems." They are then drilled on their weakest subjects, advancing to more difficult problems as their mastery increases.
It's a pretty thrilling project. The College Board has made no secret of its desire to make the SAT more equitable; cooperating with Khan Academy to create free, high-quality test-prep materials is clearly a step in that direction. (Its partnership with Boys & Girls Clubs is another one.) It's also, as Slate's Jordan Weissman writes, a compelling opportunity to test the promise of online education, which has faced a series of setbacks lately.
But it also points to a radical idea---that the best way to prepare for a test might be to master the material it purports to cover, rather than amassing a series of tricks and hacks around the act of test-taking. That's clearly the message that College Board president David Coleman wants to send. In a call with reporters, he said he hoped this initiative would "stand up against the artificial advantages of costly test prep." If the SAT is really aligned with schools' curriculums, then the test itself becomes a natural extension of that schoolwork, not yet another hurdle on the way to a college education. "It doesn't help you in life to know whether it's best to guess or leave an answer blank," he says. "No longer should students choose between studying for class or studying for the SAT."
That being said, the notion of a game-proof test is probably impossible. There's a well-established principle that, the more important a metric becomes, the more subject it is to gamesmanship---and it's hard to imagine a more fraught metric than the SAT score. And to paraphrase Mencken, no one ever went broke by overestimating the anxieties of the American parent. At least publicly, the Kaplans of the world aren't exactly quaking. After all, the point of the SAT isn't just to score well---it's to score better than everyone else. If everyone has access to free prep tools, there will still be plenty of parents eager to pay for an extra leg up.
Still, any test-prep program that focuses on the subject matter, rather than the tricks of test-taking should be applauded. As Khan Academy continues to find ways to integrate itself into the educational infrastructure, it's also demonstrating that the best path to a more fair college-testing system may be to leave the maps behind.
