Newsweek just released its list of the top 100 U.S. high schools. Like the more famous U.S. News college rankings, Newsweek relies on a numerical formula. Here is Newsweek’s formula:
Public schools are ranked according to a ratio devised by Jay Mathews: the number of Advanced Placement and/or International Baccalaureate tests taken by all students at a school in 2004 divided by the number of graduating seniors.
Both parts of this ratio are suspect. In the numerator, they count the number of students who show up for AP/IB tests, not the number who get an acceptable score. Schools that require their students to take AP/IB tests will do well on this factor, regardless of how poorly they educate their students. In the denominator is the number of students who graduate. That’s right – every student who graduates lowers the school’s rating.
To see the problems with Newsweek’s formula, let’s consider a hypothetical school, Monkey High, where all of the students are monkeys. As principal of Monkey High, I require my students to take at least one AP test. (Attendance is enforced by zookeepers.) The monkeys do terribly on the test, but Newsweek gives them credit for showing up anyway. My monkey students don’t learn enough to earn a high school diploma – not to mention their behavioral problems – so I flunk them all out. Monkey High gets an infinite score on the Newsweek formula: many AP tests taken, divided by zero graduates. It’s the best high school in the universe!
Why does Newsweek use this formula? There are two reasons, I think. First, they seem to conflate AP courses with AP exams. It is indeed good if more students take genuine AP courses, which teach the most challenging material. But there’s no point in having students take the AP exams if they’re not prepared. Some schools require their students to take AP exams, whether they’re prepared or not. The Newsweek formula rewards those schools. Here’s Jay Mathews, in Newsweek’s online FAQ:
If I thought that those districts who pay for the test and require that students take it were somehow cheating, and giving themselves an unfair advantage that made their programs look stronger than they were, I would add that asterisk or discount them in some way. But I think the opposite is true. Districts who spend money to increase the likelihood that their students take AP or IB tests are adding value to the education of their students. Taking the test is good. It gives students a necessary taste of college trauma. It is bad that many students in AP courses avoid taking the tests just because they prefer to spend May of their senior year sunning themselves on the beach or buying their prom garb. If paying your testing fee persuades you, indeed forces you, to take the test, that is good, just as it is good if a school spends money to hire more AP teachers or makes it difficult for students to drop out of AP without a good reason.
Second, it appears that better data would have been harder to get. Schools report the number of AP tests taken, but it appears that many don’t report anything about the scores their students receive.
Given Newsweek’s questionable formula, is it picking the best schools in the U.S.? Not likely. My wife, on reading Newsweek’s list, was surprised to see Oxnard High (of Oxnard, California) ranked as the 60th best. She was born in Oxnard and went to a nearby high school, and had never thought of Oxnard High as an elite school.
(To be clear: in no way am I comparing Oxnard High to Monkey High. Oxnard High seems like a pretty typical school by U.S. standards. Many of my wife’s friends graduated from Oxnard High. But, despite what Newsweek says, it’s not one of the very best schools in the country.)
Looking at standardized test scores – the actual scores, not the percentage of students who showed up for the test – Oxnard High appears to be a bit below average among California schools. Oxnard High students had an average SAT score of 997, compared to a state average of 1012; and 23% of Oxnard students took the SAT, compared to 37% statewide. 28% of Oxnard students met University of California admissions requirements, compared to 34% statewide.
What really makes Newsweek’s formula look bad is the data on AP test scores. If we use an improved version of Newsweek’s formula – dividing the number of AP scores of 3 or above (on a 5-point scale), by the number of enrolled juniors and seniors – Oxnard High scores 0.08, compared to a state average of 0.24. Many Oxnard High students take AP tests, but few score well. These are not the statistics of a top-performing school.
Here’s my report card for Newsweek’s high school ratings:
| English: | Proficient |
| Math: | Needs Work |
Leave a Reply