Scores for the June 2018 SAT have been released and high-scoring students are not happy.  The curve on the Math section was really unforgiving to high-scoring students.  The test is a reminder that students should be ready to retake the SAT and ACT and that no one actually wants an “easy” SAT.

UPDATE (7/23):  The May 2018 curve has been added to this post. The curve for June 2018 has been expanded to include scores between 600 and 800.  The scaled score for a raw score of 52 has been corrected to be 670, not a 680 as we had it previously.

UPDATE (7/12):  Some material has been added to this post to explain why we classify the June SAT Math sections as “easy.”

In recent months, the College Board has been sending a survey to students asking them which test the think is easier, the SAT or the ACT? Here is a screenshot of a survey I received after taking the March 2018 exam. (Apologies for the grainy quality.)

March 2018 College Board Survey for SAT Examinees

It is hard to imagine why the College Board would ask this question unless it wanted to change the image of the test in order to make it more popular among states, school districts, and students. For years, the ACT has had the reputation of being an easier exam, so perhaps the College Board wants to reverse that image.

The problem is that an easier test is no good for students or for colleges using test scores to evaluate applicants.  To explain why, we need to discuss one of the fundamental aspects of standardized tests:  equating.

For a standardized test to be of any value, it needs to be possible to compare the scores of someone who took the test in June 2018 to someone who took it in March 2018, June 2017, October 2016, etc.  College Board cannot just give the same test at each administration, and it’s really hard to make each test exactly as hard as every other test.  As a result, test makers need to adjust the scaled score, which is based on the raw number of correct answers, on each test to make sure they’re comparable.  Here’s how ETS, which used to write the SAT, explains equating.

Statistically adjusting scores on different forms of the same test to compensate for differences in difficulty (usually, fairly small differences). Equating makes it possible to report scaled scores that are comparable across different forms of the test. (ETS website)

For the most part, equating works really well, in part because there are not large differences between the tests.  Most test makers have been doing this for a long time, and they are good at doing what they do.

College Board took over the responsibility for writing the SAT from ETS when they redesigned the exam.  The roll-out, as Reuters noted in a series of stories on the new SAT, has not always been smooth, particularly with respect to the math section of the exam.

In a nearly 5,000-word letter from August 2014, one reviewer told College Board officials that he had “never encountered so many seriously flawed items” in the 20-plus years he had been screening math material for the organization. (Reuters, 2016)

Despite these early jitters, things seemed more or less fine with the new test’s Math sections. Until today.

June 2018 SAT scores came out, and students took to Reddit to decry the Math curve for the exam.  Students who got fewer questions wrong on the test than on previous attempts woke up to lower Math scores. Here are the Math scales for the SAT exams that have been released by the College Board, including 4 tests from the Official Guide.  Tests 1-4 were never used for an official testing date, so we excluded them.  Tests 5-8 were.

Math RawJune 2018 Math ScaledMay 2018 Math ScaledMarch 2018 Math ScaledOctober 2017 Math ScaledMay 2017 Math ScaledApril 2017 Math ScaledOfficial Guide Test 8Official Guide Test 7Official Guide Test 6Official Guide Test 5

The explanation is equating.  The June Math was easier, and as a result, the curve was harsher than usual.

Difficulty, to be sure, is in the eye of the beholder.  Many people (e.g., me) will think knitting is exceedingly difficult, but some people (e.g., my wife) think it is easy and fun.  Some, or perhaps most, students were bound to find the June Math sections to be difficult.  When we call the Math section “easy” we do not mean that everyone should have found it so.  We mean that the scoring curve indicates, objectively, that students tended to get fewer questions wrong than they did on other SATs.  That made the curve less forgiving.

It’s worth noting that the College Board’s own difficulty ratings for questions, found in the Question and Answer Service (QAS) and Student Answer Service (SAS) forms that students can order for their tests have been very unreliable for the revised SAT.  Their ratings should be, in a word, ignored.

Score equating is done before the test is ever given, so it’s worth saying that the actual performances on test day did not affect the curve.  College Board knew it was going to administer an easier test, which meant more students would get more questions right, and the scale would need to undergo adjustment.  As a result, small differences had a larger impact than usual.

To a degree, this is how it should be.  A student who misses two questions on an easier test should not get as good a score as a student who misses two questions on a hard test. Equating takes care of that issue.

However–and you knew there was going to be a “however”–the equating applied to the June 2018 SAT suggests that the College Board made the test far too easy to distinguish among high scorers who received a score of 600 (76th percentile) or higher. That is a problem for those colleges who treat a 600, 650, a 700, a 750, and an 800 as accurate indicators of real differences in Math ability.

It is a problem, too, for high-scoring students who make the occasional careless error or who mis-bubbles on questions that they are quite capable of answering.  With a typical curve, there’s some cushion to mitigate the impact of such errors.  There was no cushion on the June 2018 SAT.

It might be fair to say that the most accomplished students shouldn’t make those kinds of errors, but is that true?  Wouldn’t it be more accurate to say that the most accomplished test takers don’t make those kinds of errors. Small mistakes under time pressure can make a big difference in life, no doubt, but doing well in college tends to be about doing well over time with the possibility to revise, rethink, and do better.

The students shocked by the June 2018 SAT will have a couple more chances to retake the test.  But what if the same thing happens in October or November, when seniors often take their last shot at the exam? We have to hope that this exam is an anomaly, and the College Board won’t be administering too many more “easy” tests.

The fact that two Reading questions and two Writing and Language questions were omitted from scoring on this same exam does not inspire much confidence.

We would be completely and utterly shocked to see the College Board rescale the exam, as many students and families are demanding. It is important to note that college admissions officers are not going to weigh how how many questions a student got wrong. They will look at the scores.  Nor will they discount a June 2018 SAT score as somehow compromised. If a student did well on the June exam, she or he should be proud and not worry at all about admissions officers thinking that the June test was a bad one.