SAT Essay Test Rewards Length and Ignores Errors

[HeavyRescueTech]

SAT Essay Test Rewards Length and Ignores Errors
By MICHAEL WINERIP

Published: May 4, 2005


CAMBRIDGE, Mass.

IN March, Les Perelman attended a national college writing conference and sat in on a panel on the new SAT writing test. Dr. Perelman is one of the directors of undergraduate writing at Massachusetts Institute of Technology. He did doctoral work on testing and develops writing assessments for entering M.I.T. freshmen. He fears that the new 25-minute SAT essay test that started in March - and will be given for the second time on Saturday - is actually teaching high school students terrible writing habits.

"It appeared to me that regardless of what a student wrote, the longer the essay, the higher the score," Dr. Perelman said. A man on the panel from the College Board disagreed. "He told me I was jumping to conclusions," Dr. Perelman said. "Because M.I.T. is a place where everything is backed by data, I went to my hotel room, counted the words in those essays and put them in an Excel spreadsheet on my laptop."

In the next weeks, Dr. Perelman studied every graded sample SAT essay that the College Board made public. He looked at the 15 samples in the ScoreWrite book that the College Board distributed to high schools nationwide to prepare students for the new writing section. He reviewed the 23 graded essays on the College Board Web site meant as a guide for students and the 16 writing "anchor" samples the College Board used to train graders to properly mark essays.

He was stunned by how complete the correlation was between length and score. "I have never found a quantifiable predictor in 25 years of grading that was anywhere near as strong as this one," he said. "If you just graded them based on length without ever reading them, you'd be right over 90 percent of the time." The shortest essays, typically 100 words, got the lowest grade of one. The longest, about 400 words, got the top grade of six. In between, there was virtually a direct match between length and grade.

He was also struck by all the factual errors in even the top essays. An essay on the Civil War, given a perfect six, describes the nation being changed forever by the "firing of two shots at Fort Sumter in late 1862." (Actually, it was in early 1861, and, according to "Battle Cry of Freedom" by James M. McPherson, it was "33 hours of bombardment by 4,000 shot and shells.")

Dr. Perelman contacted the College Board and was surprised to learn that on the new SAT essay, students are not penalized for incorrect facts. The official guide for scorers explains: "Writers may make errors in facts or information that do not affect the quality of their essays. For example, a writer may state 'The American Revolution began in 1842' or ' "Anna Karenina," a play by the French author Joseph Conrad, was a very upbeat literary work.' " (Actually, that's 1775; a novel by the Russian Leo Tolstoy; and poor Anna hurls herself under a train.) No matter. "You are scoring the writing, and not the correctness of facts."

How to prepare for such an essay? "I would advise writing as long as possible," said Dr. Perelman, "and include lots of facts, even if they're made up." This, of course, is not what he teaches his M.I.T. students. "It's exactly what we don't want to teach our kids," he said.

SAT graders are told to read an essay just once and spend two to three minutes per essay, and Dr. Perelman is now adept at rapid-fire SAT grading. This reporter held up a sample essay far enough away so it could not be read, and he was still able to guess the correct grade by its bulk and shape. "That's a 4," he said. "It looks like a 4."

A report released this week by the National Council of Teachers of English mirrors Dr. Perelman's criticism of the new SAT essay. It cautions that a single, 25-minute writing test ignores the most basic lesson of writing - that good writing is rewriting. It warns that the SAT is pushing schools toward "formulaic" writing instruction.

This is a far cry from all the hoopla when the new SAT was announced two years ago. College Board officials described it as a tool that could transform American education, forcing schools to better teach writing. A "great social experiment," Time magazine said.


In an interview, five top College Board officials strongly defended the writing test but sounded more muted about its usefulness. "The SAT essay should not be the primary way kids learn to write," said Wayne Camara, vice president for research. "It's one basic writing skill. If that's all the writing your high school English department is teaching, you have a problem."

They said that while there was a correlation between writing long and a high score, it was not as significant as Dr. Perelman stated. Graders also reward good short essays, they said, but the College Board erred by failing to release such samples to the public. "We will change that," said Chiara Coletti, a vice president.

As to facts not mattering, they said it was a necessary accommodation on such a short, high-pressure test. "We know students don't write well when they're anxious," said Ed Hardin, a College Board test specialist. "We don't want them not to go forward with that little detail. Our attitude is go right ahead with that missing date or fact and readers should be instructed not to count off for that."

Cynics say the new essay is window dressing added to placate California officials who in 2001 were calling the old SAT outmoded and were threatening to stop requiring it. In a recent paper, Edward White of the University of Arizona notes, "As long ago as 1999, in College Board Report No. 99-3, a research team pointed out that 'writing assessments based on a single essay, even those read and scored twice, have extremely low reliability.' "

Indeed, the College Board's own advanced placement tests require multiple essays, but officials say that is not possible for the SAT, which at nearly four hours, is being criticized as too long.

"You can't base a lot on one essay," Dr. Camara of the College Board admitted. He said that was why the new SAT writing section also included 49 multiple-choice questions on grammar and style. Multiple-choice counts for 75 percent of the new writing grade; the essay 25 percent. "The multiple-choice makes the writing test valid," he says. In short, the most untrustworthy part of the new SAT writing section is the writing sample.

E-mail: edmike@nytimes.com

http://www.nytimes.com/2005/05/04/ed...rssnyt&emc=rss

[DennisTheMenace]

Thank GOD I never have to take them again!
I did very well, but would not like to do the new format at all.

[sbfdco1]

VERY interesting article Parasite, thanks for that.

I have to agree with theMenace, even though I didn't score as well as I should've, enough to play college ball though - priorities!

[DaSharkie]

I never took them and recommend to every high schooler I encounter to not take unless absolutely necessary. Way too much weight is put on on exam. Plus, if you take a few classes here and there then matriculate into a program you have no need to even take the SAT or ACT. Or, you take a few classes at the community college and transfer in.

I had to take the GRE to get into my PA program, and even the faculty says that it doesn't mean much in the whole scheme of things, it is all about the candidate (of course that is at the graduate level.)

[DennisTheMenace]

Originally posted by DaSharkie
I never took them and recommend to every high schooler I encounter to not take unless absolutely necessary. Way too much weight is put on on exam. Plus, if you take a few classes here and there then matriculate into a program you have no need to even take the SAT or ACT. Or, you take a few classes at the community college and transfer in.

I had to take the GRE to get into my PA program, and even the faculty says that it doesn't mean much in the whole scheme of things, it is all about the candidate (of course that is at the graduate level.)

I can't think of a single accredited four year BS/BA program that will let you in on community college credits alone, no matter how many you have. If you would like to attend a real program you are going to need to take the SAT or ACT. As for the graduate level, most programs you will need decent scores, not to get in, but just to get the interview. Sad, but true.

[DaSharkie]

Originally posted by DennisTheMenace
I can't think of a single accredited four year BS/BA program that will let you in on community college credits alone, no matter how many you have. If you would like to attend a real program you are going to need to take the SAT or ACT. As for the graduate level, most programs you will need decent scores, not to get in, but just to get the interview. Sad, but true.

My B.S. is in Fire Science. Never needed to take the SAT or ACT. I got two Associate Degrees and simply enrolled in the bachelor's classes. Many school consortiums are the same. Maybe it is a thin in Massachusetts, but I could have gotten my A.S. in one of several programs and then driven over to a large number of schools (Worcester State College, Holy Cross, Clark University, Assumption College) and enrolled in a B.S. program without having to have taken either exam.

It was the same in Hawai'i when I began my educational process at Honolulu Community College, I could have transferred right into a 4 year program at the University of Hawai'i or one of several other schools.

These exams are nothing but a money making scheme and have little, if any, bearing on how a person is going to perform scholastically.

The only time I would tell anyone different is if they want to attend one of the military academies where there is no other way into them without taking the exam.

[DennisTheMenace]

Originally posted by DaSharkie


My B.S. is in Fire Science. Never needed to take the SAT or ACT. I got two Associate Degrees and simply enrolled in the bachelor's classes. Many school consortiums are the same. Maybe it is a thin in Massachusetts, but I could have gotten my A.S. in one of several programs and then driven over to a large number of schools (Worcester State College, Holy Cross, Clark University, Assumption College) and enrolled in a B.S. program without having to have taken either exam.

It was the same in Hawai'i when I began my educational process at Honolulu Community College, I could have transferred right into a 4 year program at the University of Hawai'i or one of several other schools.

These exams are nothing but a money making scheme and have little, if any, bearing on how a person is going to perform scholastically.

The only time I would tell anyone different is if they want to attend one of the military academies where there is no other way into them without taking the exam.

UH required me to provide my SAT scores as well as a good GPA to transfer from HPU, they also looked at my HS gpa. This was 1993.

[stm4710]

I guess this follows the theroy if you cant impress them with facts,dazzle them with bull****.


In ever took the SAT......and I am in college and a straight A student I might add.

[DaSharkie]

Originally posted by DennisTheMenace
UH required me to provide my SAT scores as well as a good GPA to transfer from HPU, they also looked at my HS gpa. This was 1993.

The high school GPA seems a bit rediculous at a certain point in life. I mean for cryin out loud, I have been out of high school for 12 years now, and attended other institutions and have a relatively high GPA, does anyone's high school GPA mean squat at this point in the game? Yet some programs that have my major want them. Why?

I know several colleges require the SAT, but there is no need for them. They prove nothing, show nothing, have nothing to do with anything. They are a typical standardized test, taken on one day. They mean squat, just like any other exam that is the same type of circumstance.

[DennisTheMenace]

Originally posted by DaSharkie


The high school GPA seems a bit rediculous at a certain point in life. I mean for cryin out loud, I have been out of high school for 12 years now, and attended other institutions and have a relatively high GPA, does anyone's high school GPA mean squat at this point in the game? Yet some programs that have my major want them. Why?

I know several colleges require the SAT, but there is no need for them. They prove nothing, show nothing, have nothing to do with anything. They are a typical standardized test, taken on one day. They mean squat, just like any other exam that is the same type of circumstance.

For a BA/BS they want to know what type of a student you have shown yourself to be. H.S. being the last diploma received it is what they have available.

I think that the SAT's mean a little more then you seem to. They are a base line to judge you among other applicants. I guess if you are applying to a non-competative admissions program then there might not be a legit reason to require them, but anywhere that they have to narrow the field down for a limited number of spots they are a good objective indication of applied intellegence vs. the major fluctuations in the way that grades are given, school to school, system to system and individual to individual.

[Maverick9110E]

this just proves my theory, SAT's are a waste of time. yea i took mine. yea i did avearage. its completley not feasable to base an entire persons apptitude on one freaking test!

[DennisTheMenace]

Originally posted by Maverick9110E
this just proves my theory, SAT's are a waste of time. yea i took mine. yea i did avearage. its completley not feasable to base an entire persons apptitude on one freaking test!

That would be true if SAT's were the ONLY thing that admissions departments looked at, but they are just one of many. Essentially used for intially sorting the kids out. I know some kids who did very well on their SAT's but did not belong at the top of the class, but I have NEVER met anyone that was or belong at the top of the class that had crummy SAT scores(unless it was a very crappy school where the top of the class was not saying much).

SAT's are a hassle, i am VERY glad i NEVER have to take them again, but they are an essential tool for any program where the admissions are even slightly competative.