CSAP troubles revealed
Despite staff training, pages of guidelines in a thick procedures manual and growing familiarity with the exam, Colorado Student Assessment Program testing doesn’t always go smoothly.
Ninety-eight CSAP tests have been invalidated during the past five years in School District 51 because of actions that may compromise a test’s accuracy. These actions can include anything from a student skipping ahead to another section of an exam booklet, the most common reason for a test to be thrown out, to tests not being locked up properly overnight or a teacher helping students answer test questions.
District 51 students take about 43,000 CSAP tests each year and recorded eight such incidents in 2010 as well as in 2009. The state, which invalidated 6,844 tests for administration problems in 2010 and 927 tests in 2009, gives about 1.6 million CSAP tests annually.
The Colorado Department of Education investigates about 50 to 60 allegations of faulty test administration each year, according to Colorado Department of Education Director of Student Assessments Joyce Zurkowski. Zurkowski said that number hasn’t budged much over the years.
“Because (CSAP tests) are ad-ministered by people and taken by people, there’s always going to be error that occurs. Our goal is to minimize that as much as possible,” she said.
District 51 decreased its number of tests thrown out from 49 in 2007 to eight in 2010.
“We’ve improved our training and are more specific about training in the procedures manual,” District 51 spokesman Jeff Kirtland said.
Minor and obvious administration problems can be decided through a conversation at the school building level, and those are often not recorded in documents available to the public. That’s why it’s hard to pinpoint exactly how many potential administration errors a district has experienced. More serious or questionable cases of possible administration problems are investigated by district administrators, making those situations easier to count.
District 51 Assistant Director of Elementary Schools Lesley Rose, who has conducted administration investigations, said the district takes seriously any hint of a test being compromised.
“CSAP is such a high-stakes test. We really need to be thorough and secure in how CSAP is given in every school,” she said.
Rose added, “There are going to be mistakes,” but she feels bad for the students who receive a zero because of a test-administration error.
“It not only hurts the school, but it hurts the child because the child and the family don’t get a true reading of the student’s progress that year,” she said.
The district had two investigations this year of possible administration problems. In both cases, the allegations were declared unfounded.
One investigation at Clifton Elementary was discussed verbally only, so there is no physical record of the incident. The other event was documented and investigated by Rose and District 51 Assessment Director Sean Taylor.
Glen Sirakavit, CSAP Principal Consultant for the state Department of Education, asked Rose and Taylor to investigate Rocky Mountain Elementary after the department received an anonymous letter in March that alleged Rocky Mountain Principal Patti Virden looked at the third-grade CSAP reading test, which is given before the other CSAP tests, and asked a third-grade teacher to teach poetry before the test was administered because there were questions about poetry on the exam.
After Rose and Taylor investigated the claim, Taylor sent a letter to Sirakavit saying he did not believe Rocky Mountain students were given an unfair advantage on the test because the teacher and others did not change their lesson plans to include poetry.
Taylor said he and Rose interviewed teachers and Virden, and Virden admitted she “previewed the CSAP test briefly prior to the administration, but she had not ever made any direct comments about teachers needing to focus on any specific topic (poetry or any other),” according to Taylor’s letter. The CSAP manual does not prohibit administrators from viewing a CSAP test, but they are not allowed to influence what is taught in a classroom based on what they see.
The teacher Virden spoke with told her interviewers that Virden asked her what units she had been teaching and felt Virden was upset that she had not mentioned poetry “and that she got the impression that Mrs. Virden wanted them to focus on poetry,” according to the letter. She said Virden’s interest in having her teach poetry was implied, not stated directly.
The investigation satisfied Department of Education personnel, Zurkowski said.
“After a thorough investigation by the district it was determined that testing was not compromised,” she said.
Three incidents in 2007
In an examination of CSAP administration problems recorded since 2007, four investigations were severe enough to involve documented contact with the Colorado Department of Education. Three of those incidents happened in 2007, when 49 tests were invalidated in the district because of administration problems.
In one 2007 incident, fourth-grade students at Shelledy Elementary informed their teacher a substitute teacher from the previous day had helped them take a CSAP math test. The substitute walked around the room during the test, pointed at test questions and told students to be more specific or check their answers.
The permanent teacher asked the class how many students received help from the substitute. Ten children raised their hands. Those students’ tests were invalidated.
Another 2007 report led to the invalidation of six students’ tests at Grand Junction High School, where a test proctor allowed students to take reading and writing tests earlier than other students.
Another report from that year did not lead to any tests being tossed, but an administration problem was recorded because a class of 10th-graders was not allowed to use scientific calculators on a math test.