REPORT OF THE

ACHIEVING

CLASSROOM EXCELLENCE II

TASK FORCE

 

 

Adopted December 27, 2007

 

 

Dr. Jo Pettigrew and Dr. Janet Barresi, Chairs

 

 

 

Report of the Achieving Classroom Excellence II Task Force

April 15, 2008

 

Introduction and Purpose:

 

The Achieving Classroom Excellence (ACE) II Task Force was created pursuant to SB 921 of the 2007 Session, authored by Sen. Clark Jolley and Rep. Tad Jones.  The study was authorized out of concern over differences in test scores by fourth- and eighth-grade students in reading and mathematics on the Oklahoma State Testing Program (OSTP) as compared with that seen on a representative sample of students on the reading and math assessments on the National Assessment of Educational Progress (NAEP).  Students performed significantly lower on the NAEP than did the same age students on the OSTP.

 

2007 NAEP and OSTP Test Scores

READING

 

NAEP

OSTP

 

Nation

Oklahoma

 

4th Grade

 

 

 

Scale Score

220

217

 

% at Proficient

24%

22%

86%

% at Advanced

7%

4%

4%

 

 

 

 

8th Grade

 

 

 

Scale Score

261

260

 

% at Proficient

27%

25%

70%

% at Advanced

2%

1%

9%

 

2007 NAEP and OSTP Test Scores

MATH

 

NAEP

OSTP

 

Nation

Oklahoma

 

4th Grade

 

 

 

Scale Score

239

237

 

% at Proficient

33%

30%

63%

% at Advanced

5%

3%

19%

 

 

 

 

8th Grade

 

 

 

Scale Score

280

275

 

% at Proficient

24%

18%

54%

% at Advanced

7%

3%

23%

 

 

Based on these differences in test scores several questions were raised concerning the rigor of state content standards as well as the rigor and structure of state assessments currently administered under the Oklahoma State Testing Program.  The task force agrees that there are many factors that affect academic performance.  Some of these are the curriculum that is utilized, effectiveness of classroom instruction, class size, school leadership, length and number of instructional days, as well as the level of school funding.  While these and more may affect academic performance, it is the content and process standards that serve as the foundation upon which all teaching and learning are built.  Oklahoma’s content standards are definitive statements about what all children must know to be productive citizens and to compete in a national and global marketplace. If these state content and process standards are rigorous and if state guidelines for test construction result in assessments that achieve a high degree of alignment with those standards then it should follow that students who take both the state assessments and the NAEP should perform equally well on both examinations. 

 

In addition, the reporting of test results should be of such a form so that they facilitate interpretation by all stakeholders.  Classroom teachers and administrators should be able to use these test results as a tool to evaluate existing education programs and to modify others that will enhance learning.  In like manner, government leaders and the public should be able to monitor school performance and outcomes.  Parents should be able to easily monitor their own child’s performance while comparing overall performance outcomes of their school with other schools.

 

With all of this in mind, the committee was charged with examining the apparent disparity in scores between Oklahoma’s student performance on the National Assessment of Educational Progress with their performance on state mandated tests under the Oklahoma State Testing Program.  Specifically, the task force was assigned to study the following issues and submit a report of findings and recommendations to the Governor and Legislature by December 31, 2007:

 

  1. Comparison of the Priority Academic Student Skills with other states’ curricular standards, primarily states that score highest on the National Assessment of Educational Progress (NAEP);

 

  1. Alignment of the Priority Academic Student Skills with the National Assessment of Educational Progress (NAEP) standards;

 

  1. Feasibility of realigning the state performance level standards to NAEP performance level standards;

 

  1. Differences in achievement levels among states based on exclusion rates on the NAEP; and

 

  1. Feasibility of aligning the cut scores on state-mandated tests to NAEP cut scores.

 

 

Membership:

 

The task force was comprised of seven members selected from among public and private school educators and members of the business community, excluding any elected officials.  The members, their professional affiliation and appointing authorities were as follows:

 

  • Keith Ballard, Ed.D., Executive Director – Oklahoma State School Boards Association, appointed by the Governor to replace Jo Pettigrew who resigned;
  • Janet Barresi, DDS -- Dentist, appointed by the Speaker of the House of Representatives;
  • Susan Harris, Vice President -- Tulsa Chamber of Commerce, appointed by the Chair of the House Education Committee;
  • Phyllis Hudecki, Ed.D., -- Executive Director - Oklahoma Business & Education Coalition, appointed by Co-President Pro Tempore of the Senate;
  • Cleatta Johnson, M.Ed., -- Licensed Professional Counselor and Retired Educator, appointed by the Co-chairs of the Senate Education Committee;
  • Diana Leggett -- Asst. Supt. of Curriculum, Instruction and Personnel, Stillwater Public Schools;
  • Rick Martin, M.Ed. -- Superintendent of Prague Public Schools, appointed by the Minority Leader of the House of Representatives; and
  • Jo Pettigrew, Ed.D, -- Education Consultant, appointed by the Governor, but resigned prior to the expiration of the task force.

 

Dr. Pettigrew was elected by the members to serve as chair of the task force; however, her acceptance of an appointment by Governor Henry to serve on another commission precluded her continued service on the ACE II Task Force, and she was unable to preside at the final two meetings of the task force when this report was adopted.  The Governor then appointed Dr. Ballard to replace Dr. Pettigrew.  Dr. Janet Barresi was elected by the members to serve as chair of the task force for the final meetings.

 

Meetings:

 

The task force held seven meetings from September 24, 2007, to December 27, 2007, and heard presentations and comments from the State Department of Education, school districts, testing vendors, the Regional Education Laboratory Southwest, the Southwest Educational Development Laboratory, and the National Assessment Governing Board.  A list of all presenters follows:

 

Presenters:

 

·         Dr. Mary Crovo – Deputy Executive Director, National Assessment Governing Board

·         Dr. Vicki Dimock – Program Director, Southwest Educational Development Laboratory

·         Debra Ensminger – Director of Student Assessment, Jenks Public Schools

·         Shan Glandon – Director of Curriculum and Instruction, Jenks Public Schools

·         Dr. Cindy Koss – Asst. State Supt., Office of Standards and Curriculum, State Dept. of Education

·         Diana Leggett -- Asst. Supt. of Curriculum, Instruction and Personnel, Stillwater Public Schools

·         Rick Martin – Superintendent, Prague Public Schools

·         Dr. Maridyth McBee -- Vice President, Assessment Services, Pearson Education

·         Dr. Lisa McGlaughlin – Asst. Superintendent, Western Heights School District

·         Dr. Dean Nafziger – Director, Regional Education Laboratory Southwest

·         Todd Nelson – Director of Student Assessment, Union Public Schools

·         Don Rader – Superintendent, Alva Public Schools

·         Jennifer Stegman – Asst. Supt., Office of Accountability and Assessments, State Dept. of Education

·         Becky Szlichta – Coordinator of Testing, Stillwater Public Schools

·         Kerri White – Mathematics Curriculum Director, Office of Standards and Curriculum, State Dept. of Education

 

Findings:

 

The findings and recommendations of the task force are organized below under each of the items the task force was charged with studying.

 

ITEM 1:  Comparison of the Priority Academic Student Skills with other states’ curricular standards, primarily states that score highest on the National Assessment of Educational Progress (NAEP).

 

We compared Oklahoma’s Priority Academic Student Skills (PASS) with other states’ curricular standards indirectly by way of review of national studies conducted by the U.S. Chamber of Commerce, Achieve, the American Diploma Project, Thomas B. Fordham Foundation, Regional Education Laboratory Southwest, The Education Sector, and various colleges and universities.  None of these organizations have completed a comprehensive comparative review of K-12 curricular standards in each state.  Most of these studies involved an indirect comparison by comparing Oklahoma’s standards with its student’s performance on the NAEP.  

 

In its report, Leaders and Laggards: A State-by-State Report Card on Educational Effectiveness, Oklahoma was given an overall grade of “C” for the rigor of our standards.  Oklahoma’s English and Math standards were given a grade of “C” while our science standards were given a grade of “F”.  However, Oklahoma was only one of eight states that has aligned high school graduation requirements with college and workplace expectations.  While no specific information was given concerning the methodology used to derive these grades for each state, the information in this report shows that 11 states scored higher than Oklahoma and 19 states had the same grade of “C” as Oklahoma.  The states of New York, California, Indiana and Massachusetts received an overall grade of “A”.

 

Oklahoma’s curricular standards are developed by committee with support from the State Department of Education.  The committee is comprised of classroom teachers, curriculum directors, higher education professionals and when possible, individuals from Oklahoma’s business community whose area of expertise is in the subject area being developed.  The State Department of Education is also responsible for setting test construction guidelines, developing the tests through an independent contractor, defining performance level descriptors, setting test cut scores and administering the test.

 

The NAEP framework is similar but not identical in form to state content standards.  The framework was developed with the help of educators, curriculum directors, higher education professionals as well as members of the business community from across the country.  It has gone through exhaustive independent review.  The NAEP framework, much like state content and process standards is meant to be a statement about what all children should know and be able to do in each subject and at each grade level assessed.

 

The State Department of Education presented a comparison of state content and process standards to the NAEP framework using the Surveys of Enacted Curriculum (SEC).  The comparison covered math and reading in grades 4 and 8.  The SEC is a data tool which was principally designed to provide educators a subjective self analysis of the alignment of their own teaching practices with state content standards.  The tool was developed by the Wisconsin Center for Ed Research under the sponsorship of the Council of Chief State School Officers (CCSSO).  According to the CCSSO web site, CCSSO and project partners in 26 states are assisting educators to implement applications of the Surveys.  While teacher practices and instructional content are self-reported, standards and assessments are analyzed by independent content experts, such as state level curriculum specialists from other states, university faculty, and business professionals.  Therefore, cohort partners from other states performed the coding for this analysis.

 

The analysis shows that Oklahoma’s mathematics standards are more closely aligned to the NAEP framework for Grade 4 than are those of Massachusetts which scored higher on the NAEP assessment (OK – 234, MA – 247).  According to the SEC comparison, Oklahoma’s mathematics standards for Grade 4 have a high degree of alignment with NAEP with a score of 0.329 as opposed to that of Massachusetts with a score of 0.279.  A comparison of alignment scores for other states that outperformed Oklahoma on the NAEP is included in the table below.

Grade 4 Mathematics

 

2005 NAEP score

SEC alignment score

Oklahoma

234

0.329

Massachusetts

247

0.279

Idaho

242

0.294

Montana

241

0.206

Iowa

240

0.257

 

A similar comparison for Grade 8 Mathematics does not reveal accurate information due to the fact that Oklahoma’s Grade 8 Mathematics Process Standards were not included in the Surveys of Enacted Curriculum study when it was originally conducted.  This oversight by the Wisconsin Center for Education Research resulted in a low alignment score for the Grade 8 mathematics standards as compared to the NAEP framework.

 

The SEC shows that Oklahoma’s standards, along with the standards of most other states, include a wider variety of content in Language Arts than is included in the NAEP Reading framework.  Both the Grade 4 and Grade 8 Language Arts PASS standards include content not covered by the NAEP Reading framework such as Vocabulary, Writing Process, Writing Components, Writing Applications, Listening and Viewing, as well as topics with minimal coverage in the NAEP Reading framework such as Phoneme Awareness, Language Study, Speaking and Presenting.  It should be noted however that NAEP has a separate Writing framework and assessment.  The broader body of content used by most states results in lower alignment scores, but comparisons between states shows Oklahoma with a higher alignment score than many states that outperform Oklahoma on the NAEP. 

 

Grade 4 Reading

 

2005 NAEP score

SEC alignment score

Oklahoma

214

0.231

Vermont

227

0.223

Maine

225

0.208

Ohio

223

0.212

Idaho

222

0.215

Wisconsin

221

0.185

 

Grade 8 Reading

 

2005 NAEP score

SEC alignment score

Oklahoma

260

0.217

Vermont

269

0.192

Ohio

267

0.173

Idaho

264

0.145

Indiana

261