Letter to the Editor:
Educational testing has changed substantively. It’s likely many of us do not understand how much and in what ways, leading us to misinterpret test results. This misinterpretation can lead to erroneous conclusions such as ‘the system is broken’.
To start with, the tests used have changed numerous times over recent decades. Each test measuring different content in different ways making comparisons complicated. Starting in 2015 states were allowed to design their own test. NH chose to start using the New Hampshire Statewide Assessment System (NH SAS) in 2019. This is a statewide assessment administered in grades 3-8 to assess English language arts (reading and writing) and mathematics, and in grades 5, 8 and 11 science. NH also offers the Dynamic Learning Maps Alternate Assessment for students with the most significant disabilities. And since 1969, the National Assessment of Educational Progress (NAEP) has been administered to a representative sampling of NH students in grades 4, 8, and 12 in public and nonpublic schools. So be aware of which test, how long it has been used and how it compares to others you’re referencing.
Contrary to some people’s impression, current tests are designed to push student achievement and hold students to high standards. What does this mean - to test to a high standard? Here’s an example. To achieve “proficient” on the NAEP, today an 8th grade student in math must master the same concepts as a 12th grader had to in the 1990s. So be aware of the test’s standards.
Today’s tests assign performance labels, such as “basic”, “proficient” and “advanced”, rather than the numerical or letter designations many of us are use to. Since these performance labels are common terms we each interpret them applying our understanding of the words. For example, I may think “proficient” means competent, you may think it means average and someone else interprets it as very good. Obviously, these are vastly different interpretations. In the context of the NAEP, proficient means “demonstrated competency over CHALLENGING subject matter”. And as noted above in the example of the 8th grader and math — “challenging” is a higher level of knowledge than in previous tests. Now, consider the level of knowledge associated with ‘advanced.’ So be aware of the applicable meaning of each performance label as well as the knowledge standard.
NH SAS scores are mapped into four performance levels: Below Proficient, Approaching Proficient, Proficient and Above Proficient. Generally, students performing on NH SAS at Proficient and Above Proficient are “on track to mastery of the knowledge and skills necessary for college”. It’s not a high school graduation standard which many of us might assume. In fact, while the tests are administered at specific grade levels they are not designed to measure whether the student is ready to move on to the next grade. These standards for content and skills can vary substantially from the NH SAS. So be knowledgeable about the context of the test.
It’s also important to know whether the number used represents one performance level or a combination. For example, is the percentage representative of just proficient or a combination of proficient and advanced. Obviously, single numbers reveal a different level of achievement for the population than a combined number.
The four performance levels listed above are associated with minimum and maximum scale scores. These minimum and maximum scores are the cut scores. For example, in grade 3 reading proficiency numerical cut scores: Below Proficient = 420–556; Approaching Proficient = 557–586; Proficient = 587–615; Above Proficient = 616–750. So the highest ‘below proficient’, along with the lowest ‘approaching proficient’ scores are in fact 74% of a perfect score. Obviously, ‘below proficient’ and ‘approaching proficient’ students are learning and making substantive progress. So know the associated numerical cut scores before declaring what the performance level indicates.
In addition to an overall score, students receive reporting category scores, which represent distinct groups of knowledge within each level and subject. For example, math reporting categories for grades 3-5 are: 1. Operations and Algebraic Thinking, 2. Number and Operations in Base Ten and Fractions, 3. Measurement, Data, and Geometry, 4. Modeling and Problem Solving, 5. Use Mathematical Reasoning. Performance in these categories is reported using three achievement categories: Low, At or Approaching, and On or Above. If a student’s performance is categorized ‘At or Approaching’ it indicates there is not enough information to tell whether the student reached the Proficient mark because responses lay both above and below the proficiency level. The testing is substantively more dimensional than most of us realize.
Today’s testing is much more complicated than discussions suggest — particularly when they only focus on what percentage of students are ‘proficient.’ As well, this is only the ‘the tip of the iceberg’ when it comes to understanding test results. We should all consider if we know what’s necessary to make informed and accurate conclusions. If we are not willing to do the research, ask questions and adjust our thinking accordingly, we should not be making declarations about a ‘broken system.’
Karen Burnett-Kurie
Comments