Untangling Assessments: What are they all for?

At WCSU, we use the assessment terms and definitions outlined in the Vermont Multi-tiered System of Supports Response to Intervention and Instruction (MTSS-RtII) Field Guide. The goal is to provide a supervisory union wide "balanced and comprehensive assessment system" that:

• Identifies students who require a closer look (screening); 
• Investigates and analyzes learning difficulties (diagnostic); 
• Informs "core" instruction (formative); 
• Monitors progress (progress monitoring); and 
• Verifies learning over time (summative).

Various common assessment tools are used to provide a full picture of students’ academic and/or behavioral knowledge, abilities, and dispositions. This is the point in developing the WCSU Local Common Assessment System, which is admittedly a work in progress. It's important to note that currently we are at the beginning of this process, especially in mathematics.

When we look at how math is being assessed across WCSU, we see a host of assessments. Not all of these assessments are common across the supervisory union. Not all of them are aligned with our non-negotiables. As we continue to move forward, we will be replacing and revising these assessments to create more cohesion and alignment.

The aim of this blog post is to take a look at some of the assessments that we are using right now and to provide a clear message for those of us who use them. What are these assessments? How should they be used? How should they influence our instruction? How should they influence grades and reporting?

easyCBM

What is it? easyCBM is a universal screener for grades K-8. It is a multiple choice, norm referenced assessment - which means it compares our students' performance with the performance of all of the other students taking the test across the country. It is meant to identify students who might be having difficulties and flags students in the 25th percentile and below as having "some" or "high" risk of having difficulties.

Although easyCBM is aligned to the Common Core States’ Standards, it is not fully aligned with our non-negotiables (and no universal screening tool is).

easyCBM also includes some progress monitoring capabilities. However, we are not using these currently.

How should it be used? Data teams take a closer look at students who are flagged as well as students who were expected to be flagged but were not. Other data points (such as classroom performance observations, additional assessments) are considered in addition to easyCBM performance to decide which students need an intervention.

Screening tools like easyCBM are not intended to be used for item by item analysis to diagnose difficulties. Ideally, we would use a diagnostic assessment for that.

How should it influence instruction? As mentioned above, easyCBM is not fully aligned with our WCSU Non-negotiables. Therefore, it should not be used to measure progress in relation to the benchmarks (even though confusingly it claims to be a "benchmark screener").

easyCBM should not be used to determine the conceptual content of your instruction. Instead, use the WCSU Math Non-negotiables to determine scope and sequencing.

WCSU Benchmark Assessments

What are they? The "benchmarks" are our WCSU Math Non-negotiables which are developmental content standards that describe sequences of growth. Benchmarks are usually measured three times per year (fall, winter, spring) using benchmark assessments.

Benchmark assessment tasks should be "non-routine" (where students must connect and apply concepts in new ways) and encourage higher order math thinking, but they should also accommodate a range of strategies so that we can see if students' strategies are developmentally appropriate.

The Math Steering Committee developed draft benchmark assessment tasks for grades K-9 last year with the guidance of Karin Hess in a joint effort with Orange North Supervisory Union. These benchmark tasks were included in the August 10, 2015 draft of the WCSU Local Common Assessment System. However, after closer scrutiny, the Math Steering Committee feels that these assessments are not ready to be rolled out universally. Although they are a solid foundation, they still require revision and greater alignment with the WCSU Math Non-negotiables.

This being the case, rather than requiring them to be administered universally this year, we will be piloting revised, aligned tasks.  All teachers of mathematics will participate in the pilot by volunteering to pilot a task at their grade level.

How should they be used?  Benchmark assessments should be used to determine if students are making adequate progress in overall performance in relation to predetermined grade expectations and to monitor, over time, students’ progress with respect to the benchmarks.

How should they influence instruction? They should be aligned with scope and sequencing of math concepts and in that regard can be used to guide conceptual content. However, currently only the revised tasks should be used that way. These revised tasks will be released only to piloting teachers initially to minimize confusion.

Since they are designed to be non-routine, the tasks should not influence the specific contexts you address in class. For example, if there is a task that relates to running a fraction of a lap around a track, you should not be having them do a similar problem in class with slightly different numbers. We want to see how flexible our students are.

Instead, make sure you expose your students to a myriad of contexts, be sure to have them to explain their thinking using multiple representations (pictorial, symbolic, etc.), have them consider other strategies, look for patterns and connections, and make and test out conjectures. In other words, embed the Math Practices in your students' math experience everyday like nobody's business.

SBAC

What is it? SBAC stands for "Smarter Balanced Assessment Consortium." It is a computer based summative assessment for grades 3-8 and 11 that replaces the NECAPs. It is aligned with the Common Core State Standards and is designed to assess mastery of those standards in schools statewide. In order to be eligible for federal funding, we are required to have our students take the SBAC. The test was developed between 2010 and 2014 and was administered this past spring for the first time.

The SBAC is adaptive and includes multiple choice questions as well as open ended performance tasks.

How should it be used? Since the SBAC provides statewide data (in the 18 states administering SBAC), it can inform state and district education policy. The test is also intended to be used to communicate with caregivers about a child's academic progress. Additionally, it is intended to inform instruction at a systems level.

However, as this is the first year we have used the SBAC, there is much that is unclear about how this year's results can or should be used by each of these stakeholders. There is a WCSU SBAC Site that you can visit for more information.

On October 19, Bill, Jen and building administrators are attending the first regional workshop on how to analyze and interpret results.  After that, they will be creating a plan for sharing the data with teachers and families.

How should it influence instruction? Teachers should familiarize themselves and their students with the online environment and take the training and practice tests. Teachers should not feel the need to veer from their instructional path to do "review for the SBAC" or use released items in isolation. However, if there is a released item that fits in well with a teacher's instructional path, it is fine to use it.

It is important to note that the SBAC is intended to target depth rather than breadth. So, the best preparation for the SBAC is to provide coherent instruction that focuses on getting all of our students to a point where they can apply our non-negotiable skills flexibly to multiple contexts and communicate their deep understanding of how and why that concept applies.

PNOA

What is it? PNOA stands for "Primary Number and Operations Assessment." It is a multipurpose assessment tool that was developed by Vermont educators for grades K-2.
The assessment is conducted through a one to one interview with a student.

How should it be used? It can be used as a universal screener, diagnostic assessment, summative assessment, and/or formative assessment. In WCSU, some schools use parts of the PNOA to screen kindergarten students in the fall and primary grade students that are new to the district. We are moving towards using it as a diagnostic assessment to take a closer look at students who are identified as having "some" or "high" risk using easyCBM. This assessment can help shed light on a student's strengths and weaknesses in relation to number concept and number sense.

Please note: If you have had PNOA training, you may have noticed that some items are not aligned with the instructional models that we use at WCSU (ten frames for example) see this blog post for more info: Loree and Mahesh don't completely ma-mesh...

Teachers of older students could use other skills inventories (such as this one) in a similar interview format. The important part is to use the interview to find out:

1. What do kids know? - Keep a running record to find out their thinking.
2. How do they apply and explain that knowledge?

How should it influence instruction? Data from the PNOA can be used to inform intervention design. It can also inform "core" instruction, by putting a focus on clear precise communication of math reasoning. Hey, that's math practice number 3, folks!

Pre-Assessments, Entry/Exit Cards, Running Records, Checks for Understanding, etc...

What are they? These all fall under that fuzzy umbrella of "formative assessment." These are the day to day assessments that we do to inform (and sometimes reform) our core instruction.

In a perfect world, we would already have a collection of common daily formative assessments across WCSU available. However...

We all can use the levels of math knowing to create our own formative assessments.  Of course, we can share these. For example, I'll share this one that can be adapted and used for any operation: take this blank assessment and fill in any section. Have students fill in the missing sections. I have used this with success in the past. It doesn't tell me everything, but it tells me quite a bit. It's a pain it at first (especially the last section), but soon enough students get used to the format.

How should they be used? They should be used systematically to make day by day and even minute by minute decisions about how to change "core" instruction. They should be meaningful (more purposeful than "bell work") and aligned with the non-negotiables and levels of knowing.

Almost every time I enter a classroom now, I find myself completely unaware of what students are ready for. So, I always start with an entry card, think prompt, or just something to give me a sense of where they are. For example, when I begin by asking students to get pull out random Cuisenaire rods, I can quickly scan. If there are students who are not automatic, I know that I'll need to do some tool building getting them more familiar with the lengths.

How should it influence instruction? Instruction without formative assessment is like bowling with a paper bag over your head. It influences the quality of our instruction a heck of a lot. 

Enough said.

Quizzes, Tests, Performance Tasks, etc...

What are they? These all are generally considered "outcomes/summative assessments" which should be assessments that help teachers evaluate and verify learning over time (after a "unit" or "chapter," for example).

Using our bowling analogy to explain it, the teacher is the bowler practicing and using formative assessment (all senses blazing) to get better at knocking down the pins. The apparatus that descends from the ceiling and shows us which pins were successfully knocked down and which remained standing is our summative assessment. It tells us something about our pins (might that one be glued to the floor?), but it also tells us about our bowling (why do we always seem to miss the one on the far right?). It may also tell us something about the assessment itself (is it really so simple as who was knocked down and who was standing?).

How should they be used? Traditionally, these assessments have been lumped together and averaged in a single category "Assessments" that constitutes some percent (50%) of an overall grade along with things like homework (25%) and class participation (25%).
Folks who have shifted to standards-based grading can use summative assessments to assess for mastery. They can do that in much the same way that a person advances in the martial arts where a person "tests" into a new belt that signifies a new level of mastery. There is no limit to the number of times a student can assess for mastery. The student uses feedback and time to develop the skills needed to re-test successfully.

Can you imagine how many of us would be taking the bus if we had only one shot at passing our driving test?

How should it influence instruction? According to the Vermont MTSS Field Guide, these assessments should aid teachers in planning future instruction, informing classroom decisions (i.e. use of groupings), evaluating curricular changes, and making wide decisions regarding curriculum and instruction. You will notice that there is no mention of grades or reporting in that description. We'll address that below.

Common local summative assessments should influence our instruction a lot. However, we are at the beginning stages of developing these assessments (just as we are with common formative assessments). The Math Steering Committee will be spending time this year developing common assessments that are aligned with the non-negotiables and the levels of math knowing. We identified this as one of our top priorities.

What does all of this mean for grading and reporting?

I knew you were going to put your foot on the third rail and ask that... It's an important question. The honest answer is that the more we dig into this one the more we realize that we have work to do as a supervisory union around defining the purpose of grades and reporting. There is a lot of variety in grading and reporting philosophies and practices. This is being looked at currently by the WCSU leadership team. Stay tuned...


Q: What is your take away? What questions do you still have? Please comment below...