Formative Assessment in the FOSS Program

By Kathy Long, Assessment Coordinator, FOSS Project, Lawrence Hall of Science

As Larry mentioned in his “Observations” article, FOSS developers have been thrashing around in assessment for at least the last six years. And the results of the thrashing are beginning to take some shape. Through the Assessing Science Knowledge project (ASK), with hundreds of teachers and thousands of students as research partners, we developed an assessment system that we initially saw as having two parts, one formative and the other summative. Embedded assessments, the formative part, would have a narrow focus and occur on a daily basis (whenever a lesson is taught). Benchmark assessments, the summative part, would have a broader focus and happen at the beginning and end of instruction, as is traditional in many curricula, and also occur after each investigation (proposed by the ASK teachers). In benchmark assessments, students would need to differentiate between the various concepts and principles they were learning as well as be able to apply these ideas in a variety of contexts.

As ASK evolved, science notebooks became a central part of the science inquiry endeavor. We quickly saw that notebook entries provided an excellent context for finding out how students were building their conceptual understanding. Students used the formative assessment prompts as a “writing-to-learn” tool, helping them organize their thoughts and find gaps in their understanding so they could ask questions. Teachers used them to gauge students’ understanding and to plan further instruction. In another project called Formative Assessment in Science through Technology (FAST), we came to the conclusion that students’ writing and thinking in their notebooks provided a more reliable way of looking at student understanding than can be gained through classroom observation and discussion. In fact, we developed a mantra: “If the kids can’t write about it, then they don’t know it well enough.”

It is difficult to use classroom observation and discussion to gauge student understanding for several reasons. First, as you move from group to group, it’s hard to get a handle on who knows what. Once one student has given you a correct answer, you have no real way to gauge the other students’ understanding. Second, in discussion, teachers typically call on a few students and if they are hearing the right things, they assume everyone “got it” and move along. If they hear wrong answers, they ask more questions to scaffold students to the correct understanding. All of these are things that teachers should absolutely do, but it doesn’t give the teacher an accurate picture of what all students are thinking, independent of their peers or the teacher. Further, it often results in teachers having an overly optimistic impression of what the students have learned. This is why specific notebook entries aimed at assessing understanding provide a better tool for gathering reliable evidence of learning. Students do the assessment work as part of the regular lesson, and teachers can look at their work after class, when there is time to read through their responses and thoughtfully reflect on what students are thinking.

“Are you kidding? After class?” you may be thinking. So were all of the teachers in ASK and FAST. They all commented that they got great information about students’ understanding from reading through the notebooks, but it was much too time-consuming. And they weren’t really using the information they got to adjust instruction. Working together we came up with what we now call the reflective assessment process.

Teachers told us they could spend 10—–15 minutes after class to look at student work, but no more. We agreed that they would spend ten minutes looking at student work, followed by five minutes to reflect on trends and patterns and to plan how to adjust instruction for the next lesson. Here are the key steps to make this process work.

• Anticipate what is going to be assessed before you begin the lesson. Keep the grand instructional goals and standards in mind (know the place where you are headed), then think about what pieces of knowledge and connections are important in the day’s lesson.
• Plan the specific assessment activity. FOSS has tried to make this easy for teachers by making a suggestion in each part of each investigation, then pointing out one or two things to look for in students’ work that will provide evidence about what they know and need help with. Keeping a narrow focus is crucial when it comes to the time factor.
• Engage the students in the lesson (the investigation)
• Review the students’ work. Students turn in their notebooks open to the page you will be looking at. This is a detail worth mentioning because it can take twice as long to find the page you’re looking for as it does to review a student’s writing. It’s also important to stick with the evidence (not to read anything into the response). If you have any doubt that students understand, assume they don’t fully understand so you can take steps to clarify or help them adjust their thinking.
• Adjust instruction as you move into the next investigation part to help students clarify any problems you saw in their conceptual understanding. If you maintain this practice throughout the module, you will need to make only small corrective adjustments along the way. This is much easier than having to make major corrections after multiple misconceptions have accumulated.

Through a synergistic collaboration between the FAST and ASK projects, we completed a very small study that suggests that the reflective assessment process can make a significant difference in students’ performance on summative tests. For further information about that study, which includes more detail about the process itself, see the FOSSweb research database (http://lhsfoss.org/scope/research/search.php) and this article, Reflective Assessment Technique (Kennedy, Long, and Camins, 2009).

As mentioned above, the goal was to create summative assessments that would be technically documented so that districts could use them for accountability measures (to supplement standardized tests). But an interesting thing happened as the ASK project proceeded. The benchmark assessments that followed each investigation became known as “I-Checks,” short for “I check my own understanding.” Teachers embraced the opportunity to check their students’ understanding more frequently (rather than waiting for the posttest), but we also found that there was a significant paradigm shift in the classroom culture when I-Checks were used formatively. In other words, students took the assessments, teachers reviewed the student responses, then returned the unscored, unmarked I-Checks to the students for further discussion and peer- and self-assessment activities. In classrooms that used this process to create a clear dialogue between teachers and students, not only did achievement improve, students’ attitudes towards assessment changed dramatically. Students no longer saw the assessments as a competition. Students were willing to help each other get the “right answers,” and they even began to ask their teachers if it was time to take another I-Check so they could check their understanding. How many times have your students told you they thought is was time to take another test?

Many may ask, “What about the grades we are required to give students?” This is a very good question that we don’t have a definitive answer for at this time. We are currently looking for examples from teachers who are using formative assessment and finding other ways to give grades. It seems from our perspective, the moment grades are given, the magic is gone. Students are no longer willing to think or even risk sharing their understanding because they’ve already been judged.

Matthew Wigdahl, a fifth-grade teacher in Eau Claire, Wisconsin, provides us with one example of what can happen in the classroom when grades are not the focus.

I look at the I-Checks before we begin an investigation, so I am sure that I have covered everything the students might need to know to complete the I-Check. When we complete an investigation, I hand out the I-Checks and give the students time to work on them in class. I tell them to work independently and do their very best to try to answer the questions. If students are stalled or confused, I ask them to write an answer using a language stem such as: “I tried doing ___ but it didn’t make sense.” Or, “I need to understand more about ___ to answer the question.” Or, “The most important thing I know about ___ is ___.”

When I-Checks are complete, we go over them in class, sharing ideas as scientists would. The students exchange I-Checks. I emphasize every time that we are not “correcting” the assignment like we used to. We do not mark the questions right or wrong with a big slash through the question! We are seeking to comprehend what a fellow scientist means by her or his answer, and whether she or he understands the question. If the answer is reasonable, and it is clear he or she understands the question, we leave it alone. If it appears he or she might not get it, we write something helpful to clarify his or her thinking. When students raise their hands to ask me if an answer is “right,” I ask them, “do you think that person understands the question?” Usually, they need no further clarification. Students return the I-Checks with no score orgrade, just helpful written notes about how to do better. We model and practice what it might look like to go over these notes for review or before a test.

Teachers often express the concern that if the I-Checks aren’t going to be graded, the students won’t take them seriously. Matthew has found the opposite to be true. In fact, he reports that his students take the process very seriously and discuss the concepts thoughtfully and in a mature way.

One thing some people have been skeptical of is how I might hold students accountable for incomplete or missing work if I don’t collect/correct the assigned I-Checks myself on the due date. Emphasizing understanding over the responsibility of being prepared (a separate skill to be taught in my opinion) has sharpened our focus on science. While there may be a few students who are not finished with their I-Checks on the day we go over them, it has been much less of a problem than in other subjects. I explain that they lost the opportunity to check their understanding but should still participate in the exchanging and discussion of the I-Checks. I believe this has helped shift the motivation for completing assignments from external consequences to an internal desire to learn.

Matthew has taken formative assessment to the core of his instructional practice. He has clearly transferred the responsibility for learning to his students and is using the assessments as a learning tool; students are responding in a very positive way. This is evidence that understanding and not grades can be a very motivating experience for students.

As Larry mentioned in his “Observations” article, formative assessment has become quite an industry. Unfortunately, many educational publishers have misinterpreted the essentials of formative assessment simply to create a product. At FOSS, we believe that formative assessment is the process that teachers and students use in order to enhance the dialogue among them to improve learning. It requires thoughtful planning and reflection, but the payoff is tremendous.

Reference

Kennedy, C., K. Long and A. Camins. December 2009. "The Reflective Assessment Technique: A new way of evaluating in-class student work." Science and Children, 50-53.