From Eyeballing to Evidence: Assessing for Learning in Hawaii Library Media Centers

The development of "library evidence folders" for assessment purposes was featured in a previous SLMAM article, "Building Evidence Folders for Learning through Library Media Centers" (Harada, November 2006). The project developers challenged participants to change their mindsets from "eyeballing student performance" to designing and implementing measurable evidence of student learning. These developers were part of a collaborative yearlong project that brought three organizations together: Hawaii Association of School Librarians (HASL), the Hawaii Department of Educations School Library Services (SLS), and the University of Hawaii's Library and Information Science (LIS) Program.

Since the publication of that article and the Fall Forum that focused on assessment, various state library leaders have contacted the Hawaii project developers expressing their desire to conduct similar assessment training in their own regions. This article will focus on this growing and heartening interest in assessment as central to student learning.

To briefly recap, the Hawaii initiative began with a weeklong summer institute and continued into the fall and spring in the form of individual consultations and online communication with a pilot group of twenty-four K- media specialists. The goal was to create more effective instruction that targeted key learning goals in the schools. This follow-up article summarizes critical lessons learned, accomplishments achieved, and continuing work identified by the Hawaii group.

Lessons Learned

Building rigor. The library media specialists were advised to take a "strategic approach" to assessment. This meant identifying lessons that directly connected with learning gaps in their respective schools. For all of them, this involved standards that students were not achieving in the state tests. It was not surprising, therefore, that they selected information-related lessons that dealt with the language arts standards since youngsters were having problems in various aspects of reading and writing. However, a pivotal insight for the library media specialists was that the lessons they identified were not the "conventional library lessons." For example, these were not the lessons on parts of the book at the elementary level or the one-shot orientation sessions at the secondary level.

As a group, they began to understand that instruction must challenge students to strengthen skills in higher order thinking and knowledge construction. In short, even before they wrestled with the issue of assessment, they had to tackle the issue of what was worth learning. As one library media specialist noted:

It might have been obvious to my colleagues, but it was a huge aha discovery for me that so many of my existing lessons were really superficial. As an elementary librarian, I was spending a lot of my instructional time on scavenger hunts in the library and craft-making sessions based on books I read to the students. While the kids had fun and loved the time they spent with me in the library, I realized that I wasn't really addressing deeper learning issues such as how to help them generate meaningful questions as they read literature or how to make intelligent inferences from a text.

Table 1 provides examples of the shifts in teaching emphases of the Hawaii group based on new understandings about the learning needs of students.

Planning with the end in mind.

Project planners emphasized the importance of starting with clearly stated learning targets and designing performance tasks that facilitated students achieving these targets. They also stressed that the criteria to assess whether or not students reached these goals had to be aligned with the learning targets. Importantly, both targets and assessment criteria were written in language that the students understood. The library media specialists in the group admitted that they had not previously designed lessons in this way. One of them confided:

I have always started by hunting for lesson ideas from other librarians and from the Internet. I hate to admit this but I never started with a clearly stated target. I sort of intuitively knew why I was doing a lesson but I didn't spend much time actually thinking about this. And I only did informal assessment... you know, observing the class as they worked. Working "backward" was a tremendous design challenge for the group. After re-thinking a particular lesson, a library media specialist remarked: This is giving my brain a real workout!! After yesterday's session, I went home and kept thinking about my lesson. I began to realize that the reason why this particular lesson had never worked the way I wanted was because I had not done this kind of backward mapping. I had always tagged on my goal after I had my lesson idea in place. It's no wonder that the lesson itself did not match the goal! In addition, I did not assess what the students did except by eyeballing the group. To facilitate planning in this backward fashion, a lesson template was devised that put standards, goals, and assessments before the lesson procedure itself (see Table 2).

Library media specialists paid special attention to numbers 5, 6, 7, and the template: learning target, assessment criteria, performance task, and the tool used to assess student performance.

Without clear alignments of these critical instructional elements, they encountered problems in actually implementing their lessons. For example, one library media specialist found that her pretest of the students' ability to analyze a literary character did not actually measure one of her critical learning targets of being able to infer personality traits. By reflecting on what was happening, she was able to modify the assessment tool before administering it to a second class.

Involving students in assessment. While the library media specialists recognized the importance of students participating in the assessment process, they frequently struggled with when and how to involve the students. They raised pragmatic questions such as: Can students as young as kindergarten and first grade be involved? When do I find the extra time to let the students do this? Can they accurately assess themselves? As the school year progressed, the library media specialists gradually uncovered the following answers to these types of questions:

Age of students. An elementary library media specialist teamed with one of her kindergarten teachers to see what the youngsters understood about "finding information." They decided to have the students create a wallsized concept map to assess student knowledge. At the beginning of their mini-research project, the students simply knew that a person had questions and looked in books to find the answers. By the end of the project, however, the concept map included the following critical components: thinking about what is already known, looking in more places (not just books), not making up the information, and sharing the information with other people. By creating the concept map, the students could see their own progress. They were particularly excited about showing their parents what they had learned during the parent conference session.

Time to assess. Library media specialists found that involving students in assessment did not have to take an inordinate amount of "extra" time. Some of them used simple checklists that students completed as they worked on a particular activity. This kept the students focused and on task. Several of them also experimented with exit passes. Depending on the learning target, students were given slips of paper (passes) with spaces for their names, dates, and one question connected to the learning target. For example, following a lesson on main ideas, the question on the exit pass was: Can you describe one way to identify a main idea in an article? Students could quickly write a response on their passes and share them as they exited the library media center.

Accuracy of student assessment. Everyone discovered that learning how to accurately assess ones own progress was in itself a crucial learning process. It did not matter whether the library media specialists used surveys, rubrics, or other forms of assessment. At the beginning, students frequently over-rated themselves. Both the teachers and library media specialists began to realize that they had to follow up with class debriefing sessions and conferences to facilitate student growth in this area. One library media specialist noted that "we had to go over how to use the rubric, criterion by criterion, and we had to have samples of student work for the kids to begin understanding how to look at their own work with a more discriminating eye."

Starting small. At the beginning of the Hawaii project, almost all of the library media specialists indicated they wanted to build assessment into major portions of their program, i.e., incorporate formal assessment measures across all grade levels or with entire departments. They were advised to start with a workable piece of their larger goals. For some, this meant targeting just one teacher. For others, it meant focusing on a single skill or concept that aligned with classroom teaching priorities. As one library media specialist reflected at the end of the year:

At first, I wanted to do something with all of the freshmen students in my high school. As I thought about this and learned more about what would be involved in assessing their work, I realized that I was being overly ambitious in this first year. You know what I mean—I had to get my toe in the water before diving in the pool! So I decided to work on one particular skill that the teachers wanted taught and started my assessment work with two of the teachers that I knew would follow through in their classrooms. By doing this, I made things doable for myself. This also made it possible for me to succeed without killing myself in the process! Being resourceful collaborators.

Lack of time to meet with teachers was a persistent challenge for all the library media specialists. At the same time, they empathized with the classroom teachers who felt the pressures of meeting the mandates of No Child Left Behind. While they squeezed joint meetings into wedges of time before and after school, some of the library media specialists were fortunate enough to meet with teachers during the teachers' prep time. In most cases, however, the library media specialists had to be resourceful and flexible. Many of them found email a valuable means of discussing lessons. One library media specialist devised an online lesson template that became a handy tool for asynchronous discussions with her teachers. In many of the schools, teachers were devising curriculum maps across grade levels and within departments. These maps were valuable documents for the library media specialists. They could use them as initial indicators of classroom learning targets and begin to plan ways to help busy and stressed-out teachers meet these targets.

Achievements Accomplished

Equal partners in teaching and learning. By working on assessment as an essential component of their work with students, the library media specialists discovered that teachers and administrators began viewing the library media specialists efforts with different eyes. The author uses pseudonyms below but the examples are real ones:

Jane devised a simple observation checklist for a lesson that she conducted on understanding the sequence of events in a biography. While Jane was teaching the class, she asked the teacher if she could help out by using the checklist to assess the students. The teacher agreed to do this. In the process, she realized that Jane's checklist was a valuable tool for measuring individual student progress and that working as a team had mutual benefits. The teacher took the assessment data back to class and used it to help students who had encountered difficulties during the lesson.

Lorna's teachers were very concerned about the students' inability to write adequate reader responses to literary pieces. Knowing this, Lorna volunteered to reinforce reader response skills through her instruction in the library media center. First, however, she conducted a pre-assessment of what students could already do by having them submit samples of previous responses. Based on her analysis of their work, she helped the teachers identify specific weaknesses in the students' samples. As a team, they devised criteria checklists that helped the youngsters focus on elements of a successful reader response. Together, they also introduced and reinforced thinking and writing skills students needed in a yearlong effort toward improvements.

Velma devised a learning matrix that displayed how the lessons she taught in the library media center aligned with the standards being emphasized in the classrooms. She included examples of lesson plans and her fledgling efforts to assess for learning in a few of the lessons. She shared her products with both the administrator and the faculty. The teachers were impressed and asked Velma if they might include her lesson plans and assessments in their class portfolios as evidence of learning that extended beyond the classroom.

Darlene was already working with a team of high school teachers on research projects when she joined the pilot group. She was motivated to do this because "assessment was the missing piece in our high school projects." Throughout the school year, she shared her work in assessment with the other members of her high school team. She is currently serving on a special task force to devise a "skills progression matrix" for the senior projects at her high school.

When Carol joined the Hawaii pilot group last year, she was nervous and apprehensive about her ability to devise and implement learning assessments. She started small by tackling a specific skill on character analysis in literature. Her teachers were pleased with the assessment results, and this emboldened Carol to work with more classes and teachers. When the principal invited her to join a school team focusing on student assessment for the upcoming school year, she said, "I was ready!"

Emerging leaders. The ability to assess for learning is more a mantra than a reality in many schools; teachers know they should be doing it but they are not sure how best to do it. When library media specialists have training in this area, they can emerge as potential teacher leaders. Here are examples from the pilot group:

At one high school where every teacher and resource specialist is expected to produce an evidence folder, the library team members serve as mentors to faculty members who might need additional support.

An elementary library media specialist was selected by the faculty and administration as a member of the school team that attended a national assessment conference in Oregon.

Another elementary library media specialist was recommended by her principal to join a state cohort training program for promising teacher leaders.

A high school library media specialist has taken the initiative to organize a study group on student assessment in her own complex.

Continuing Work

Pursuing professional inquiry. The library media specialists involved in this project realize that their work as instructional designers is just beginning. They continue to wrestle with the alignment of learning targets, assessment criteria, and tools for assessment. As one of them noted, "This is much harder than I imagined. I now realize how easy it is to use vague terms and have only the muddiest idea about what you actually mean!"

They still struggle to craft precise and measurable criteria in assessing how well students actually perform. As they work on more lessons, they find themselves asking increasingly complex questions such as: What am I actually trying to measure? Are there other ways to assess students' achievement of this target? How do I design this experience so that it facilitates the learning I want to see happen? What does the assessment data really tell me? Caught up in this type of inquiry, one of the library media specialists reflected:

I started off with assessment but now I realize that I need to know much more about ALL elements of teaching that results in real learning. I guess I didn't know how much I didn't know! There are all sorts of questions I am now asking myself. This is scary and exciting at the same time.

Collaboration at the state level. Through our work on this project, the developers have realized the importance of embracing a shared vision and collaborating to achieve that vision. Follow-up efforts on this initial work have included the following:

Conducting a summer one-day clinic in the pilot group. This was an opportunity to reunite members of the group and reflect on lessons learned as well as to share next steps.

Mounting a second summer institute in a new cohort of library media specialists. While the author coordinated the original institute, two HASL members took the lead this year. They also involved members of the pilot group to join the sessions and serve as informal mentors. The results were hugely positive.

Collaborating on a first-time effort to produce a multi-year plan for professional development that unites the three organizations in a comprehensive effort to strengthen library media specialists as important teaching partners in 21st century schools. The SLS unit is coordinating this initiative and involving HASL and LIS to leverage the resources and talents from these groups.

Conclusion

The Confucian adage, "The journey of a thousand miles begins with a single step," fittingly captures the initiative described in this article. The Hawaii project marks the beginning of a journey that has not been without its hurdles and stumbling blocks. Importantly, however, the rich lessons that are learned along the way hold immeasurable promise for students to succeed as critical learners. At the same time, these experiences are also raising expectations to excel as model teachers who are also lifelong learners.

Cited Reference

Harada, V H. "Building Evidence Folders for Learning through Library Media Centers." School Library Media Activities Monthly 23, no. 3 (November 2006): 25-30.

Recommended Readings

Harada, V. H."Working Smarter: Being Strategic about Assessment and Accountability." Teacher Librarian 33, no. 1 (October 2005): 8-15.

Harada, V. H., and J. M. Yoshina. / Assessing Learning: Librarians and Teachers as Partners. Greenwood/Libraries Unlimited, 2005.

Harada, V. H., L. Kam, and L Marks."School Librarians Help Students Achieve: Here's the Evidence!" MultiMedia &lnternet@Schools 14, no. 2 (March/April 2007): 25-28.

Stiggins, R.J. Student-Involved Assessment for Learning. 4th ed. Pearson/Merrill Prentice Hall, 2005.

Todd, R.J."School Libraries and Evidence: Seize the Day, Begin the Future." Library Media Connection 22, no. 1 (August 2003): 12-18.

Wiggins, G., and J. McTighe. Understanding by Design. Association for Supervision and Curriculum Development, 1998.

Acknowledgments

The author expresses special gratitude to the other members of the Hawaii team for their vision and leadership in this project.

From the Hawaii Association of School Librarians: Linda Marks, 2007 precedent, and Lynette Kam, 2007 vice president in charge of programming.

From the Hawaii Department of Education's School Library Services: Donna Shiroma, unit director, and Hilary Apana-McKee, specialist.