From Observation Fatigue to Purposeful Reporting
Amy Holcomb, the learning experiences manager at Skokie Public Library, oversees the management of three learning spaces and up to 20 staff members as well as the coordination of all public programming. With a youth space that changes activities weekly, a studio with numerous pieces of high-tech equipment, and 30+ public programs a week, it’s a lot to keep track of.
Initially, Amy intended to use the Observation Deck as a means of shift reporting for the three learning spaces—space for her team to record qualitative outcomes of the learning spaces, rather than just quantitative numbers of how many people spent how much time in a space. But the initial test was not promising.
After a week of completing observations for each shift, her team was already experiencing data fatigue. Summary responses were coming in blank or with rote information like “good shift,” “engaged with several patrons.” And having so many busy, part-time staff in 3 different spaces meant dramatically varying quality in observation notes.
However, there were glimmers of excitement. When staff had particularly wonderful interactions or stories to share, they now had a place to record them, tag them, and share them. Before, their main means of doing so was a running Slack channel.
Amy decided to change course. First, using staff feedback, she updated the Observation Deck framework to prioritize the observations staff were already reporting and eliminate redundancies across dimensions. Then, she worked with her team to re-test the new framework with the expectation that staff would report at least 1-2 observation sessions per week. This adjustment proved helpful to determine and then resolve pain points for staff, and resulted in the complete adoption of Observation Deck as the tool for shift reports.
Welcoming staff feedback on the tool and then applying that feedback led Amy to be able to use Observation Deck as initially intended. Experiential learning staff participated in a group training session during a day-long learning summit, where they were able to explore the tool together and in a more playful way. This training allowed them to have a better understanding not only of how to use the tool as expected, but also the reasons the library adopted the tool in the first place—including incorporating staff observations in service design.
One of Skokie Public Library’s strengths is its dedication to community engagement and well-defined, timely strategic plans.
Their reporting structure for strategic plan updates is robust, with quarterly reports directly tied to goals. However, the reports center almost entirely on summary narratives.
In January 2024 they reported:
"In conjunction with the exhibit by the Chicago Weaving School, 'Weaving Workshop: Make Your Own Wall Hanging' was one of the most well attended hands-on programs since prepandemic times."
Wouldn’t it be wonderful to connect stronger observable learning outcomes, both quantitative and qualitative, to a successful program? To highlight not only that it was well attended, but what participants took away from the experience?
Amy shifted her plans again. Rather than just using the tool for learning space shift reports, staff would utilize a general framework for a month-long period of program observations. Done periodically and strategically throughout the year, these observations would allow for a stronger year-end review of the outcomes from public programming as well as from each learning space.
Skokie understands the strength of Observation Deck as a space for staff members to record powerful stories, anecdotes, and moments when they arise—moving those stories and outcomes off that Slack channel or email thread and into an organized, sortable system.
Skokie’s Director of Public Services, Amy Koester, expressed the value of including stories and outcomes from Observation Desk in regular reports:
“It’s incredibly important that we can tell the stories of the value and impact of our work with and for the Skokie community. Being able to include not only quantitative usage data but also qualitative stories of the types of experiences that take place in our spaces—and from the perspectives of each of our excellent staff—means that the stories we tell to our Board of Directors and our community give a full-color picture of the vitality of experiential learning for all ages in our library and the village at large.”
Next steps for Skokie include training program team leads on Observation Deck so they can modify the existing general programs framework to be more specific to programs in their purview. For example, storytimes have different goals than a tech workshop, and staff are learning how to effectively use Observation Deck to capture stories connected to specific programmatic goals that are tied to strategic plan priorities. The goal is to have workgroups within Public Services use data collected through Observation Deck consistently represented in quarterly and annual reports.