The data were grouped into four representative themes after several rounds of analysis. These include (1) benefits of the paper puzzles, (2) thoughts and conversations occurring during paper puzzles, (3) group interactions, and (4) challenges associated with paper puzzles.

Benefits

Students described many benefits of the paper puzzle activities. This theme encompasses those described benefits, which were divided into four categories: provision of structure, promotion of learning, designation of high-yield concepts, and work with peers.

Provision of Structure

Students used the paper puzzle sessions to provide structure to their day-to-day learning: “I usually attend for structure mainly. I have a hard time figuring out when to fit everything in my day, when I don’t attend things that are synchronous. So I do the things that are synchronous when they’re scheduled, and then that helps me fit other things in around that,” (Focus Group 3). Participants also noted that the sessions provided motivation to prepare and stay on top of assigned material. As one student described, “the interactive sessions kind of motivate me to be more prepared. Whereas normally if I see a lecture, then … I’ll attend and … I’m not prepared for the information,” (FG1).

Promotion of Learning

Students appreciated the active learning component of the paper puzzles: “I like that they push you to generate answers rather than to listen passively,” (FG1). Some used them as a way to evaluate their own knowledge as well. One student stated, “they can be a good way to test general knowledge, like quiz yourself on general knowledge” (FG3). In addition, students enjoyed the novelty and variability from typical lecture formats: “I appreciate having a new way of trying to learn something and having it be different than a normal lecture. It’s more interactive,” (FG3).

Designation of High-Yield Concepts

Discovery of which concepts constitute high-yield, important material was another student-reported benefit of the paper-puzzle activities. Some students focused on the testability of certain concepts, as noted when this student described that they “learn more about what the question writer or whoever designed the puzzle, what they think is important. If there’s going to be just one small fact about something in the paper puzzle, it might be moderately an important takeaway. But if there’s going to be multiple puzzles, as you keep going about the same concept, the same like physiological concept or graph or something like that, then it kind of enforces in me more their priorities of our learning outcomes, as well as how to take a step-by-step to the problem solving,” (FG3). Other students emphasized the value of receiving input from clinician instructors during the activities: “I really appreciate the opportunity to talk to clinicians … I like the kind of real-life perspective,” (FG2).

Working with Peers

Though students felt that the paper puzzle experience varied depending on their assigned group, they also stated that the chance to work and interact with peers was a benefit of the paper puzzle activities. Students found value in being able to verbalize their thought process and talk through problems, as expressed by this student: “I really liked the chance to talk about it … when I can verbalize what I understand and somebody can tell me, yes, that’s right or, well, I think it’s this way,” (FG1). Students expressed satisfaction in being able to learn from their peers as well: “I learn a lot from how students process things too, so I like to listen to that,” (FG3).

Thoughts and Conversations

As students reflected on thoughts and conversations they had during completion of the paper puzzle activities, their reported internal and external dialogue generally fell into one of two categories: action and process-oriented thoughts and conversations, or metacognitive thoughts and conversations.

Action and Process-Oriented

The action and process-oriented dialogue did not include a lot of internal analysis while completing the task. For example, one student reported, “I’ve never thought about what I’m thinking about while doing [the paper puzzles],” (FG3). Similarly, another student described that they just started moving pieces instead of reflecting on why and where they were sorting things: “I see the big jumble and I’m like, ‘Okay, let’s start separating these by texts, by picture, by function.’ And then once you get them all separated to start match them,” (FG2). These descriptions were often accompanied by acknowledgments by the students of the lack of intentional deeper thought unless specifically prompted: “Usually, we don’t go any deeper than just like being like, okay, I think this goes here, I think this goes here, unless someone asks a question,” (FG1).

Metacognitive

Other times, students described their thoughts and conversations as being metacognitive, more aware of their thought process and analysis. These more analytic responses appeared in several contexts, one of which was comparison of self to peers: “Usually if I’m in-person working on [the paper puzzles] with people, I’m thinking about how much I need to study to catch up. I’m … comparing myself to everything that everybody else knows” (FG2). Another was internal comparison of current knowledge level to the level of knowledge required by the assigned task; or use of the paper puzzle task to assess their own current knowledge. As one student described, “I use [the paper puzzles] as a way to test my learning, but also I do want to see … what do I need to review afterwards? And … even with their explanation … after I struggled through it … did I … get the right concept? Do I need to review this concept? So they’re kind of like a way to test my knowledge,” (FG1). The last group of metacognitive dialogue examples involved making connections between the paper puzzles and the greater context of related topics. Students reported asking themselves, “then what does it matter, as in how does this relate to the system we’re learning overall?” (FG3).

Interactions

The theme of interactions encapsulates the students’ reported perception of group dynamics during paper puzzle activities. These descriptions of interactions can be grouped into three categories: turn-taking, resolution of contradictory ideas, and effects of group dynamics on learning.

Turn-Taking

Students described that groups tended toward turn-taking while completing the activities: “it’s often just a process of … ‘I’ll take the first pair and then you pair them off and then pass it off to somebody else,’” (FG3).

Resolution of Contradictory Ideas

If discrepancies arose, groups worked to resolve any contradictory ideas. One student reported, “If [group members] do disagree, then the conversation does kind of center around why they disagree or why we disagree and what’s going on,” (FG2). If any group members struggled to understand a topic, participants described situations in which this was able to be resolved: “If someone doesn’t understand, it has led to some good like group discussions. And I think everyone is … pretty happy to explain what they know … it helps the speaker understand better too, when that happens,” (FG1).

Effects of Group Dynamics on Learning

Participants also emphasized that the paper puzzle experience is heavily influenced by group dynamics, and that commonly “conversation does get dominated by maybe one or two people” (FG2) or some group members struggle to participate. For example, “If somebody comes in and they know [the material] really well … and we don’t collaborate and they just totally arrange everything, it’s not really a learning opportunity. And then on the other hand, if you show up and the group is unprepared, then you spend a lot of time looking at things and just saying, ‘Well, I don’t know,’” (FG3). As outlined by this example, both of these different dynamics can impair learning.

Challenges

The theme of challenges captures difficulties and negative perceptions that students had while completing the activities. These challenges are divided into five categories, including inefficiency, feelings of chaos and confusion, desire not to learn the wrong thing, imbalance between timing and complexity, and decreased utility without preparation.

Inefficiency

Many students described frustration with the inefficiency of the paper puzzle activities. As one student reported, “paper puzzles take up a lot more time than I would like them to. And sometimes I feel like the application it’s good … to get a basis and it might be helpful for a quick review, but I feel like I learn a lot more with clinical cases and studies,” (FG3).

Feelings of Chaos and Confusion

Students commonly reported feeling lost and confused during the activities, labelling them as “chaotic,” (FG1, FG3) and saying, “there [are] too many details that are very easy to get mixed up, and if you get mixed up, you aren’t learning anything,” (FG3).

Desire Not to Learn the Wrong Thing

Participants described a fear of learning incorrect information while completing paper puzzles: “I ascribe more to this idea of practice makes permanent, not practice makes perfect. And so, if you do something incorrectly on a paper puzzle, you might remember it that way because it’s so interactive rather than the correct way. And even when [the instructor] goes through the answers, there’s so much going on, on the slides, so many different moving pieces, that it is very difficult to figure out which piece you got wrong, and identify the best way to remediate that mistake,” (FG3). As a result of this fear, students describe a difficulty engaging in the content: “There [are] times where I don’t fully know the information. And so when we’re doing the paper puzzle, it, it kind of feels like I’m learning it wrong or I don’t trust what I’m learning. And that makes it hard it kind of makes me like disconnect from what I’m looking at, because I don’t want to retain it if it’s wrong and then have it forever solidified in my mind … in a bad way,” (FG1).

Imbalance Between Timing and Complexity

Variability and imbalance between time allotted for and complexity of the paper puzzle tasks also created some challenges for students. One student expressed, “there are some paper puzzles that really, you can finish in that one or two minutes, but there’s some where I felt like I wish I had more time to really walk through it and then kind of dig into the concepts that are confusing with them,” (FG3).

Decreased Utility Without Preparation

If students did not adequately prepare, they did not feel that the paper puzzle activities were useful: “A lot depends on if I’ve done the pre-work because I feel like I don’t get much out of it [the paper puzzle] if I haven’t prepared properly,” (FG1).