Collecting and Analyzing Observation of Music Teaching and Learning Data (2024)

Humans are born observers—observation is a primary tool that permits us to learn about and make sense of the world around us. However, if we noticed every detail in our environment, we would quickly become overwhelmed. Instead, our brains simplify routine visual information and direct our attention to more important or meaningful features. With practice, we can become discriminating observers. For example, a collector of Dresden china will observe details in a flea market display that others pass by. A music teacher will notice details about the behavior and musical performance of students in his classroom that a parent volunteer might not see. A cardiologist will see more than squiggles in an EKG report. These people have developed what Eisner (1991) calls “connoisseurship” of a particular facet of human activity, well-honed perceptual skills that allow them to observe more detail in and attribute greater meaning to what they observe than one who has a mere acquaintance with that activity.

John Dewey (1934) distinguishes between two modes of observing: recognition, or “bare identification,” and perception, where we “begin to study and to ‘take in’” what we observe (52–53). Such perception is the goal of qualitative observation. Adler and Adler (1994) suggest,

Collecting and analyzing observation data relies on the researcher’s connoisseurship of the research setting, the willingness to observe “regularly and repeatedly,” and the ability to perceive details, “taking them in” in order to make meaning from them through analysis. In this chapter, I discuss nine aspects of observational research: preparing for observations, determining the researcher’s role, planning to enter and leave a site, processes of observation, collecting data, maintaining a data record, analyzing data, writing the research report, and ethical considerations in working with observation data.

13.1 Preparing for Observations

Qualitative studies that rely solely on observation data are rare; however, nearly all researchers collect data through observation. Even a study that draws almost completely on interview data involves some observation of interviewees’ body language, verbal pauses, and tone of voice. Observation data are often used to corroborate or triangulate data from multiple other sources. For example, by comparing participants’ observed actions with their perspectives as revealed through interviews, written documents, or informal conversations, a researcher can learn a great deal about unnoticed, implied, or unvoiced rules or relationships within the setting under study. For example, Norgaard (2008) observed seven professional jazz musicians as they improvised a blues form. By comparing his observations of their performances with the comments they made as they listened to a recording of their own improvisations, Norgaard was able to identify common strategies they used to improvise.

A study’s research questions help determine whether observation offers a useful source of data and, if so, what might be observed to address those questions. Qualitative research methods texts offer a variety of perspectives on site and participant selection or “sampling” ( Glesne 1999 ; Stake 1995 ), including, for example, purposeful, intensity, criterion, critical case, convenience, or maximum variation sampling ( Creswell 2007 , 127). “The researcher should have a connoisseur’s appetite for the best persons, places, and occasions. ‘Best’ usually means those that can best help us understand” the study’s questions ( Stake 1995 , 56). A pilot study in the prospective or a similar setting may provide opportunities to identify potential benefits and challenges, as well as to practice and refine observation techniques. Selecting sites and participants often requires balancing different tradeoffs, including practical issues such as time and funding required and accessibility and availability of participants. Tobias (2010 , 89–98) provides extensive detail about his selection of research site and participants in a study of a high school songwriting class, describing the range of options he identified and his reasons for his final selection.

Gaining access to a desired research setting requires approval of procedures by the appropriate Institutional Review Board (IRB). In some cases, researchers make preliminary contact with a proposed site to determine feasibility of the study in that site before submitting a proposal to the IRB. In other cases, IRB approval precedes contacts with possible sites. Gaining access may be a relatively simple or an extremely complex process. For example, to collect observation data in a public school classroom, a researcher may need approval from both a university and a school district IRB. Sometimes the process begins with contacting official gatekeepers, such as a school principal, the administrator of a club, or the director of a community ensemble. Other times, a friend or acquaintance with access can introduce the researcher to the official gatekeepers. When a study involves only adults in everyday settings, an IRB may declare it “exempt,” allowing researchers to contact participants directly. For example, although Pellegrino’s (2010) study included classroom observation, because she focused her observations only on the teachers, the IRB did not require her to obtain permission from the school or the students’ parents.

Obtaining access usually involves collecting informed consent forms signed by participants or their guardians, and assent forms signed by minors under age eighteen or other vulnerable individuals. The researcher prepares an overview of the study’s purposes and methods in language participants can understand. Particular concerns that must be addressed include whether photos, student work, interviews, audio, or video recordings will be collected. Researchers must provide specific information about how data will be used, with appropriate guarantees of anonymity and other privacy protections for all participants. In some cases, the formal language required by IRBs in consent/assent forms may be off-putting to participants, and researchers may plan a verbal “translation,” speaking with participants in more familiar terms when they present the forms. The appendices and method chapters of many dissertations include examples of consent/assent forms and other IRB-approved documents.

Issues of access may require changes in research focus or design as the researcher proceeds with observations. Glesne (1999) writes,

For example, although Soto, Lum, and Campbell (2009) had planned to collect data from two school sites, they discovered more limited access than anticipated at one of the schools, and had to reassign that school to “an occasional rather than the central feature of the project” (339). They also had to revise their intentions to interview all the participants, as some “could not commit the time to sit-down interviews with one or more members of the research team” (340–41). Similarly, a researcher may intend to study interactions among students in a classroom, but several parents may not provide consent for their children to participate. In such cases, the researcher may need to reposition recording equipment to exclude those participants, eliminate some collected data from analysis, or revise the research questions.

13.2 Determining the Researcher’s Role

In each observation study, researchers create their role from an array of possibilities. These choices commonly involve identifying one’s place on the participant-observer continuum, using oneself as the primary research instrument, and establishing relationships with participants.

13.2.1 The Participant-Observer Continuum

As data-collectors, researchers function as both participants and observers in any research study. Spradley (1980) distinguishes five possible levels of participation as an observer. Non-participation is the rare case where the researcher simply observes, with participants sometimes unaware of being observed. For example, Duke and Simmons (2006) analyzed 25 hours of video-recorded studio lessons taught by three artist-teachers, and described the teaching strategies they observed.

Passive participation involves minimal interaction with the participants, as when the researcher sits in a corner of a classroom taking notes. Studies of young children’s musical activities often rely primarily on observation of the children’s music-making on the playground or other settings, with minimal interaction between children and researchers (e.g., Jackson-Gough 2003 ; Lum and Campbell 2007 ; Stauffer 2002 ).

An example of moderate participation is studying interactions in a third-grade music class, where the researcher determines ahead of time to chat with children and answer their questions before and after class, but also remains on the sidelines unless approached while the class is formally in session. Seddon and Biasutti (2010) video recorded three college students using a text in an asynchronous e-learning environment to create 12-bar blues; only when participants requested assistance did the researchers provide support.

Spradley’s fourth level of involvement, active participation, is when the researcher “seeks to do what other people are doing, not merely to gain acceptance, but to more fully learn the cultural rules for behavior” (60). Manes (2009) took shamisen lessons himself, observed other students’ lessons, and interviewed the teacher to study traditional Japanese teaching strategies and playing techniques.

Complete participation involves the researcher in a setting in which he or she is a full participant, as in an action research study of one’s own classroom, or an examination of the implicit workings of an ensemble of which one is a member. Gackle ( Gackle and Fung 2009 ) studied her choir’s experience learning traditional Chinese songs in a community youth choir organization of which she was founder, artistic director, and conductor of the choir. She and co-author Fung together analyzed videos of her rehearsals. Most often, researchers function as moderate or active participants, becoming directly involved in some degree, as determined by both the research questions and the nature of what is being observed.

13.2.2 Researcher as Research Instrument

In qualitative research, researchers themselves are the primary research instruments; they cannot “completely disappear [as] a distinct person” ( Angrosino 2005 ). Peshkin (1988) defines this researcher subjectivity as “an amalgam of the persuasions that stem from the circ*mstances of one’s class, statuses, and values interacting with the particulars of one’s object of investigation” (17). The researcher’s own positionality, of having particular experiences of gender, class, ethnicity, profession, and place, may provide valuable insight into the situation under study. However, it may just as easily create blinders or encourage “selective inattention, tuning out, not seeing, and not hearing” ( Spradley 1980 , 55). Researchers are obligated to account for both the benefits and limitations of their subjectivity. For example, Tsugawa’s (2009) 25 years of band and orchestra teaching experience allowed him to quickly understand the rehearsal processes of a senior adult band and orchestra. However, in reviewing his observation notes, Tsugawa noticed he was critiquing the participants’ performance skills as if he were the teacher. He realized that, to better understand the participants’ experiences, he needed to move beyond his prior experience and “change role by removing [his] teacher-adjudicator glasses and putting on [his] participant-observer glasses” (56–57).

Members of a research team can challenge each other’s selective vision (e.g., Soto, Lum, and Campbell 2009 ), while individual researchers can invite peer review of their work to help them eliminate blind spots in collecting and analyzing data. Creswell’s (2007) Chapter 10 provides one overview of multiple “validation strategies” for qualitative research. He outlines eight procedures that are commonly accepted as standards to establish credibility and account for researcher subjectivity. These include prolonged engagement and observation in the study site, triangulation of multiple data sources, peer review, negative case analysis, clarification of researcher bias, member checks, “rich, thick description,” and external audits (207–09). Creswell recommends that researchers plan to employ at least two of these strategies.

13.2.3 Relationships with Participants

Atkinson and Hammersley (1994) argue that “we cannot study the social world without being part of it;” therefore they suggest “that participant observation is not a particular research technique but a mode of being-in-the-world characteristic of researchers” (249). Deciding on this “mode of being-in-the-world” for a study is less straightforward than it may seem. In the ways researchers design a study and collect, analyze, and report observation data, they convey their understandings of their relationship with participants (e.g., Angrosino and Rosenberg 2011 ; Atkinson and Hammersley 1994 ; Flinders and Richardson 2002 ; Spradley 1980 ). Relationships can range from an authoritative Researcher positioning participants as Others to a view of “observer and observed as inhabitants of a shared social and cultural field, their respective cultures different but equal, and capable of mutual recognition by virtue of a shared humanity” ( Atkinson and Hammersley 1994 , 256).

Spradley (1980) suggests that much useful information arises as researchers position themselves as learners, allowing participants to serve as teachers and experts: “Fieldwork, then, involves the disciplined study of what the world is like to people who have learned to see, hear, speak, think, and act in ways that are different. Rather than studying people, [participant observation] means learning from people” (3). The music education researchers cited in this chapter describe ways they assured participants that they wanted to learn their perspectives. For example, Abramo (2009) studied his own high school students rehearsing in rock bands. Concluding that “the social dynamic of ‘outnumbering the adult’ would yield more honest answers than the asymmetrical power relationship of teacher/student” (120), he sought to empower the students by conducting the study in spaces where they normally “hung out,” providing food, using their language, and allowing their friends to be in the room during individual interviews.

13.3 Collecting Observation Data: Entering and Leaving a Site

Stake (1995) points out, “Almost always, data gathering is done on somebody’s ‘home grounds.’…[The] burden on the host should be acknowledged. The researcher may be delightful company, but hosting delightful company is a burden” (57–58). In beginning an observational study, researchers will want to plan carefully how they will enter those “home grounds” in a respectful way. Will the researcher suddenly appear, or will a collaborator already familiar to the participants prepare them for the researcher’s coming? Will the researcher begin formal observations immediately or start with a period of “informal hanging out” ( Jansen and Peshkin 1992 , 709)? How will the study be explained to the participants? What will they want to know about the researcher’s personal background and interests?

Answers to these and similar questions will depend on variables such as the study’s purposes, the researcher’s familiarity with the site or participants, the researcher’s anticipated role on the participant-observer spectrum, and the type of relationship the researcher intends to establish with the participants. No matter the approach chosen, the goal is to bring the researcher into the setting in ways that promote his or her ability to eventually become a part of the setting, so that participants act much as they normally would were the researcher not present.

To blend in, researchers aim to establish trust with participants. Rapport may be created in a variety of ways, as fits the purpose of the study. Sometimes, the researcher is already familiar with the research site and participants. In her dissertation, Lum (2007 , 18–27) provides a detailed explanation of selecting and entering an elementary school in her native Singapore, where she had taught for five years and had already earned a high level of trust. When researchers have no prior relationship with the site or participants, they will need to plan ways to build trust. Tobias (2010 , 115–19) describes his theoretical goals and internal debates as he negotiated his relationships with high school students and their teacher. Koops (2010) wanted to study the learning processes of children in The Gambia. In the year prior to the study, she learned the Wolof language. While in The Gambia, she lived closely with the children and their families, and found that her limited knowledge of Wolof was helpful in establishing rapport, allowing her “to greet people, bargain at the market, interact with children, and explain [her] research” (23).

The credibility of both the researcher and the study may be enhanced when relationships with participants are consciously negotiated. In his study of a middle-school technology classroom, Ruthmann (2006 , 73–77) describes his progression from being a stranger in the classroom to becoming a “collaborative participant” as students began to seek his advice on their compositions, and the teacher asked his ideas about curricular, technological, and pedagogical issues that arose in her work. Alert researchers use their intuition as a guide to identify and resolve challenges that arise in establishing and maintaining rapport, as Blair (2009) did in studying her colleague Sue’s classroom:

Blending in, however, can become a potential concern in leaving a site at the conclusion of the study. The researcher often comes to feel a part of the group, and may be reluctant to sever ties. If a close relationship has developed, or if the researcher has been providing assistance to them, participants may feel abandoned once the researcher ceases to be part of the setting. It may be helpful to keep open the option to return, in case additional observation is needed to clarify questions that arise during later analysis. If appropriate, researchers can gradually withdraw by coming less frequently until visits cease ( Bogdan and Biklen 2003 ). Stake (1995) recommends that researchers use good judgment, common sense, and good manners to respectfully enter and leave a research site, having “made no one less able to carry out their responsibilities” during or at the conclusion of the study (60).

13.4 Collecting Observation Data: How to Observe

Initially, novice researchers may experience unsettling feelings of not knowing how or what to observe. With practice, they can develop the skills to notice key features relevant to a study’s purpose. Stake (1995) suggests that effective observation requires both sensitivity and skepticism: the sensitivity to remain alert for potentially useful information, and the skepticism to constantly question whether other important things are being overlooked. The skilled observer seeks both to “make the obvious obvious” ( Wolcott 1992 , 24) and to “make the familiar strange” ( Spindler 1982 , 313). The most effective researchers constantly search for alternative perspectives and explanations, being slow to conclude that they have observed the whole picture.

Skilled researchers learn to determine clear foci for each observation; a large quantity of unfocused observation notes might actually needlessly complicate data analysis, by obscuring the information most pertinent to the study’s research questions. Spradley (1980) describes three levels of observation. In a descriptive observation, researchers try to address the question, “What is going on here?” (73). At this level, Angrosino (2005) suggests observing as many details as possible, setting aside preconceptions and taking nothing for granted. This wide lens, he suggests, will yield a great deal of data. Although some of these data may later prove to be irrelevant, the discipline of detailed looking assures that important information is not too easily dismissed.

A second level is focused observation. Spradley (1980) suggests using five or six broad categories derived from the research questions to guide increasingly more focused observations. Key information begins to surface over repeated visits, as the researcher becomes more familiar with the setting and more aware of “what’s going on.”

Selective observation is a third level, where the researcher plans to observe for specific details. These foci change and evolve throughout the study, often based on questions that arise in reviewing data from previous observations. “Most researchers find they do their best work by being thoroughly prepared to concentrate on a few things, yet ready for unanticipated happenings that reveal the nature of the case” ( Stake 1995 , 55). Spradley’s (1980) book, Participant Observation, offers detailed guidance in maintaining the “balanced tension” (102) between big-picture and in-depth observation.

Rarely will a researcher begin with descriptive observation, proceed in order through the next two levels, and conclude data collection. It is also unusual for an observation study to close with the original research questions unchanged. With each return visit to the site, the researcher tacks back and forth among all three levels, as newly observed data raise questions that provoke a shift in observation focus or suggest refinement or revision of research questions. The method chapter of dissertations often provides details about the evolution of a study. For example, Lewis (2004 , 22–23) describes the gradual revision of her research questions. She intended to study whether kindergarten teachers made connections between music and other subject areas. As data collection proceeded, reading and literacy emerged as the key focus.

13.5 Collecting Observation Data: What to Collect

Observers typically collect multiple types of data. Field notes and audio or video recordings are common. Pertinent documents and artifacts, such as songbooks, photographs, music program brochures, or examples of students’ work, may be appropriate. In addition, the researcher maintains a journal throughout the study ( Creswell 2007 ). The more different kinds of data sources, the more credible the findings will be. Miles and Huberman (1994) suggest,

In this section, I discuss field notes, artifacts, preliminary data analysis, and the researcher journal.

13.5.1 Field Notes

Researchers make field notes in a variety of ways, depending on the circ*mstances of a particular study. Glesne (1999) , Stake (1995, 2010 ), Spradley (1980) , and Bogdan and Biklen (2003) are among the many texts that provide detailed examples of processes for making field notes. If the researcher’s role is primarily as observer, notes may be made by hand in a notebook or on a computer while observing. Sometimes, however, this may create a disturbance or discomfort for the participants, or researchers may be participating fully during an observation. For example, Tobias’s (2010) data gathering procedures in a high school songwriting class required considerable multitasking, keeping him too busy to make notes on-site beyond “in-process analytic writing such as asides, commentaries, and in-process memos.” After leaving the site each day, he reviewed the videos to make detailed descriptive notes (105, examples on 106–07).

Spradley (1980) notes a particular challenge related to field notes: “the moment you begin writing down what you see and hear, you automatically encode things in language…[and while] this may seem a rather straightforward matter, the language used in field notes has numerous long-range consequences for your research” (64). Especially important are the use of participants’ own language and concrete, descriptive terms. The most useful field notes record terms and phrases participants themselves use to label, describe, or discuss what the observer sees. Additionally, precise descriptions of a participant’s behaviors and actions will yield more useful data than simply noting that a child is “angry.” Evaluative descriptors, such as “wonderful,” “boring,” or “enthusiastic,” may record more about the researcher’s perspective than the views of the participants.

To separate their own feelings and interpretations from description of what is observed, researchers may find it helpful to record field notes on a divided page, using one side to describe events and the other side to record the researcher’s own thoughts or questions. Another option is writing researcher memos, entered directly in the field notes using a different font or other special notation, as Schmidt did (1994, 471–75). Berg’s (1997) Appendices H and I offer complete transcripts of her observations of two high school chamber ensembles’ rehearsals; Jackson-Gough’s (2003) Appendices D and E provide both raw observation notes and descriptive summaries of her video recordings.

13.5.2 Artifacts

For some studies, site diagrams, participant e-mails, concert programs, and other artifacts may provide invaluable information, while in others, they may be irrelevant. To record the contexts in which four urban music teachers worked, Fitzpatrick (2008) “drove around the school neighborhood, taking photographs of local housing, area businesses, and otherwise trying to document aspects of the local culture” (110). Soto, Lum, and Campbell (2009) , studying a school-university partnership, collected “class handouts, songs and song sheets, teaching schedules, and lesson plans,” as well as “the school’s annual calendar, local news coverage on the program, and articles published in the university magazines and faculty/staff weekly” (351), to triangulate observation and interview data. To supplement her classroom observations, Blair (2009) took photos of student group work completed on the classroom white board, reviewed student work saved to classroom computers, and archived e-mail conversations with the classroom teacher.

Occasionally, participants may not mention key documents and artifacts simply because they assume they are not important. A researcher’s ability to follow hunches and ask good questions can be useful in revealing or locating important artifacts. Stake (1995) suggests using logic, intuition, and careful observation, recommending that “one needs to have one’s mind organized, yet be open for unexpected clues” (68).

13.5.3 Researcher Journal

Because the researcher is the key research instrument, another important data source is the record of the observer’s own experience of conducting the study ( Bogdan and Biklen 2003 ; Creswell 2007 ; Stake 2010 ). The research journal “contain[s] a record of experiences, ideas, fears, mistakes, confusions, breakthroughs, and problems that arise during fieldwork. A journal represents the personal side of fieldwork; it includes reactions to informants and the feelings you sense from others” ( Spradley 1980 , 71).

Peshkin (1988) believes an awareness of feelings can alert researchers to important data. He writes,

Schmidt (1994) was surprised by the strong emotional reactions the student teachers in her study revealed as casual conversations after class evolved into discussions of major issues. She later made notes about these exchanges, and also recorded the tensions this created for her, finding that “keeping records of ‘friendly’ conversations reflected an attitude [she] did not want to convey: that the student teachers were ‘subjects’ for research” (42).

The journal also documents the evolution of research questions or data collection methods. Spradley (1980) explains that “rereading your journal at a later time will reveal how quickly you forget what occurred during the first days and weeks of fieldwork” (71). The journal becomes part of the data record, and can be used in creating the final report to inform readers about the researcher’s process, permitting readers to better interpret the study’s findings.

13.5.4 Preliminary Data Analysis

Novice researchers often fail to plan adequate time for review of data as it is collected. Although it is tempting to assume that recordings will preserve data adequately for analysis several months later, timely review of field notes and recordings are essential components of data collection and analysis. As Stake (1995) points out, “There is always too little time.…We need some deep thinking, perhaps a data-gathering plan, a plan that protects time for the less attractive work, such as writing up observations” (51).

Following each observation, researchers return to field notes to “thicken” them, reviewing recordings or mentally recreating the visit and filling in additional details to describe the people, setting, and events as thoroughly as possible. Memories are most vivid within the first 24 hours following an observation. Observation data will be most complete if researchers review field notes within that time frame, recalling their own sensations during the observation and adding information about participants’ actions or body language. As researchers note possible interpretations, questions, or foci for subsequent observations, this review process informs preliminary data analysis.

Ongoing preliminary analysis is also important in determining when enough data have been collected. Sometimes a study is limited by the amount of time granted for access, for example, when a music program’s schedule dictates the end. The ideal end of data collection occurs when data become “saturated” ( Adler and Adler 1994 ; Glesne 1999 ), i.e., when repeated visits yield redundant information or “diminishing returns” ( Bogdan and Biklen 2003 , 62). To help in deciding when data are saturated, Spradley (1980) recommends that occasionally “you need to climb a very tall tree and gain a broad perspective on how far you have come, what tasks lie ahead, and which direction you should take” to keep “from losing sight of the forest because of the trees” (35).

13.6 Maintaining a Data Record

Observational studies can generate a large volume of data, requiring the researcher to develop systematic and accessible methods for organizing it. Technology has made it possible to collect hundreds of photographs or hours of recordings. These data permit repeated examination and allow researchers to review an observation in more detail. However, the benefits of digital records should be weighed against the time, energy, and intrusiveness required to collect and transcribe the data. Wolcott (2001) warns,

Very early in the study, researchers need to develop systems for handling data, clearly labeling files with dates, participants’ names, locations, and other pertinent information. Some researchers like to work with an electronic archive of field notes; others prefer to maintain hard copy of field notes in binders. Recordings may be kept on a computer or transferred to other digital formats. Photographs and other documents may be digitally scanned or stored in paper files. Frequent backup of digital files can avoid catastrophe. (Tobias’s chapter 16 in this volume offers practical suggestions for data management.)

Markers within field notes that easily identify the location of data are particularly helpful during analysis and writing the final report. For example, labeling field notes with “4/12.O3A.6.23” could identify the date the notes were made (April 12), the observation (the third observation of classroom A), and the page (6) and line number (23) (e.g., Schmidt 1994 , 471–75). Most word processing systems will configure documents with this type of marker for easy reference. Similarly, a system for classifying and labeling recordings can simplify review and analysis. It is important to be aware that transcription can take five to eight hours per hour of recorded data; sometimes a paid transcriptionist can expedite the process, getting the basics on paper for the researcher to thicken in reviewing recordings. An indexed timeline entered in the corresponding transcript for each recording, marking major points or predetermined time intervals, will facilitate later analysis. For example, Hornbach’s (2005) Appendix E shows field notes using a timeline to note shifts in children’s musical activities.

13.7 Analyzing Observation Data

Preliminary analysis, described in Section 13.5.4 , continues from the first observation to the last. While preliminary analysis reveals some important information, once data collection is complete, formal analysis involves a more systematic search. Most approaches to analysis share a common understanding of the researcher as the primary research instrument, whose job is to describe and interpret what was observed. Through analysis, researchers seek to document the “complex meaning systems [people use] to organize their behavior, to understand themselves and others, and to make sense out of the world in which they live” ( Spradley 1980 , 5).

For novice researchers, analysis may seem mysterious or unsettling, with few easy-to-follow guidelines. Stake (1995) defends a view of data analysis as a largely intuitive, subjective process, learned through practice.

In this section, I discuss the analytical processes of developing and refining data codes.

13.7.1 Developing Data Codes

Formal analysis requires “perform[ing] some kinds of dissection, to see the parts separately and how they relate to each other” ( Stake 1995 , 72). Generally, researchers begin by reviewing all the collected data, rereading field notes and documents, listening to or watching recordings, and making notes about larger patterns, themes, or categories that emerge. In this initial review, it is important to keep an open mind, looking not only to confirm ideas that have arisen during preliminary analysis, but also searching for things that may have been overlooked or that challenge those initial thoughts.

Researchers often use this first review of data to develop an initial coding scheme. Qualitative research methods texts provide a variety of approaches to and detailed examples of coding (e.g., Bogdan and Biklen 2003 ; Creswell 2007 ; Glesne 1999 ; Miles and Huberman 1994 ; Stake 2010 ). They generally describe a process of identifying key categories, or codes, with subcategories of related data. The data itself may suggest codes; for example, Brewer (2009) found that his data documenting preservice music teachers’ conceptions of good teaching could initially be grouped into three large categories: personal skills and knowledge, teaching skills and knowledge, and musical skills and knowledge. Over time, these data categories suggested a model of role-identity in music teaching, which he continued to refine over the course of his study (72–82). Alternatively, codes may be initially determined from prior research. Abramo (2011 , 121) set out to specifically study the ways that high school students performed gender roles as they worked in small rock groups. He began initial data analysis by coding the data for issues of power, gender, and musical gesture that informed the study’s theoretical framework, developed through his review of literature.

Using these preliminary codes, researchers review the entire data set again, coding individual bits of data, either by hand or using a software program. Others may prefer tactile manipulation of data; Wolcott (2001 , 41–43) writes data on cards and manually sorts them into coded piles. Hornbach’s (2005) Appendices E and H show her field notes with codes written beside each data segment. Some researchers prefer to work with lists of codes; for others, charts, tables, or diagrams may help clarify codes, subcodes, and the relationships among them. Berg (1997 , 301–16) provides a detailed table of her final set of codes, with a definition and example given for each.

In some cases, numerical analysis of codes may be helpful, such as Norgaard’s (2008 , 202–05) Code Table, which counts the number of references to each code made by each participant, or Jackson-Gough’s (2003 , 51–52) Tables 12 and 13, which count the frequencies with which kindergartners engaged in specific musical events and the types of musical events observed at each of her eight research sites. A comprehensive quantitative analysis of rehearsal communication and thought patterns appears in Berg’s Chapter 4 (1997, 110–39). She also includes diagrams to explain relationships in her coding scheme.

13.7.2 Refining Codes

In the process of assigning codes to specific data, initial codes are often revised by combining, dividing, adding, or deleting categories, until the researcher is satisfied that categories account as well as possible for the collected data. Although coding is time-consuming and can seem tedious, it is important to spend adequate time in this stage of analysis, recognizing that “as you choose what to attend to and how to interpret it, mental doors slam shut on the alternatives” (Agar, cited in Jansen and Peshkin 1992 , 706). Stake (1995) suggests being deliberate, “looking [data] over again and again, reflecting, triangulating, being skeptical about first impressions and simple meanings” (78). Not all data, no matter how interesting, will be relevant to the purpose of a study; therefore, those data are not coded. Alternatively, data that do not fit a code may signal the need to revise or divide codes. They may also point to inconsistencies or contradictions in participants’ beliefs and behaviors that the researcher should consider.

Matsunobu’s (2011) study of North American shakuhachi practitioners offers details about the evolution of his analysis. He began his coding process using five dimensions of spirituality identified by other researchers. From those initial categories, four contrasting pairs of concepts emerged, which in turn suggested questions that guided further analysis. In the process, Matsunobu was forced to question his original understandings of music as culturally specific, and identified a key theme which he had not considered earlier in his research. As Wolcott (2001) suggests, “Good qualitative research ought to confound issues, revealing them in their complexity rather than reducing them to simple explanation” (36); it ought to “trouble certainty” ( Barrett and Stauffer 2009 ). While this may sometimes lead to researcher frustration and somewhat messy research findings, it can also lead to a more holistic understanding of the complexity of the observation setting and participants.

13.8 Writing about Observation Data

The previous section described the process of data analysis, which involves searching for patterns and themes within the data themselves. As the researcher reviews the data set multiple times and refines codes, larger patterns or themes begin to emerge. These themes and broad categories, now supported by specific coded data, help determine the organization of the report of the study’s findings, and writing of findings can begin. Wolcott (2001) suggests first writing a description of what was observed. A separate process, interpretation of the findings, involves researchers in “sensemaking” ( Wolcott 2001 , 33), explicating their own understandings of the meaning of what was observed. In published research, descriptions are often titled “findings” or “results,” and interpretations “discussion” or “implications.” In this section, I first consider writing descriptions, then writing interpretations, followed by a discussion of the author’s voice in research writing.

13.8.1 Writing Descriptions of Observation Data

The first phase of writing is to describe what was observed, creating rich and vivid descriptions of the context and providing key details that “develop vicarious experiences for the reader, to give them a sense of ‘being there’” ( Stake 1995 , 63). A major premise of qualitative research is that no one “correct” view of reality exists; “reality is socially constructed, complex, and ever changing” ( Glesne 1999 , 5). The most carefully crafted descriptions still position some things to be noticed while others are ignored. Therefore, the researcher faces “the possibility of arriving at different descriptions of the same data. This is sometimes regarded as a problem, but I would argue it could also be regarded as an opportunity of seeing new dimensions in a phenomenon” ( Larsson 1986 , 38). In this sense, writing descriptions of observed data is both an analytical and a creative process, as the researcher makes decisions about what to include and omit, as well as choices of writing style and language. The goal is to craft a story that conveys the most important details from one’s vast collection of data, those details that best illustrate the large patterns or themes determined through the analysis process. Reflecting on a previous study, Peshkin (1982) described his goal in writing:

13.8.2 Writing Interpretations of Observation Data

Description of findings is the section of the written report that brings the researcher’s observations to life for readers. In contrast, “interpretation involves explaining and framing your ideas in relation to theory, other scholarship, and action, as well as showing why your findings are important and making them understandable” ( Bogdan and Biklen 2003 , 147). The interpretation section presents researchers’ own understandings of key relationships and connections among the people, actions, and beliefs they observed, and discusses these in relation to the crucial questions, “So what?” and “What difference does this study make?” ( Flinders and Richardson 2002 , 1169).

Writing interpretation is similar to constructing a convincing argument. Carter (1993) states, “We are, in the very act of story making, deciding what to tell and what to leave out and imposing structure and meaning on events” (9). Carefully crafted descriptions will include details that foreshadow points made in the interpretation. Skilled writers make many revisions of both description and interpretation sections, until the vivid portrayals of observed data hang together in a convincing way with the ideas raised in the interpretation. Bogdan and Biklen’s Chapter 5 (2003, 185–207) presents a thorough discussion of choices writers face; Wolcott’s chapter, “Tightening Up” (2001, 109–34), offers invaluable practical advice for editing one’s own writing.

Because a goal of qualitative research is to present multiple perspectives on the observed setting, writers are obligated to consider and report participants’ differing perceptions and to account for plausible alternative or discarded explanations. Equally critical in writing interpretations is connecting a study’s findings to other research, articulating where the current findings support prior work and where they may challenge others’ findings or suggest questions for further research. In addition, interpretations include implications of the findings for practitioners in the field.

Music education researchers use a variety of formats for reporting a study’s findings and interpretations. Koops (2010) presents her data in a “Results” section, followed by “Interpretations” revealing the three main themes she found, and “Discussion” suggesting implications of those themes for music educators’ practice and for further research. Rather than writing a separate “Findings” section, Silverman (2011) describes the choral ensemble she studied in the “Method” section, then presents specific descriptions to support her interpretation of the four themes that emerged from her data.

Dissertations offer additional examples for organizing description and interpretation. Paise’s (2010) Chapters 4 and 5 present her data, first as individual case studies of participants, then as cross-case analysis of similarities and differences in their teacher role identities; in Chapter 6 she discusses interpretations and implications. Eros (2009) , Berg (1997) , and Schmidt (1994) each wrote individual chapters to describe findings related to one of their three research questions; subsequent chapters offer further interpretation of themes and discussions of implications of the three description chapters.

13.8.3 Establishing the Writer’s Voice

Language “is not merely a conveyor of [experience]. Language shapes, focuses, directs our attention; it transforms our experience in the process of making it public” ( Eisner 1991 , 28). By the language they use, researchers craft a writer’s voice, assuming a position that implies varied degrees of omniscience and authority over, under, or alongside participants and readers. Most often, researchers are the primary writers, although they may invite participants to read and comment on some or all of the final report (e.g., Eros 2009 , 71–72; Gray 2011 , 99–100; Pellegrino 2010 , 88–89). Schmidt and Canser (2006) demonstrate one example of researcher and participant sharing in analyzing and writing a study’s findings. Researchers will want to consider whether their language reflects the kind of relationship with participants and readers they intend.

The voice a researcher assumes is thus an ethical decision, because “all types of educational research, both quantitative and qualitative, employ rhetorical forms and thus privilege those who know how to use the rules of the discourse to their advantage” ( Flinders and Richardson 2002 , 1167). Voice and vocabulary concern not only whose perspective is presented, but whether the language of the report permits participants to read, debate, or own it ( Carter 1993 ; Eisenhart and Howe 1992 ; Stake 1995 ). The tone may range from detached or scholarly to warm and poetic. No matter the choices made, writers are obligated to articulate their decisions for readers. In writing about a study, Peshkin (1985) was surprised to catch himself “red-handed, with [his] values at the very end of [his] pen,” but then decided that is “right where they belong” (277). Such disclosures allow readers to evaluate the credibility and applicability of the findings for their own work.

Recognizing that multiple possible interpretations of events exist suggests that readers will add their own voices to the researcher’s discussion. It is important for researchers to “inform readers where self and subject have been joined;” otherwise, the researcher’s experiences remain “beyond control in the research process” ( Jansen and Peshkin 1992 , 710). Although the word “generalization” is problematic in qualitative research (see Bogdan and Biklen 2003 , 31–32; Stake 2010 , 197–98), Wolcott (1990) makes a useful point in distinguishing “propositional generalizations” (stated directly by the researcher) from “naturalistic generalizations” (made as readers compare their own experiences with those the researcher reports):

Words suggesting that readers “should,” “must,” or “need to” do something based on the study’s findings position the writer as more of an authority. In contrast, words suggesting that readers “may,” “consider,” or “might” do something invite more input from readers’ perspectives in determining the usefulness of the findings for their own situations. Flinders and Richardson (2002) provide a good discussion of issues of authorship, suggesting that qualitative researchers “write in a voice that frees the reader to find the meaning of the qualitative research report” for themselves (1169).

13.9 Ethical Issues in Observational Studies

Deyhle, Hess, and LeCompte (1992) offer a history and extensive discussion of ethical issues that may arise in qualitative research. For example, a researcher may be observing a group where several participants have not provided consent forms. No matter how interesting the data these participants provide, their words and work cannot be used in the study, unless the researcher devises a mechanism for collecting such data that meets IRB approval.

Other issues, while within the ethical guidelines of IRBs, are less clear-cut. When the video or audio recording captures the interactions of the whole group, including those participating without permission, can researchers use the video for data analysis, if they use only data pertinent to participants with appropriate permission? Conversations not intended for the researcher’s ears may be overheard, or confidential information may be offered off the record or in informal exchanges. Is this data appropriate to include for analysis? Skilled researchers also remain aware of participants who find recording or photography uncomfortable, disruptive, or even disrespectful. For example, Blair (2009) turned off her video recorder because two girls in the class she studied were very sensitive to being photographed. Similarly, in studying his own teaching of private percussion lessons, Smith made only audio recordings and notes, deciding that video would be too intrusive for his students in that intimate setting ( Smith and Durant 2006 ).

Other ethical concerns may arise as researchers seek to establish positive relationships with participants, for example, offering something to participants in exchange for the privilege of observing. To establish rapport and thank participants, Berg (1997 , 94) regularly brought snacks for students in the high school chamber music programs she studied, offered to help the coordinators and provide them with information about their programs and, at the conclusion of the study, bought a meal or small gift for the teachers and coaches.

Stake (1995) recommends caution in making promises or offers of help, particularly in the early stages of a study. For example, researchers may promise participants the opportunity to review the final research report, and then discover that findings raise issues that participants are not able to or may prefer not to acknowledge. Descriptions that participants find unflattering or that could reveal sensitive information may be important findings. Are these findings essential to reporting the study? Should they be shared with participants?

Angrosino (2005) offers guidelines for such slippery questions under a principle he calls “proportionate reason.”

Angrosino proposes that proportionate reason may be judged through intuition, trial and error, and common sense guidelines such as respecting community norms. Acknowledging that researchers cannot always predict the consequences of their choices, he also offers, “The moral advantage of the proportionate reasoning strategy is that it encourages researchers to admit to errors once they have occurred, to correct the errors so far as possible, and to move on” (737).

13.10 Parting Thoughts

Observational studies have made important contributions to our understanding, illuminating processes of music teaching and learning, and articulating a wealth of individually and socially constructed meanings among music-makers, listeners, teachers, learners, and music itself. More studies of formal and informal music teaching and learning in outside-of-school settings could deepen our understanding of individuals’ motivations to make music and the meanings of music-making in their lives. Observational studies of small groups who gather to make music for themselves or to share with others, such as garage bands, woodwind quintets, drumming groups, country-western bands, vocal quartets, laptop orchestras, or fiddle bands, are lacking. Also unexplored are the behaviors of those who, as their primary way of interacting with music, choose to listen to live or recorded music, as audience members, fans, or avid and knowledgeable collectors. Such studies could offer insights into teaching and learning processes and suggest ways for music educators to more effectively prepare individuals with skills for lifelong involvement with music.

Technological advances continue to provide expanded options for data collection. (See Chapters 16 and 18 .) Researchers are gaining the technical capabilities to conduct observations remotely, rather than on-site. What are the possibilities for combining on- and off-site observations in a study? What is gained or lost, for example, observing music-making in a remote area by Web camera, rather than in person? In addition, Angrosino (2005) warns that it is easy to forget that photographs, recordings, and other digital data are partial and decontextualized. He urges researchers to remember that, far from being a neutral record, technology “has the perceived power to objectify and turn into ‘data’ everything it encounters” (743). He therefore recommends that, rather than focusing on digital records themselves, researchers concentrate on understanding participants’ and their own lived experience.

In my experience as a researcher, I have been privileged to be allowed to observe and share in participants’ worlds and lives. In the process, I have gradually learned to become a more skilled observer. I have learned to plan more carefully and to deal with unanticipated occurrences that disrupt my careful plans. There is no shortcut for developing observation skills—we learn to observe by doing observation. Through continued practice, we discover that

Studying music teaching and learning through observation offers an adventure with few specific rules or recipes. General principles, such as those outlined in this chapter, can help us develop useful questions and determine appropriate methods of data gathering and analysis, offering vivid descriptions and considered interpretations to participants, readers, and our field.

References