Identifying Problem/Framing Research Question

Collecting Data

Analyzing Data

Developing Plan
 

Getting Started
  Now that you have all of this data, what do you do with it?
      The purpose of data analysis is to organize and make sense of all the information that you have collected so that you can understand and explain the question under investigation. Many have compared the process to the peeling away of layers of an onion. As you begin to peel away layers, new layers emerge that need to be examined, compared, and contrasted to previous layers.(1)  Throughout the process, you will often be revisiting and refining your research question and your data collection techniques as themes begin to emerge that you did not previously consider.

• OBSERVE and review the data that you have collected in its relationship to your research question.
• REFLECT upon that data and begin to analyze it for reoccurring themes.
• DISCUSS your observations and reflections with peers for feedback and input on the themes or factors you have identified.
• INTERPRET your reflections and begin to develop relationships among the factors you have identified. Possibly revisit and revise your research question or begin to collect more data.

Identifying Themes
       While you are examining your data, you will be attempting to recognize themes or issues that appear repeatedly or unusual.

Coding the Data
        After you have identified several reoccurring or unusual themes, you will set up a process of coding or tracking the data. You will then begin to identify and notate trends and relationships among the themes as well as all factors that are related to your themes.

Verifying the Data
        As you are coding the data, you will validate it with numerous sources that you have available, i.e. student work, observation, and interviews. This process is often referred to as triangulation and usually involves comparing data with two or more other sources. 

Organizing the Coded Information into Relationships
       This process often involves identifying the themes and the factors or variables involved and the relationships among those variables. The relationships may be of several different types:

• Correlational

Involves identifying the extent of similarities or differences among variables. The extent to which they correlate with one another. The correlations can be positive or negative. For example: You may identify that each time a behavior occurs, another occurs almost as frequently. This would be a positive correlation. Or , you may observe that each time a behavior occurs, another behavior does not occur. This would be a negative correlation. It is important to keep in mind taht just beacuse two of the factors are correlated, it doesn't necessarily mean that they affect one another.
• Cause and Effect

Involves identifying the extent to which one variable affects another variable.  the manner in which one variable influences another variable. In order to determine a cause and effect relationship, you must eliminate or also consider other possible causes and effects and report them as well.
• Causual Comparative

Involves identifying the extent to which two or more groups possess similar or different characteristics. Normally involves more factors than correlational comparison.
• Content Analysis

Involves identifying how many times a particular factor (behavior, speech, characteristic) is noted in a particular setting or context.

Techniques

Procedures

     Data analysis can commonly be categorized into two main procedures--- quantitative and qualitative. Many use a combination of both, giving them a more diverse and holistic picture of the phenomena under investigation(2).
       Quantitative analysis often involves transposing the information into numerical form.For example you may choose to count how many times a student performs a particular behavior such as raising their hand or how many times a teacher performs a particular behavior such as giving praise or rewards as opposed to informative feedback. It is always not that clear cut. For example you may detect from an interview that a particular technology device used in the classroom was beneficial. This would be an emergent theme. You then would want to go back through your interview data and count how many students thought it was beneficial and why. The challenge is that most students will not utilize the same terminology in expressing what they believe is "beneficial". You then would need to first "code" the words and phrases in the dialogue that you believe are related to "beneficial" as "beneficial". This requires some reflective interpretation on your part. Getting a partner or two to assist you in this task would help your validate your choices. 
     Quantitative data is helpful in detecting trends, averages, differences, etc., but it often does not provide a comprehensive or in-depth view into the meanings behind the numbers. For example, we may determine that X number of students believe a particular technology device to be beneficial, but do we know why? Do we know what aspects or characteristic of the device was beneficial? Do we understand the various factors, be they environmental or instructional, that contributed to the satisfaction? Do we know to what extent they were benefited. These are just some of the questions that quantitative analysis may not be able to answer but qualitative analysis may contribute.
      Qualitative analysis involves inferring meaning from holistic chunks of information.  It attempts to reveal a comprehensive picture of all the factors affecting a particular phenomena and the various meanings that may be derived from such.  It gives more credibility to the content of the utterances rather the number of utterances.  For example, an interview of a student may reveal that they believed that a particular technological device assisted them in understanding the material, then qualitative analysis would reveal that it was beneficial because it induced understanding. It would also examine the other phrases surrounding the statement, which would vary depending on the questions asked, but may reveal environments, moods of the subjects, their attitudes, and motivation levels. Quantitative analysis can control for these variables if they are incorporated into the design ahead of time and the questions are specifically asked of all respondents, but often not all of the variables would be thought of and that is where qualitative data comes into play. In other words qualitative analysis puts the themes and their related factors into context.

Strategies
   Each of these strategies may be implemented under the framework described above, and may incorporate both qualitative and quantitative analyses. You may want to refer to the framework as you implement these techniques.
      Hubbard and Power (1993) suggest the following data analysis techniques:(4)

Indexing
     Indexing involves gathering together the various amounts of data that you have collected and weeding through it in order to begin to organize it into various categories that you may have identified as emergent themes. This a step that you actually can begin as you are collecting the data. This technique allows you to organize your data , identify themes that may call for further investigation, and perhaps refine and focus your research question. You would include all notes, observations, interviews, case studies, etc. 

1. The first step is to read through your notes and transcripts of data making notations in the margins of various themes, similarities, differences, interpretations, etc. that you may notice and begin developing headings and categories based on these themes.  For example, you may read in your notes that you observed several students yawning during a task. You would may write in the margin that yawning occurred during this task. Yawning may then become one of your categories.
2. The second step is to  list your categories on a separate sheet of paper and then list either the page numbers or section of your notes that a behavior related to that category occurred. 
3. The third step is to begin asking yourself questions about the categories you have uncovered. You may want to ask the following: 

      * What do these categories tell me about my research question?
      *  Which of these categories are most intriguing and why?
      *  What have I learned about my students, myself, my school based on these categories?
      * Which of these categories do I need to investigate further and why?
      *  What data do I need to collect that may help me to further explore these categories.

For example, you may have observed yawning, daydreaming, or eyes closing frequently through an activity or several activities. You may want to discover why. So you may choose to interview some of the students and ask them if they thought the activity was boring or if they got enough sleep the night before. This is also an example of combining quantitative and qualitative data. Your observations will allow you to count how many students yawned and how frequently they yawned. Interviews will allow you to discover what factors caused them to yawn.  Your list of categories will at first most likely be lengthy as you begin to peel away at the numerous factors that surround each of the situations.

4. The fourth step is to begin charting or diagramming your categories and themes to uncover trends or relationships. For example your list of categories may include activities, student or teacher behaviors, common settings, student interactions, student work, teacher's use of materials, student comments, etc. After you have identified these categories, you will then begin charting relationships among the categories. For example you may notice that each time we are in this setting, doing these activities, students are yawning, doodling, or sleeping. This would be an example of a pattern of behavior that may warrant further investigation an data collection. You may choose to survey your students about characteristics of that activity if it already was not built into your design or you may want to conduct some interviews asking relevant questions. It is important to be careful during this stage and avoid making pre-mature inferences. Just because a student is yawning does not mean the activity was boring or even that students were not engaged. They may have been engaged but just yawning. After all yawning is contagious.  You also do not want to assume that the activity was even an issue at all. You may want to ask what characteristics of the activity made students not want to engage or what else was taking place in the classroom or school? Was there dance or party the night before? Is is a sunny day or rainy day? You want to begin narrowing the focus of your question without answering it before you have considered other possibilities.
5. The fifth step is to begin recording your relationships and determining if you need to collect more data. You may choose to list the relationships you have identified and what sources you relate to the information.

 

• Using post-its or separate sheets of paper attached to the works identifying interpretations or uses.
• Separating the works in folders by themes.
• Assuring that all student samples are annotated with the name, date, activity, setting, or any other information that you may find relevant later on identifying the sample. The main idea is to put the sample into a proper context, so that when it is time to relate it to other data, you have a more holistic picture of what is taking place and various factors underlying the situation.

The Constant Comparison Method
       The constant comparison method involves the process of analyzing your relationships and drawing conclusions. The following steps are suggested by Glaser and Strauss (1967):(3)

1. After you have organized your data by concepts and categories of emerging themes, review these categories and ask yourself: What concepts are represented in these categories? How do these categories relate to my original question? What new interpretations or themes have arisen? How can I identify and refine my categories?
2. Next, begin to merge the conceptual categories and the factors that relate to it, and then attempt to try to discover how these categories are related to a larger framework.
3. After you've narrowed your focused identified just one or a few major themes that are related to one overall theme or idea. After you have identified that theme, revisit your data and assure that it backs it up.
4. Finally, write up your theory as it relates to your over all question. Describe it and summarize it.

Triangulation

        Triangulation may be one of the most crucial steps. Triangulation refers to the process of validating the data that you have. Triangulation refers to verifying your interpretations, conclusions, or hypothesis with three or more sources. It is helpful to consider triangulation as your begin to collect your data. (See triangulation).
        When you can show multiple sources to support your findings you can make a stronger case for your conclusions. No one piece of data is enough to draw  conclusion. 

Tips

Commonly Asked Questions

When is it time to collect the data?
       Data analysis can take place in various manners. You may decide to collect data during the process or after all the data has been collected. If you are using several different techniques (and in most cases, you will) then you may choose to analyze each technique separately and then as they compare to other techniques. For example, if you are doing an observation and an interview, you may choose to first identify themes that arose during the observation and record your interpretations and then do the same for then interview. And then after you have completed both, you may choose to find common themes in both sources. You may just use one source to validate the other source.(2)
        Often before summative conclusions are made, the question is refined and more data is collected.See Action Research Model

How do I sort through this mess?
        Hubbard and Power (1993) suggest the following in preparing and organizing your data for analysis:(4)

     Notes and Journals

• Convert your "raw notes" into "cooked notes" Cooked notes includes what was observed and your reflections on those observations.
•  Begin organizing your notes by themes such as methodological notes, field notes, theoretical notes, and personal notes---organizing them by setting.
• Interpretations of your raw notes can be written in the margins or on separate pieces of paper.

• The first step is to transcribe the data
• Data can be transcribed by typing out every word spoken in the format that it was spoken or
• You can use a topical analysis: Review the tape in chronological order and track who the participants are, the topic, how it came up and what is being discussed. You may want to include a few quotes.

• Xerox the works
• Separate into folders by themes
• Annotate on each piece the date, setting, student, and any other environmental factors that may have influenced the work

• Read through notes (best to do immediately following the interview)
• Make notations in the margins of observations and interpretations or possible connections that you can verify later. 

Data Matrix
 

Theme #1

Data Collection Method	Factor/ Variable #1	Factor/ Variable #2	Factor/ Variable #3	Factor/ Variable #4
Scociogram
Observation
Interview
Student Work

Topic Analysis
 

Title: Setting: Participants: Date:

Digital Counter	Topic	Participants	How it Came Up	What Was Said	Interpretations

Indexing Chart
 

Source
Category	Pages	Interpretations/ Connections

Coding Worksheet
 
 

Student	Utterance	Classification/Interpretation

Observation Worksheet
 
 

Name: Title: Date:

What I Saw:

What I Heard:

What I Did:

What I Said:

Thematic Matrix
 

Hypothesis/Conclusion

Themes	Sources	Factors	Interpretations
#1	1. 2. 3. 4.
#2	1. 2. 3. 4.
#3	1. 2. 3. 4.
#4	1. 2. 3. 4.

Charting Relationships