10.25 Interview protocols + final short analysis project

Short analysis projects:  Most of you have attended (or will soon attend) your conference on the short analysis project. The final draft will be due November 6.  We will have one more workshop (next Thursday).  
Suggestions:

• Set up your essay with a clear statement of your research question.  State the purpose of your essay (what you will show) with respect to that question.
• Make sure you define your codes and categories.
• Present your analysis in a way that illustrates how the codes and categories relate to (what they show about) your research question => for most of you a table will be a good idea.
• Discuss patterns = the relationships between codes and categories and the focus of your research question (e.g. how features of the punchline contribute to the "funniness" of shaggy dog stories; how the features of a comment's complexity contribute to whether students will respond to that comment; etc)
• Use your conclusion to point on what your analysis shows, what it can't show conclusively, and what questions it raises

Interview protocols.  We spent the whole class today talking about interview protocols (the set of questions you will use to guide your interview) and theories for conducting interviews and for designing questions.

We developed general guidelines for conducting an interview that included the following suggestions..
Beginning questions should set up the purpose of the interview, get the participant oriented to the interviewing process, and establish your persona (about 10 minutes)  These questions should be "easy."
Provide an opportunity for an overall story about the interview's focus near the beginning of the interview. This story can set up "themes" or reveal the participants general orientation to your focus.
Many interviews are organized chronologically.
Allow participants multiple opportunities to answer the same or similar questions from multiple perspectives.
Ask primarily open questions, though changing-up with yes/no questions can allow participants to access their memories from different orientations.
The interviewer's primary task is to direct the participant's attention to the focus; at the same time, do not assume that "off-topic" answers are always off-topic (follow up with = can you tell me more, or what do you mean. . .) 
Choose respectful, non-confrontational language

You then read through Hawisher & Selfe's interview protocol from Literate Lives in the Information Age and described how it was "built,"  and whether or how you could use their organization as a pattern for your interview.

For next class:
Read: Gee, Chapters 6 & 7 = these chapters are primarily examples and applications of terms introduced earlier.  This week
Blog 16: Post your draft interview protocol

BRING COMPLETED PERMISSION FORMS.

During class I will present the assignment sheet for your final research project, and we will we will continue to work together on applying methods from discourse analysis to different data sets.

Conferences on Short Analysis

Tuesday
12:30 Kel see, 12:45 Sara, 1:30 Brigit, 1:45 Nahimot,
2:00 Liana

Wednesday
1:00 Valerie,  1:45 Tshandi, 2:00 Kevin

Thursday
10:00 Rachel, 10:30 Josh, 12:30 Shana

Monday
11:15 Andrea

10.23 Permission forms and Figured Worlds

Permission forms and debriefing. I handed out signed copies of the permission form you need to provide to your research participants.  I role-played (thank you, Tshandi) the kind of talk you need to go through with your research participant to present the permission form, and pointed out the different places on the form that you need to sign and provide contact information.

I have posted electronic copies of the forms to the right.  If you lose the signed copy - you will need to print another copy (from the link to the right) and ask me to sign it.

You will need to give one signed copy to each participant, and one collect one signed copy to give to me.  You should provide me with the signed copies either before or when you begin collecting your data.

We also looked at the de-briefing form.  Be sure to fill in your name, email, and phone information so that your participants can be in touch if they want to.

Form-function correlations, situated meanings, and figured worlds.   During the second part of class we talked through a transcript from an interview with a Kean student who was very experienced with computers and gaming.  The focus of the interview was on defining literacies and gaming practices - and whether there were any points of overlap in these definitions.   We noticed in the talk between Ch and B that Ch very much wanted to identify the kinds of learning, problem solving, and social interaction associated with gaming with academic literacies, and that B was - if not reluctant to do that - then at least he was not initially seeing the connections.  We noted the persuasive moves Ch made and the ways B received those moves.  We also noted B's comments with respect to gaming (fooling around, for leisure & recreation, not as important as work - but at the same time it was his passion) and noted that taken together these two conflicting sets of orientations towards gaming seemed to include assumptions, values, and beliefs about video games that might come from two different figured worlds/cultural stories = one for the mainstream or standard story about of video games as a waste of time or "fooling around,"  and another where they are part of an identity (Discourse) for a group of people who are very good at a set of interactive practices, who see themselves as "outsiders" in some ways (at least outsiders with respect to the mainstream - maybe including school)..  Ch's comments seemed directed toward linking the learning strategies in video games with literacies where literacies are ways to make meanings through practices associated with reading + writing  ("that's literacy. . ." "so you know a lot of software?" etc).  Her assumptions, values and beliefs about games (and literacy) seem to come from a different cultural story => one common among composition teachers and researchers who study learning and new communication technologies.  So in some ways - Ch and B were using the same words = but with different assumptions, values and beliefs attached to them, so they were not really understanding each other.  B would need to re-arrange the way he thought about "gamer Discourse" - in some ways - to receive what Ch was saying about gaming and literacy, and Ch needed to think about the difference B was pointing out in more detail.

At the end of class, we took a minute to do some writing/thinking about the nature of what was at stake for B to remain in his story, and for Ch to get him to change, and vice versa.  Keep thinking about that one - as it brings together the three points in chapter 5 about form-function correlations (how meanings are made with "rules"), situated meanings (how the ways certain uses in particular times & places make meanings) and figured worlds - how the assumptions, values & beleifs attached to certain language and use ALSO make meanings. 

For next class:
We will be working on interviewing - so come to class with some notes, ideas for the kinds of questions you want to ask your interview subject.
Blog 15:  Post your best version of your research question for your final research project.  Write a description of what kind of information you want to gather from your research participants, and begin a list of the kinds of questions you want to ask to gather your information.

10.18 Workshop on Short Analysis Project + using some of Gee's terms

NOTE: assignments are now significantly different from what is listed on the calendar. DO NOT use the calendar as a reference for what to do next class.  Use the Read/Blog assignments listed at the end of Blog posts as directions for how to prepare for class.

During the first part of class you used the set of questions on the worksheet for workshopping the short analysis project (posted to the right) to make sure you had a strong question, appropriate codes and categories, and a good strategy for using your codes & categories to answer your question through analyzing your data.  I checked in with each of you and it looks like you are all on the right track.

Using Gee's language in our own example.  During the second half of class, we started an analysis of a community of practice (school classroom participants in general and our research methods class in particular ) in terms of the kinds of saying, doing, and being that takes place within that community.  This was an effort to "play the game" (as in the Yu Gi Oh example) as a way to get a more clear understanding of the terms Gee has introduced.  We made a list on the board of the kinds of things we do in the classroom, the kinds of communications we engage in, and the ways we are => in terms of being students and a teacher.  We included information about what we all knew was expected of us in order to be accepted in the classroom (eg the teacher stands at the front of the class and can move about, students will not circulate around the  class, in class discussioins they address their comments to the teacher, the said, how we interacted, etc).  Within this example, this list of what it is OK to say/do/be is the set of practices defined by (or expected) in our classroom (a community of practice).  These ways of saying, doing and being are also important features of  the  Discourses associated  with this community of practice = Discourses for being a student, and Discourse for being a teacher.

As we noted the ways of saying, doing, and being associated with our community of practice, we raised questions about:
1.  what kinds of meanings (significance) were created through what particular kinds of activities & language;

2   what kinds of practices were both created through our talk = and  assumed as normal or important to the functioning of our classroom;

3.  how our physical location in the room, and how the kinds of things we said and didn't say created particular identities (eg. teacher, student)

4. how our use of language, the physical structure of the room, and our agreed upon practices created certain kinds of relationships (where the teacher "gives" up some power in group work - but continues "surveillance" by circulating through the groups)

5.  the relationships from 4 often are related to the distribution of power or control over social goods (grades, the right to speak, prestige), and we looked at how the "social goods"  created through teacher/student interactions influenced what we would and would not say => as when you observed that students often do not ask about material they don't understand because the social goods of "respect" and "being thought of as a good student=> able to understand the material on your own, may feel more immediately valuable that what might be gained through asking a "stupid" question.

6. We also looked at how the (often unconscioulsy) agreed upon practices within our classroom created certain kinds of connections between identities (being a good student) and practices (coming to class prepared, answering the teacher's questions, staying on topic, demonstrating knowledge of the teacher's preferred topic) and that sometimes "being a good student" could be an obstacle to student's learning needs (asking questions, going back over material from a previous lesson, going off topic to connect to something the student can use to understand the material on the teacher's agenda).

7. Finally, we looked at the different systems for communicating  (the blog, emails, teacher comments, spoken communications) favored students or the teacher.  We might also have considered how the different modes of communication favored different assumptions, values and beliefs about what needed to be communicated regarding our class activities - and how it needed to be communicated.

The purpose of this exercise was to use Gee's language in terms of an experience you are part of.  The reason for identifying & practicing the use of his terms for the "building tasks" is because you can used an examination of what is created in your data (both for your short analysis, and the research project)= as a way to explore what is happening.  Asking about which building tasks are most important, what they are doing, and how they work in your data can help you identify codes and categories for what your subjects are "saying, doing and being" = and you can use those codes and categories to build your theory.

For next class:
Review Gee 1-4.  Go through the examples.  Practice using the language.  Pay particular attention to the terms Discourse, Social language, Conversation, and Intertextuality.

Read: Gee Chapter 5

Blog 14:  Draft short analysis project.  This post should be an ESSAY.  Present your question, codes/categories, patterns, hypotheses and theories in a story form similar to the forms used in the sample research essays we read on commenting and WAW courses.  You may want to use tables or charts to present how your data fits into categories, and you may want to use headings to break your essay into sections.  Good luck!

10.16 Composing essays and Discourse analysis

We started class today with a discussion of Mary Elizabeth Pope's "Composing Teacher Training."  This essay is both a "model" essay for the kind of writing you will create to accompany your Senior Seminar project for the Writing Option Major, and it is an illustration of how reflective, analytic writing both complements a research process, and IS a research process in its own right.

After I read you excerpts from the essay, Teacher Training (which I forgot to send you the link for!) - we talked through "Composing 'Teacher Training'" with particular attention to her composing process.

We noticed the following (below is what we wrote on the board followed by an overview of class discussion):

Activities associated with finding a focus
thinking back on a childhood experience (bad)
a journaling activity where she made a list of topics she would never writ about = conflicted material

freewriting

With respect to these activities, we noted that onflicted  material - things we feel bad or confused about - can often provide good material for research and creative work.  They are "unresolved" = so there is a drive to explore them, and the researcher/writer will have a REASON (other than whatever the assignment demands) for digging into the project. We also noted that freewriting, random associating, talking to friends, browsing the internet, taking a walk = anything that lets material pour into your mind (and turns off your editor) can work as a way to get you to open up new ideas. Putting ideas together in ways you haven't put them together before is central to seeing things "new." 

Research activities

connecting to experience (thinking back on what happend)

visiting physical  places and things associated with your idea

talking to others who were there - or have similar experiences 

peer workshop (discussions with other writers about what connects to your ideas)

As with the "brainstorming" ideas - we noted that the research process seemed to take place through out the writing process . Pope went back and forth between writing - finding more ideas - deciding how to put her ideas into words - writing - and then going through the loop again.  Research activities are not only about reading other texts - for Pope they were about going back to her early experiences.  Psychologists have observed that physical objects - and other people - can serve as "triggers" to detailed memories that might otherwise remain inaccessible.  Photos, places, objects, and other physical artifacts actually seem to "hold" memories for human beings. 

Writing process

trying to write the introduction = part of discovery process

discovery/invention takes place throughout the whole writing process

journaling - to find truth + to craft essay to meet audience demands

clustering = organization association exhaustive categorization/coding

reflective rhetorical analysis = balance between audience + individual truth

We noted that Pope seemed to use her writing process AS PART OF her research process. We also noted that she used her movement among brainstorming, researching, and writing activities as a way to negotiate HER truth into a truth appropriate for her audience.  In some ways, she found what she wanted to say by thinking (and writing) about how BEST to put her feelings into writing.  

Gee and Discourse Analysis

We spent the last half hour of class reviewing the first two chapters of Gee.  In many ways this book is very readable - with lots of examples.  You are READING this text as practice for reading other research methods texts - for when you do research on your own => so you can learn new methods through reading what others have written.  

I noted that Gee's book is set up so that it defines terms, uses them, and then used the terms it has already defined to define and illustrate new terms.  You need to be comfortable with the language in the early chapters in order to be able to understand and use the methods described later in the book.  I identified important terms and page numbers - and that was about all we had time to do.

For next class:

Read: Chapters 3 & 4 in Gee.

Blog 13:  List any terms from Gee you are having trouble understanding.  List/discuss some of the building tasks language does that might apply to your research project.

In our next class, we will begin by workshopping your short analysis projects.  That will take up a little more than half the class.  The rest of the class will be spent clarifying any issues for Gee that you don't understand => and applying his methods in an sample analysis.

Good class = see you Thursday.

10.11 Short analysis project

Today you gave presentations on your analysis of the data from the commenting papers (data set 4).  Listed below are the generalizations we came up with about how to make the analyses stronger/more in keeping with the kinds of presentations in the essays we have been reading.

1. In your presentation of your codes = identify/categorize all codes.  Discuss/set up a classification for which codes are in the same categories, and discuss connections among the different codes in terms of what they mean with respect to your research question.

 2. Present your data as a narrative discussion. 

3. Use your analytic process as a way to develop your research question.  Then - when you write up your findings => use the more detailed, more specific research question to set up your essay.  You can then use this detailed question to organize your presentation of analysis + your findings (as we discussed last class).  

4. Discuss the overall patterns in your data in light of the research question => be sure to relate what you find back to your research question.

Short analysis essay.  After talking through your analyses of the commenting data, we looked at the assignment sheet for the short analysis project.  As pointed out by Liana, this is the warm up or "baby" research paper.  For this assignment - you will develop a research question for an analysis of one of the 4 data sets we have looked at so far.  And you will get a chance to practice writing up all the parts of a research essay except the review of the literature and the methods sections.

For next class:
Read:  We will begin with a discussion of the "Teacher Training" essays - so review them.  This is a model for "auto/biograhical" research.

Also read: Introduction + Chapter 1 & 2 in your text book= Gee, An Introduction to Discourse Analysis.

Blog 12:  We spent the last part of class talking about which data set you might want to work with - and what research question you might ask. This post should include which data set you will work with, a detailed research question, and some ideas for the features of the data that you will be working with.

Also, as we discussed in class - the purpose of working out this question is to focus your post for Blog 13, next Thursday's post = a draft for your short analysis project suitable for workshopping/feedback in class.

10.09 Evaluating analytic write-ups

Today we started class by reviewing where we are and where we are going in terms of projects and learning/practicing analysis.

Review of what we have done:  Beginning with the first day of class - you have been working on doing/using analytic process in the ways that writing studies researchers use and talk about it.  You started by identifying and using analytic process (the logic puzzle + analyzing the oral history interviews & the shaggy dog stories) - and we are moving into methods for representing your analyses to other in writing (the point of the last several blog posts = the analysis of the literacy narratives + the commenting data).  We have read two sample essays - one which presents tables of data and describes patterns in that data, and one which presents a thematic narrative description of patterns in data.

Where we are: you are practicing writing up analysis of data sets that you have not collected.  For your group blog post on the comments, you developed a written analysis that did the following:

1. State your research question

2. Identify the codes/categories /patterns in the data that are relevant to your research question

3. Give specific examples of from the data to show how your codes/patterns relate your data to the research question

4. Explain how the codes/categories/patterns you identified can work as a theory to answer the research question

5. Identify any remaining questions or data that don't fit into your coding/categories = raise ideas for further research

Where we are going next: in class we reviewed and critiqued Nava's essay.  You noticed that the conclusions needed more development, problems with the organization of the discussion of the data, and confusion in the presentation of the methods (specifically the identity of the research participants was not fully explained).  The outcome of this discussion was a  list of features that would characterize a "good" write-up for an analytic research project.

Criteria
Clear, detailed statement of the research question.
Discussion of the research literature or other background that establishes how (or whether) the research question has been answered by other researchers, and why answering the question is important
Description of the methods that includes where, when and how the data was collected + analzyed, along with a description of the research subjects.  The methods should be set up so that they clearly connect to the research question.
Presentation of the data designed to answer each point in the research question.  The organization of the data should reflect the categories + order of discussion set up in the research question.
Conclusions should sum up and generalize "answers" that the data provide

As you write up your short research essays - use these criteria as a guide.

For next class:
With your group - prepare your presentation on the commenting data set.
Read: 'Teacher Training' + 'Composing "Teacher Training"' (handed out in class)
Blog 11:  Complete the points under 4 on the "developing a research plan" worksheet (posted at the right)

In class we will begin with your presentations on the commenting data.  Be sure to hit all 5 of the points listed above under "where we are."

We will use these presentations to reflect on what you are doing well - and what you need more work on in terms of developing an analytic project.  I will hand out the assignment sheet for the "Short analysis project" and you will choose a data set for the analysis.

After the presentations - we will talk about "Teacher Training"  and "Composing 'Teacher Training'".  This conversation is mainly for the writing majors - who will be required to write a composing/reflective essay for their Senior Seminar piece.  This is a particular kind of  essay that is common in creating writing programs and courses.

10.4 Presenting your data analysis

In class today you gave presentations on the analysis you did with your groups on Tuesday.  Your blog post was to write up the work you had done with the writing comments for samples 1 &2 in terms of the tasks you worked on in class (see below)

In your groups you worked to:

1. Review the data (the sample essays with comments)

2. Form a research question with respect to the comments/student's reception/features of the writing context/ comments effectiveness etc

3. Code the data (the comments)

4. Gather and analyze (code) data from your group with regard to how the features of the comments you are studying "work" or what they "do" (your responses to them).  You may find that different group members have different responses - if that is the case try to name the features/factors that account for the differences in response.

5. Identify some patterns with respect to your question - and begin to develop some explanations for why/how the patterns work.

At the beginning of class I gave you some time to review your work, and to develop a presentation that more or less follows the line of presentation in a research paper.

1. State your research question

2. Identify the codes/categories /patterns in the data that are relevant to your research question

3. Give specific examples of from the data to show how your codes/patterns relate your data to the research question

4. Explain how the codes/categories/patterns you identified can work as a theory to answer the research question

5. Identify any remaining questions or data that don't fit into your coding/categories = raise ideas for further research

As you gave your presentations - we talked about how to make sure the discussion covered each of the points listed above.  

Group work for Blog 10.  You then worked on analyzing a larger data set - using a (revised?) version of the research question you posed for the analysis of Samples 1 & 2.  In this analysis, you need to develop an answer to your question that accounts for all the relevant comments in the 6 data papers (two more samples posted in the top right of this blog, two handed out in class).  You will need to code all the comments relevant to your question - and develop a discussion of your codes + examples of how the codes work (as you did in your presentations).  Then develop your theory  and point out any remaining questions.  

Groups:

Andrea, Shana, Kelsee, [Generra]

Corinne, Kevin, Josh, Valerie, [Brigit, Tshandi]

Rachel, Liana, Sara

For next class:

Attend your conference!  Keep your notes.  In class we will talk about using notes from your conference to finish your research plan.

Read:  Where Teachers and Students Meet: Exploring Perceptions in First-Year Composition  Angelica T. Nava

Blog 10: Post your group's analysis of the 6 papers with comments.  

Good class + good conferences!   See you on Tuesday.  I will not be commenting on blogs this weekend.

Conference schedule

All conferences will take place in CAS 324, my office.

October 3, Wednesday
1:45 Tshandi;
2:00 Corinne

October 4, Thursday
12:30 Shana
12:45 Sara
1:00 Josh
1:15 Valerie
1:30 Rachel
2:00 Liana

October 8, Monday
11:00 Andrea
2:45 Brigit

October 9, Tuesday
10:30 Generra

Be in touch to set up a time: kelsee, Kevin, Nahimot

10.2 Practice designing research questions and research projects

On Thursday last week we went over Robin Martin's essay as a model for form.  Today we talked about it in terms of patterns for designing research.  As set up in her introduction, Martin's essay asks the question: how does the form and content of teacher's written comments on student papers affect whether and how they revise their papers.  She uses codes and categories from an earlier study done by Ferris. [ I passed out a copy of her codes for features of comments - and for deciding whether/what kind of revisions students made to their papers; this information is also available in Appendices A & B in her paper.]

After discussing her system for coding comments - we put together a list of features/kinds of comments that students liked and disliked.  These lists from the board are reproduced here.

Helpful comments

on flow - to help develop movement

on organization  - to help plan overall arrangmente

clarity (to help know what needs to be there - if the point came across

point out if there is too much

connect to current ideas of what writing is

help with formal conventions

experessions of interest

encouragement

explanations

references

global suggestions

  

Hated comments

bigxes without explanation

unacceptable/you don't understand

very long comments

change => but not telling how

disagree + net getting something writer cares about

bloody paper

demanding different styles

As we created these lists, we started a conversation about what DIFFERENT kinds of questions researchers might ask about connections between students' development as writers - and the kinds of written comments teachers note on their papers. 

Some possible questions researchers might ask in clude:

• what is the role of positive comments (comments that don't request change) in student response?
• how does the directness of the comment affect whether /how students revise?  do different students need different levels of directness (groups should discuss the different questions among themselves)
• how (or do) comments build a relationship to the instructor? does this relationship influence whether or how students revise their paper?
• can comments increase students' engagement with their work?  what kinds of comments contribute to increased engagement?
• what is the cut off between feeling like the teacher is committed to helping the writer - and feeling overwhelmed?  what kinds of comments contribute the most to feeling overwhelmed? how much of feeling overwhelmed is about the students' situations - and not the commenting? 
• how do negative coments affect willingness to advise?
• differences between talk and written comments
• why do teachers make "demanding" comment
• role of positive comments - can too many comments have a negative effect?
• what are the effects of harsh comments connect to teacher expectations
• which comments (what kinds of comments?  what are their features?) do students tend to take personally - and is that good or bad?
• what kind of comments (or what features of comments) hurt student relationships to writing - why?
• what features of negative comments allow them to help (or not) writers to grow as writers/improve their writing?
• how do the writer's need shape the kind of comments a teacher should give?

Some of these questions still need some work before they would be a useful basis for a research project - but they all raise interesting ideas/problems.

After we talked over the questions and how we might turn them into useful research questions (questions that direct the researcher to the kind of data they will need + the features of the data they may need to attend to) - you worked in small groups to design a question and a project focused on comments on the sample essays with comments posted to the right.  

Groups: Josh, Corinne;  Andrea, Generra, Shana;  Valerie, Brigit, Tshandi;  Sara, Liana, Rachel

In your groups you worked to:

1. Review the data (the sample essays with comments)

2. Form a research question with respect to the comments/student's reception/features of the writing context/ comments effectiveness etc

3. Code the data (the comments)

4. Gather and analyze (code) data from your group with regard to how the features of the comments you are studying "work" or what they "do" (your responses to them).  You may find that different group members have different responses - if that is the case try to name the features/factors that account for the differences in response.

5. Identify some patterns with respect to your question - and begin to develop some explanations for why/how the patterns work.

Blog 9:  Post your group's work on the task outlined above (points 1-5) so far.

In class on Thursday, you will apply your "theory" to a larger data set by looking at more comments from papers, collecting more data  (from classmates) - and seeing how well the patterns + explanations you came up with work on a larger data set.