Feeds:
Posts
Comments

Archive for the ‘Research process’ Category

I’m in the final stretch of data gathering, hoping to squeeze in a couple more interviews this weekend and perhaps one more observation.  This next Monday, April 7th is the day I’m cutting myself off. It’s time to stop the madness!! I’ve come to a conclusion that it’s not the amount of data that’s the most important…it’s the quality of the analysis.

The corpus of data assembled thus far is truly astounding. It’s so multi-faceted, with chat transcript or voice transcription, snapshots, video captures plus audio field notes indicating the on-screen action. There are so many factors to consider in even a short SL interaction.

As my analysis has progressed from inductive generation of categories through open coding, I’ve now moved on to coding only defined instances of information seeking behavior. This, in essence, is my unit of analysis. Observations include many, and the ‘critical incidence’ interview structure that has evolved yields at least one (usually more) stories of information seeking experiences.

I look forward to officially wrapping up my data gathering this weekend and moving on for the final stretch of analysis.

Read Full Post »

Coding is Hard

A grounded theory approach calls for an iterative process of data analysis beginning with open coding. I’ve looked to Corbin & Strauss, seminal authors on grounded theory, to learn that open coding involves several steps:

  1. Apply concepts to data to describe & apply meaning to observed phenomena
  2. Discover categories to group together similar concepts
  3. Apply properties to each category
  4. Define dimensions for each property

I start with a line-by-line analysis of transcripts & field notes.  With 27 pages from my first interview alone, this can be a daunting task. As the research progresses, I’m sure I will “get in the groove” and pick out a unit of analysis — such as certain sections — but in these initial phases, I believe it’s important to engage in a more granular analysis. This helps make sure that I’m not missing critical data points or only pulling out the sections that conform to existing models or my own assumptions (important to theoretical sensitivity, a la Corbin & Strauss).

Additionally, notes and memoing help to develop categories, properties & dimensions further. Corbin & Strauss outline the use of:

  • Operational notes: research procedure or process adjustments
  • Code notes: document & describe potential coding, extending as far as category, general properties and possible dimensions.
  • Theoretical notes: notes on the development of a theoretical sensitivity, or general musings that help with constant, iterative question-asking
  • Memos: Researcher’s own analysis related to the formation of a theory, including emerging thoughts and ideations
  • Diagrams: Conceptualize models, relationships among data, etc.

Memos and diagrams are particularly important for the development of abstract, creative ideation, which lends to complexity & robustness in theory development.

As I write about notes & memos, I’m merely quoting  what best practices research methodologies call for when utilizing grounded theory. I see the value. Yet, my challenge comes back to resources. My first priority is gathering data, transcribing field notes and applying a first-glance level of analysis & open coding. This eats up far more than the 10 hours each week I have earmarked for this project. So, being a pragmatist, I see my “first-glance” analysis I’m performing at this point as kin to the formalized notes & memoing Corbin & Strauss call for.

Read Full Post »

I’ve developed two slightly different analysis forms from what I’d first posted to this blog. They’re spreadsheets to help with the process of recording data from my SL interviews & observations, and then to analyze that data.

One <very obvious!> issue I’d neglected when putting together my analysis form is that a Word document doesn’t offer what I needed to analyze reams of resulting coding. As the content of my analysis forms had been working just fine, I simply converted them into spreadsheets. I’d post my new analysis templates here, but alas, WordPress is not fond of the .xls format.

Simplicity works….just in spreadsheet format this time.

Read Full Post »

IRB Final Approval Letter

At 9:01 am (CST) this morning, on March 6th, 2008, I received the following email from Dr. Charlie Stoops, IRB Chair at Dominican University. The IRB has granted final approval for my research project, so I can begin solicitation of research subjects and do real interviews & observation now.

Even though the IRB process took longer than expected, I learned a lot more by taking an extra week for pilot research. Especially considering that my first two interviews fell through, it was great to have the extra time to get a pilot interview under my belt and to explore a few more places in pilot observation mode. Never underestimate the importance of testing a research methodology!

March 6, 2008

Margaret Ostrander, Graduate Student
Dr. Michael Stephens, Faculty Sponsor

RE: GSLIS 008-001: Users of Virtual Worlds

Dear Ms. Ostrander,

I have completed the review of changes made to the above IRB application that were requested in our letter of February 29, 2008 granting Approval Pending Changes. Based on this review, your application is granted Approval.

Your approval is valid until March 5, 2009. If you complete your research within this time period, please notify the IRB in writing. If you need to make any changes to the procedures of your research, you must submit those for review by the IRB prior to making them. You will be notified 3 months prior to the deadline for renewal.

If you have any questions, you may contact the IRB administrator at irbadministrator@dom.edu.

Sincerely,

Charlie Stoops, Ph.D., LCSW
Chair

Read Full Post »

Exhibit 1: The full experience of my pilot interview lasted 3 hours. The interview itself was about an hour. Set-up, fieldnote transcriptions, & some initial analysis added up to an additional 2 hours. This doesn’t include the coding I plan to do as well. I’d say this makes it less feasible to do the 10 interviews this month I’d set out to do.

Exhibit 2: I’ve found it nearly impossible to do observations for the full 3-4 hour sessions I’d planned. The chat transcripts are far too lengthy to analyze meaningfully. With 3-4 hours of observation comes an additional 3-4 hours of transcribing audio fieldnotes; this doesn’t include the more in-depth analysis and coding I’d eventully like to do. Most importantly, observation takes level of concentration I hadn’t anticipated, and I can’t sustain that kind of focus for much more than an hour or two. It’s exhilerating, but I also find it absolutely draining. After an observation session, I’m mentally exhausted in a way I hadn’t predicted.

So, what does this mean? I need to reformulate my plans and do less. This is not as easy as it sounds. My fear is thath I’ll need to reduce things too much — so that my results are not meaningful. I desperately want the results of this project to be something I can stand behind, not just the tinkerings of a grad student. The sample size and amount of data I’m analyzing is one part of generating sound research results.

So, while this warrants further consideration, I think the most I can do is:

  • 2 hours of observation each week (instead of 4 hrs/week)
  • 5-7 interviews (instead of 7-10)

I’m still a bit wary about my ability to do 5-7 interviews, but going below this amount will not generate enough data for solid findings. And for the observation piece, I feel that 1 hour/week is completely do-able but I’m goin gto strive for 2 hours.

Overall, however, I need to remember that quantity of data is only one measure – out of many – of validity. Quality is equally, if not more, important. It’s striking the right balance that is my challenge.

The difficulty of finding balance is nothing new. With a full-time job, full-time graduate school, family life, and a (semblance) of a social life…..plus a budding 2nd life…..I struggle with it every day.

Read Full Post »

As I’ve mentioned, verbal field notes have made my research process simpler, both in interviewing and observing. A protocol involving verbal field notes is not without its challenges, as this pilot stage of my project has taught me. What have I learned so far?

  • Transcribe immediately. After every observation/interview, I budget the appropriate amount of time for transcription of verbal field notes. Waiting until later loses some of the ‘freshness’ of the experience.
  • Say time stamps every 2-5 minutes. Repeating the time stamps found in the chat transcript makes it a heck of a lot easier when transcribing. This avoids having to fish through the transcript to find the part that the audio field notes are referring to. 

    Pause audio recording. During chat sessions, there are routinely ‘silent’ periods where the other person is typing and I have nothing to comment on (this is especially true during interviews). Pausing creates a much more efficient audio transcript for transcription.

  • Remember to talk. As obvious as it may seem, this has been my main problem. The pilot observations/interviews I’ve conducted have helped get me into the ‘practice’ of keeping up a somewhat constant verbal accounting of what is happening. At first, ‘talking’ seemed to pull me away from the immersive experience of feeling totally ‘in-world’ within SL. With practice, however, I’m able to verbally record my thoughts while still remaining present in my in-world experience.
  • Include both action & reflection notes. It’s easiest to just describe the action. But an ethnographic approach calls for fieldnotes of both activity and the researcher’s reflective process and thoughts. Visualizing a table split between “Activity” and “Reflection” helped so much that I created one. While I don’t use it for recording, I do place it next to the computer as a reminder — this seems to work well for me.
  • Have a pen at the ready. Even though I’ve set up my data collection tools (saved chat transcripts, video capture & verbal field notes) to be all-digital-all-the-time, I’ve also found that I want to jot things down. When this first happened, it was during a critical moment when I did not want to leave the computer, but I had no pen/paper available.

Read Full Post »

Audacity logo 

http://audacity.sourceforge.net/

Audacity is my new favorite tool as a researcher. Thanks to another great suggestion by Jim Oliver, MLIS IT guru at St. Kate’s, I’m recording my fieldnotes verbally. I met with Jim last week over some technical frustrations around trying to make screen capture work with my SL sessions. We talked about a crazy plan I’d come up with involving dueling lap tops: 1 running only SL with my avatar Testy Outlander for ease of navigation AND #2 running SL with another avatar just for screen-capture recording purposes. Additionally, I somehow thought I’d be able to take field notes (pen & paper, no computer).  When it was all said and done, I would have needed something like 14 hands to actually make this plan workable.

Jim casually asked, why not take your field notes by voice? Bingo! Brilliant, simple, and a possibility I’d totally overlooked. The best part is that Audacity can run at the same time as SL with no disruption to SL performance that I’ve been able to detect — at least not on the faster, sleeker work lap top I’m now using for my SL sessions. (It’s just a Dell D600, not that sleek at all….but worlds better than my much older, much slower personal lap top).

So, I’m now using just one lap top, running Audacity in the background constantly as I do interviews or observations, and using Jing for occasional screen capture (read all about how I’m using Jing in my previous blog post).

Read Full Post »

Older Posts »