Brilliance rarely, if ever, happens alone.

We celebrate those that advance new ideas and often forget the context from which they came. Usually these brilliant thinkers are surrounded by students, communities, friends, and patrons that support their work, push them in new ways, and help with construction. What seems elegant in the end was actually the result of many, many hours and people pooling energy, grinding work, enthusiasm, creativity, and passion.

What would it be like to look inside these workings? How would it feel to clean stables out for Frank Lloyd Wright, to collect paints for Da Vinci, or to work endlessly as a member of the New York Philharmonic? Even for a day or two, what would it be like to work side by side with fun, smart people of expertise in their fields? How do they solve problems, ask questions, laugh, and explore new ideas? What if we brought together experts and novices in a given field and provided them with playful "work " to do together and then captured these experiences for anyone to read? This would be something to see. This would be something to read - something brilliant.

You are about to read chapter after chapter of people gathering in their spare time to work at what they love in the topic that they love - for fun. This chance to see inside their world is an opportunity for which I owe them thanks. All of the work you are about to see, (and the hours it took to assemble), is a contribution from people that care enough about their field to find time to play at it and share it with us.

The beauty of RTR, for insiders and observers, is that it captures community in a captivating way. Whether or not you are particularly interested in games and learning or game design, RTR is more about the process of playful investigation because we had a community of people willing to play for you. This is not a normal book. It is more of an orchestration that captures the expertise of educators, scientists, and designers as they work together. It's not what they found, but how they found it that stands out to me.

This project began as the brainchild of Eric Zimmerman, Constance Steinkuehler, and Kurt Squire. In the first chapter they outline how it came to be and how to conduct an RTR session yourself. They dreamed it up, ran the initial sessions, and even tell me they may come as participants in RTR sessions to come. RTR is the passion of these three and any vibrancy you see in the following pages is a shadow of their own.

This book was first published as an article in E-Learning http://www.wwwords.co.uk/ELEA/ in 2009 - Volume 6, Number 1. The editorial board welcomed this fairly non-traditional work because of topics, the influence of game design for learning, and the vision and encouragement of Colin Lankshear, Michele Knobel, and James Paul Gee.

Finally, Drew Davidson from ETC Press has been the shepherd that has guided it from article to an expandable online book. He has set up a book that can grow with the RTR projects to come in the future. Drew is part of a new vision for publishing that brings the written word to more people in more places in more current ways than we've even seen. Frankly, I'd like to see him write a book on this dream, but for now it's a privilege to work with ETC Press and be part of their growing stable of stud titles.

RTR is essentially the work of its players however. The 'music' you hear is because we had experts invested in the work, lovingly recording and writing sections, and patiently responding to the constraints of the book format we have here. Moreover, these are busy people. When we first began to think of this book happening, our expectation was that we'd only get three or four of these groups wanting to return to their 'work' groups to collaborate on this. They had games to produce, data to collect, dissertations to finish, books to write, and awards to prepare speeches for. We only hoped they'd spare time for RTR, yet of the twelve groups we were able to contact, all twelve are presented here with only a few not able to return to their groups. This I find simply remarkable and a testament to the power of playful work. Not only are these good people, they are charitable with their most valuable resource of time.

A special thanks for the GLS and GDC communities and conference staffs that first hosted RTR. They graciously put up with the demand oddities of time, printing requests, post-it note walls, sock interviews, boxes of assorted supplies, posters, and myself - none of which are the expected needs when running a conference. Without their patience, RTR isn't on these pages.

For my part, my family has been a constant support and help. Grant and Katie bring treats and "tip-toe while I type" and my wife Stephanie puts up with, even loves, my eccentricities in planning and editing for RTR. I also thank my advisor, Kurt Squire, along with Constance and Eric for bringing me on board. I'm still along for the ride and appreciate every confidence and allowance they have offered along the way. It's been a joy to work towards sharing experiences that have since evolved into 'real', larger research projects and game designs.

If you are reading this book, you are now a part those experiences that we found inspiring, challenging, and maybe even... brilliant. You are part of the RTR world, so thank you.

Seann Dikkers

by Seann M. Dikkers

How do you translate a great model

of fun learning into an approved

research project for publication?

How do you translate a great model of fun learning into an approved research project for?

RTR by itself can stand alone as an engaging model of practice in learning environments. Researching interesting questions has long been an entry into learning not only about the topic at hand, but about the practice of research itself. At a third level, those that enjoy research and the process of discovery are often leaders in their respective fields. RTR is a fun entry into practice on this level and we hoped to capture these experiences in the book you are reading now - not only so you could enjoy the work itself, but the process, practice, and even your own use.

In fact it is our hope that you'd enjoy the work here so much that you would use RTR as a way to bring playful rigor to your learning environments, classes, and practice. Use RTR to test new ideas that may be worth further study, build concepts, methodologies, and research team cohesiveness.

IRB

To share the work of RTR, as a learning tool, it was important to pursue good standing with the Instructional Review Board (IRB) and those that review research for the university. Simply conducting an RTR session in a class or business environment wouldn't require any of this, but to publish we needed IRB approval. IRB's have been in place to protect the institution, but also to protect the researchers and the fields they represent.

Each IRB is unique to it's institution so your work getting approval will essentially be local. Attached here is the language that we used at the University of Wisconsin - Madison after meetings, suggestions, and the help of the IRB panel. Below you'll find the entire submission for your use and as a working point for any IRB work you may do. If you are trying RTR informally, without intent to use the data beyond course credit, you may want to skip to the second part of this chapter. If you want to be prepared for publication, then you are welcome to use it as a starting point for building your own IRB submission for research.

We found that making initial contact with the IRB provided the opportunity to share and connect on the vision with representatives ahead of time. Setting up a time to look at the submission provided invaluable insights into the importance of good review, how to structure our submission, and also gave us a strong communication line along the way. Your local IRB will have different levels of accessibility however and these initial meetings, though useful, are not essential.

When done, approved, and you are able to broaden RTR work for publication, this book always has room for a few more good projects.

Submitted to UW-Madison IRB:

ABSTRACT

Real”"Time Research (RTR) is a conference workshop held at professional games (and learning) conferences ”" specifically, the Games, Learning & Society Conference (GLS, Madison WI), the Game Developers Conference (GDC, San Jose CA), and possibly the Digital Games Research Association Conference (DIGRA, London). The GENERAL PURPOSE of this project is (a) a learning opportunity for participants less experienced in successful interdisciplinary collaboration among academics, designers, and educators, (b) to provide a venue for piloting new research questions or replicating known ones, and (c) to provide a new and rich venue for a learning experience at these conferences. We accomplish this through a two-part workshop involving game researchers, game designers, and other professionals in game-related fields attending the host event. RTR attendees participate in a workshop at the beginning of the host conference to collaboratively design, on (typically 5-12) cross-disciplinary project teams of 5-8 individuals, separate research projects that are conducted over the course of the host conference itself. Groups identify a theory of learning and methodology to frame their project, generate a research question, and then gather the necessary data from fellow conference attendees during the host event. After data is collected, they reconvene in a second workshop at the end of the host conference to debrief on the feasibility of their methods and processes and to share their findings in the form of a 5 minutes public presentation on their project and a short "chapter" in an online RTR book to be published with ETC press. Our interest is in the project group work and the process of designing research itself. It is the final debriefs that we collect and analyze for RTR publication. The project reports in the second session become the target of study and a form of data used to write our reflections on the designs and processes employed ”" much like professors respond to class projects and write about lesson design referring to them. We are also interested in the refinement of the RTR process over time and how it evolves through much iteration.

STUDY DESIGN & METHODS

Inclusion Criteria

The inclusion criteria for participants in RTR research projects are purely voluntary. Adult professionals who are already attending the host event (GLS, GDC, or DIGRA), and furthermore select this session to attend, select themselves by taking the workshop. No underage minors are involved and no special groups are targeted in any way, nor is any personally identifiable or sensitive information kept. Because these conferences are conducted in English, all participants would have adequate English fluency. All participants have the option at any time to simply enjoy the rest of the conference without further participation.

Number of Participants

10-30 participants attend the RTR sessions and form projects. For their projects they have access to other conference attendees ranging from 300 (GLS) to 3000 (GDC).

Each project group varies in the number of cases it is willing and able to involve. This number varies as necessary depending on whether the project involves, for example, observation of participants using a specific game interface (15 minute protocol), a short interview (5 minute protocol), or a series of Likert scale questions (1 minute protocol). Longer protocols involve fewer participants given the nature of this workshop and the fact that data collection must only be done within the time constraints of the host conference. We estimate that, at most, 200-400 persons at each host event would participate in any form of the projects. This however is secondary to the core of the project, which is the smaller number of participants in the workshop and part of the RTR work.

Role of Participants

Each RTR interdisciplinary teams will participate in their projects as they see fit and these roles will vary. For the first session they are planning their projects, they carry them out during the conference, and at the second session each group of participants shares out on their project. These reports are the target of this IRB. We would ask them to write a complementary report on their projects and use this for our analysis and interaction with the data they collect.

So far involving conference attendees has included responding to short interview questions about game play preferences, answering Likert scale items about videogames and learning, briefly playing a game title at the host conference under observation, or agreeing to submit one's online twitter streams for analysis (with identifying information removed). All of these interactions are studied in public settings at the host conference venue and are engaged for as briefly as possible so as to minimize disruption of their professional event while maximizing the number of cases that can be included. We provide a guide (can and can't do list) to our participants, including a script that is attached to the IRB, that instructs them to state their name, project, how they selected the person, risk/benefit, voluntary nature of the work, and that no personal information will be kept.

We set up strict rules for the projects. No deception is involved, no identifying or sensitive information is collected (not even names), and no topics are raised that could in any way be embarrassing, diminishing, or deleterious in any way to participants (i.e. nothing transgressive, sexual, embarrassing, or unduly personal such as intimate feelings and relationships toward others or oneself). Participation needs to be entirely voluntary and, before any data is collected, oral consent is obtained and individuals are reminded that they can cease participation at any time. With consent, images, audio and video are, at times, recorded as part of data collection but only for record keeping and analysis with no such identifying data shared in any public venues either written or face-to-face.

Compensation

The only compensation given for participation is a small sticker which reads "RTR ”" I subjected" for the individual to place wherever they like (e.g. their conference badge, notebook, or computer) or not. Consenting project teams can have their work be a case used and published as an RTR outcome.

Sites

The RTR workshop is held at three host events, all of which are professional games (and learning) conferences: (1) the Games, Learning & Society Conference (GLS) held annually in Madison WI, (2) the Game Developers Conference (GDC) held annually in San Jose CA, and (3) the Digital Games Research Association Conference (DIGRA) held this Fall in London (optional if international regulations would complicate the IRB process).

Does the study involve participants from places other than common public spaces?

No

Measurement Procedures

The measurement procedures to be used in this study vary depending on the nature of research questions developed by each project group. Our observation of the groups at work in addition to the final group reports give the core information for reflection on the work, design of the study, and follow up questions. After we verbally share our thoughts about the projects, groups are invited to write up a summary and reflection piece about the experience with a template for consistency. These write ups along with our commentary make up the core of the research. Therefore, when the groups report back at the second session, we will record and keep records of the findings they present. We collect the slide shows they used and written reflections of the project along with our reflections and feedback on the projects.

Will any of the following be used as part of the study: questionnaires, measurement instruments, interview protocols, or a description of topics or an approximate script?

NOTE: Yes, but because the exact instruments will not be developed until the actual RTR workshop, we have no detailed measures to include with this protocol at this time. There are no instruments formally developed for the participants in the sessions, only ones they may create and use.

Verbal consent will be attained with any participants. Handouts will outline this process. (see attached)

Recruitment materials:

[none]

RISK/BENEFIT ASSESSMENT

Are there risks to the participants?

No

Steps to Minimize Risks

We minimize risk by clearly outlining and providing a written guide to constraints for the session projects (much like in a classroom setting). This includes not collecting any identifying information (including names and institutions) of any form and eschewing discussion of any topics that could in some way pose personal, social, material, or political risk to the participants.

Any digitally identifying information is immediately purged from the data corpus before analysis, and any images or audio or video that is collected as part of the research is not shared publicly either through presentation or through inclusion in any written products of this work. If any group were inspired by the designed projects at the conference, they would need to submit separate IRB's and replicate the research for any separate publication/s.

Any info that is stored concerning the group presentations and findings will be filed and stored in an external password protected hard-drive kept by the PI's on this project.

Medical or Professional Intervention

n/a

Alternative Treatments

n/a

Possible Benefits to the Participants

The possible benefits to participants are both immediate, short term and long term. First, because many professionals in the games industry are also game players and avidly interested in their own learning processes (as well as the processes of other players), one immediate benefit from participation is simply the opportunity to talk about investigation, meet colleagues in the field, create a collaborative project, and have an authentic assessment in the presentation of their work.

Second, because we share our general findings at the end of the conference that participants have chosen to attend, they have the opportunity to immediately see the outcomes the work that their participation has made possible. Oftentimes these aggregated findings provide an interesting context for reflection on one's own views.

Finally, because subjects are academic and industry professionals in the field of games (and learning), the findings of these small pilot studies are of immediate benefit to participants professionally in that they add to our collective knowledge about this new emerging field. RTR workshops provide a venue for exploratory and educational collaboration on research topics of interest across disparate disciplines. In our experience so far, individuals who have participated have had overwhelmingly positive things to say about both their personal involvement and they value they feel it brings to our profession. Many subjects wear the "I subjected" stickers with pride and encourage others to volunteer because participation is seen as both informative and fun.

Benefits to Society

RTR workshops provide scholars and designers in the field of games and learning an opportunity to work together on interesting questions and pilot attempts to answer those questions with minimal investment of professional time and resources. It fosters conversation across domains, which in new fields of study in particular, is especially important. As "games and learning" becomes an ever increasingly popular topic of academic and public interest, innovative hands-on educational workshops like RTR can help stave off the disciplinary "silo'ing" so detrimental to forward knowledge by fostering conversation, collaboration, and the exchange of ideas across areas that otherwise not in conversation.

ADDITIONAL INFORMATION

To date, RTR has been a very big success as an educational workshop; our main goal in formalizing the research through IRBs now is to enable us to insure that our handling of data involving participants is appropriate so as to enable broader distribution of our methods and findings in the form on an online book through ETC press (who came to us with an offer for publication given RTR's strong reputation and success).

Products/materials used in the study:

1. RTR: Research guide and consent script

2. RTR: Follow up (consent for participation)

3. RTR: Report (guide)

4. RTR: Cards

5. RTR: Supplies

The IRB process required the initial submission and edits based on follow up from the committee, two panel members consulted with us and helped guide those revisions. Finally, we got the approval for RTR and were able to take the RTR projects as data for publication.

Special Notes or Instructions: After discussions between [IRB representatives] and the research team, this protocol has been submitted. [IRB representatives] have determined that the research team has done an excellent job in addressing any IRB concerns. Therefore, this protocol is determined to be exempt pursuant to 45 CFR 46.101(b)(1).

Preparing for the session

The first and primary requirement for RTR is your understanding of how it works, and being excited about iterative research. In the previous chapter Eric and Kurt laid out the premise and basic design of RTR. Beyond this, many practitioners would be comfortable running with it and making adjustments on the fly. Your design choices will customize and make RTR come alive in your setting.

For others, you may be asking for more detail and a look inside the 'on the floor' implementation. Here is a short but useful 'to-do' list of sorts. As the IRB work is simply laid bare, below are the lists and notes we built over time to make sure everything was in place.

During the RTR sessions we kept track of both ideas for the future and needs that emerged for the sessions. With each of the three iterations this document became more refined and useful for our practice. Moreover, we can easily share it with you.

Materials Needed:

” Emphasize that RTR needs should go through graduate students, not

conference folks

” Have some example studies that we present in our intro

” Customize the card deck each year.

” Notepads/pens/markers

” power strips (for many active laptops)

” timekeeper

” set of cards - color glossy printing on card stock

First session:                                                                                                                                        

 ” keynote presentation w/ samples of RTR and template for groups

” templates of data collection tools

” sample consent scripts

” group information forms (to record contact information)

” "do's and don'ts" of research

Booth or Handy Resources for RTR teams:                                                                                                                                 

” big sign ("RTR: Real Time Research w/ GLS logo) Lamenated  2 - 2' x 3'

Poster

” printer

” internet connection

” e-mail address for RTR (so folks can send files to have printed)

” clipboards (10)

” paper (1-2 reams)

” stapler, markers, pens, notepads, string, tape, portable file/organizer, file

folders, easel w/ sheets of paper

For next year:                                                                                                                                            

” full time RTR grad student (or two) w/ parking passes (for supply runs)

” access to copy machine

” separate table, close to registration

” set up RTR forum/wiki/type thing for folks to stay in touch if wanted

” video cameras

” digital cameras

” tag boards/White boards w/ easels/public wall space

” add a panel of judges to the final presentation & give out awards for various categories

” create a new category of cards called "material constraints"

At the end of this book, you'll find everything else you'll need - including cards, handouts, and templates. Use them all as starting points for your own RTR project.

The RTR projects

Before digging in, enjoy a few samples of work. Our collection of RTR alumni are excited to present their findings and analysis of their RTR projects. Enjoy them for the interesting investigations that they are and use them as case studies of RTR in action and get to know RTR from the student's perspective so you can move towards your own use of RTR.

You should know a few things before you read on. First, these groups were invested enough in a few days of collaboration to return to this writing months later. Second, the value of sharing, writing, and working together were all the incentive available. Often the topics included here were for fun, but not necessarily in line with their research work professionally. Finally, the project chapters you're about to read were written by very diverse teams. What may appear somewhat consistent in method is actually a combination of researchers, game designers, students, teachers, administrators, and technology specialists - and I suspect some closet artists are included too. These are professionals from a few different walks that have set aside time, energy, and a bit of love to share a few days of their "play" with you.

Use these as examples for your own practice. If done right, this is the sort of work possible. More exciting is that your learners will probably improve on these. Your iterations will add to what RTR is now.

Encouraging Writing

With ETC Press we had the chance to offer the RTR groups a chance to write and share their work. Many did just this. Whether or not you are working towards publication, the process of writing and analysis over time extends the learning for those involved. This sort of revisiting of the work also extends the initial relationships built by those involved.

In order to engage participants in a writing reunion of sorts, I simply sent out e-mails to the groups and invited them to participate. At the end of this book you'll find the template (Tool 6, page _) that was attached so writing could be consistent. This template also made the process more accessible because the effort only required 'editing' their presentations from the RTR sessions into a more formal context.

Invariably the groups saw this as an opportunity to add in what they didn't have time for when we ran the sessions. Groups took the time to cite the writers that influenced their inquiry thread and methods. In addition, it was exciting to see what a new look at the data produced. Most groups used the comments from our experts at the conference to revise their work in writing.

Once the groups had a working chapter put together, the drafts were presented to Constance, our expert reader, for another round of suggestions and edits. Groups cleaned them up and submitted what you see here.

Enjoy.

Sock It To Me!

Puppets as Avatars

by Arthur Johnson & Ann McDonald

Will there be a marked difference and/or changes in behavior if we ask participants one question in the work frame and then ask another question in the play frame?

Will there be a marked difference and/or changes in behavior if we ask participants one question in the work frame and then ask another question in the play frame?

As participants in the Real Time Research (RTR) session conducted at the Games, Learning, and Society (GLS 4.0) conference in July, 2008, our group received the research cards: Methodology: Survey, Topic: Literary Media, and Theory: Behaviorism as an initial framework.

Our group was comprised of individuals of varied backgrounds: educators, designers and researchers. One group member was especially well versed in educational theory and research methods. We used the Real Time Research methodology of rapid, improvisational investigation at a conference where most were attending in a work capacity, but admittedly were present to examine what could be learned from game play to enhance learning, culture, and education.

Our group decided to explore the tensions between work and play and to test whether talking about play would elicit greater engagement on social, affective, and cognitive levels as compared to talking about work. The hypothesis was that people would respond differently when talking about work and play and that the use of a sock puppet would elicit play behaviors and greater engagement and further activate the play space.

Play theorists such as Sutton-Smith (1997) argue that work and play represent different ethos (i.e. the way that we engage with, and attribute, an activity). This distinction is particularly interesting for educators who, on the one hand, want the kind of commitment we associate with work but, on the other, also want the sense of experimentation that we associate with play. Play may impose what Gee (2003), in recalling Erikson (1963), calls a Psychosocial Moratorium (PM), where a person can take a time-out in life and retain a fluid or dynamic identity through which they are able to take risks in a less consequential environment.

Specifically, our research question was: Will there be a marked difference and/or changes in behavior if we ask participants one question in the work frame and then ask another question in the play frame?

Methods of

Data Collection

Members of our team took on roles as puppet makers, participant wranglers, interviewers, camera operators, video editors, and coding framework designers. We conducted interviews of participants to fulfill the Survey criteria, recorded video to fulfill the Literary Media criteria, and coded observed behaviors from video interviews to fulfill the Behaviorism criteria.

We recruited conference participants to tell us about their work and play, recording the interviews with videotape. To get a sense of their work role as a benchmark, we asked them first to state what they did for work.

With participant's head framed on camera we asked, "What do you do for work and how does it relate to games?"

We were inspired by the conference 'swag' as convenient and readily accessible play materials for the creation of sock puppets. Materials circulated to most conference participants included dark grey GLS 4.0 socks, round GLS avatar buttons, and white individually wrapped Life Saver candies. We added bright colored rubber bands and tape in order to create a series of sock puppets with a variety of distinctive looks.

We attempted to create a "magic circle" using sock puppets in order to create a space that would invoke playfulness and enable a play identity to emerge spontaneously. (Huizinga, 1938/1986; Caillois, 1962/2006) We offered participants a choice of puppets as avatars and provided additional materials so they could customize a puppet or use their own conference socks, eliciting a 'ludic spirit'.

Wearing their chosen sock puppet, we then asked participants to step out of camera range and let their sock puppet become the focus of the camera and asked, "What do you like to play?"

The majority of attendees were willing participants; of the fifteen who we asked to participate, only three declined. As we were conducting interviews, people lined up to participate because our interviewees seemed to be having so much fun with the puppets. Some of the interviewees gave more than four minutes of interview as a sock puppet, exceeding even our expectations. The fact that the interviews were done in a relatively safe space of a conference setting and in public concourse may have led waiting participants to be influenced by others preceding them and engage in attempts to "outdo" previous participants.

A number of the participants integrated their own play stories with those from their sock puppet's point of view,

"My new favorite is guitar hero, I can pick it with my little nose right here, its really fun to do and I really like my master, he's great, come over here (kiss)"

"My favorite games to play are ones that are one-handed so I can use them."

"I like to play anything I actually can control with my mouth, maybe, I don't know something full body and minty fresh."

Or offered revealing personal narratives such as, "I don't play many games, I'm a sad, lonely person."

The vocal shifts to more childlike or higher pitched voices when using the sock puppets suggest possible childlike assumptions about the puppets, which are thus allowed to have more playful voices and uncensored, informal, humorous responses.

Methods

& Analysis

We found the format for data collection was quite effective. The video record allowed us to edit together a series of interviews for playback and coding analysis by the entire RTR follow-up session. In the spirit of RTR and using each other as resources, all the RTR session participants were asked to code the behaviors observed in the video interviews using a printed coding form as part of our team's results presentation. We provided a framework built from elements of engagement as summarized by Chapman (2003) and codified by Dubbels (2008) and asked all to code body positioning as symmetrical (non animated, stiff, not much variation in tonal quality or facial expression) versus asymmetrical (animated, varied tonal quality, relaxed, and verbose).

Findings

Due to time constraints, we did not tally all the coding results, but rather had a group discussion about the observations made during the coding process. Some observations from the RTR session were that the work responses sounded canned and terse while the play responses through the sock puppets were clearly more relaxed and humorous. In answering the standard conference question icebreaker "What do you do?" many participants struggled at first to find words to answer a question they had likely already answered several times earlier that day. In contrast, interviewees engaged and expressed themselves readily while using the sock puppets. All those talking through the sock puppet adopted a clear frame of play through voice modulation and additional narrative. Communicating through sock puppets immediately put participants at ease talking about personal issues with complete strangers. Participants worked very hard at their play and were expressive, creative and willing to take the risk of being silly.

In the ethos of work and play, play is often not regarded as a productive learning activity. One of the challenges to educators may be the predominant metaphor of learning as work. These findings reminded us of Wohlwend's (2007) studies of teachers observing students' learning. As teachers watched children playing, they began to see their play as directed towards and around exploration of the outcome and content manipulatives and therefore reconsidered their theories of learning as work. Observing the sock puppet videos showed that people worked very hard at their play, were willing to elaborate, be expressive, creative and take risks. As all these qualities are important to innovative work, we ask, can play be a portal to tap into more productive work?

Conclusions and Future

Research Questions

The creative and impromptu nature of the research project engaged the expertise of all the team members. The puppet making and video methods were more aligned with visual designer's typical tasks and the design of the interview questions and coding methodology was more aligned with the participant's typical tasks. The collaborative, interdisciplinary nature of the team structure and the initial conceptual framing of the research project based on the cards provided, materials at hand, and the limited timeframe allowed all group members to participate without feeling inadequate because of a lack of training in research methodologies.

The mapping between sock puppets and avatars offers some intriguing opportunities for non-digital, interactive, rapid prototyping research, using sock puppets to more deeply explore player/avatar relationships. We often think of the question of the player/avatar interface as being specific to a video-game player controlling a digital character. However, the sock puppet gets at the core theoretical questions of this complex issue without all the technical hurdles involved in the creation of game characters and arguably could result in more widely relevant findings, reminding us that play and games are not intrinsically tied to the computer.

The role of the sock puppet across multiple frames could be studied more fully by adding a control group with and without sock puppets and a reversal of the work and play framed questions answered using the puppets. Using a sock puppet to immediately elucidate a sense of play is an easily replicated process. This suggests an untapped method for encouraging reluctant participants to open up for interviews. The use of sock puppets as a research methodology suggests that simple role-playing (through conventions such as sock puppets) may be an underused method for quickly creating a ludic spirit around reflection activities.

Acknowledgments

This investigation was a collaborative effort enacted by a diverse team of researchers: Brock Dubbels, Arthur Johnson, Janet Kretschmer, Christine Lupton, and Ann McDonald. Our research was facilitated by the real time research session leaders, the generous logistics and detailed support of Seann Dikkers and the contributions of GLS 4.0 conference attendees who willingly played along on camera.

References

Chapman, E. (2003). Alternative approaches to assessing student engagement rates. Practical Assessment, Research & Evaluation, 8. Retrieved October 14, 2009 from http://PAREonline.net/getvn.asp?v=8&n=13.

Dubbels, Brock. (2008) Video games, reading, and transmedial comprehension. In R. E. Ferdig (Ed.) Handbook of research on effective electronic gaming in Education, (pp 251-276) Hershey, PA: Information Science Reference, IGI Global.

Erickson, Erik H. (1968) Identity: Youth and crisis. New York: W. W. Norton & Company, Inc.

Gee, James. (2003). What video games have to teach us about learning and literacy. New York: Palgrave MacMillan.

Huizinga, Johan. (1938/1986). Homo ludens: A study of the play element in culture. Boston: Beacon Press.

Caillois, Roger. (1962/2006) The definition of play: The classification of games. In K. Salen & E. Zimmerman (Eds.), The game design reader: A rules of play anthology, (pp. 122-155). Cambridge MA: The MIT Press.

Pintrich, P. R., & Schunk, D. H. (1996). Motivation in Education: Theory, research, and practice. Englewood Cliffs NJ: Prentice Hall.

Sutton-Smith, B. (1997) The Ambiguity of Play. Cambridge MA: Harvard University Press.

Wohlwend, K. E. (2007) Reading to play and playing to read: A mediated discourse analysis of early literacy apprenticeship" In D. W. Rowe, R. Jiménez, D. Compton, D. K. Dickinson, Y. Kim, K. M. Leander & V. J. Risko (Eds.), 57th Yearbook of the National Reading Conference. Chicago: National Reading Conference.

Physiwiki

by Jason Haas

Will conference goers use a physical wiki, & what behaviors can we observe emerging by watching during conference downtime & by studying the final artifact?

Will conference goers use a physical wiki, AND what behaviors can we observe emerging by watching during conference downtime and by studying the final artifact?

The idea to create a physical "wiki" in a public space emerged very quickly from our cards. This wiki would be like a bulletin board but governed by the rules of popular wikis such as Wikipedia. Our hope was that, with the understanding that our research base was limited (GLS conference attendees), we could see some sort of "mind map" of the conference-goers, and accordingly, the field. As wikis are touted as a great loci to host "collective intelligence" and establish participatory cultures (Educause et al., 2008; Jenkins et al., 2006), we thought that the GLS conference goers could share immediately applicable information with one another. We were all very interested in observing the emergent behavior on the board, but a major question was whether to seed the wiki with topics or to simply create the wiki and leave it empty to observe the pure emergence.

Methods

Ultimately, because the time frame for the experiment was so short (less than 3 days), we seeded the wiki with a few posts as models and with encouragement for others to make similar posts. Our initial effort to set the Physiwiki up on a section of wall needed to be modified due to the conference center's rules, so we used a bulletin board on an easel (see Figure 1). The top of the board was adorned with a playful logo and rules gleaned roughly from pre-existing Wikis. The rules were:

1. Big Posts ”" To start an ent ry.

Small Posts ”" To amend or

comment.

2. Use one consistent user

name. Put it on each entry.

3. Note the time on each entry.

4. NO removing! Crossouts only.

We provided 8.5"x11" pages for major (top-level) posts, and multicolored Post-It notes for amendments and additions. We also provided Sharpie markers to title major posts and ballpoint pens for the additional notes. The seed posts were "Kurt Squire," "Things To Do in Madison," and "Games and Learning." Our hope was that the playful logo, the rules, and the seed posts would invite interaction and set a very low barrier to participation. After those were posted, we observed and took pictures at every possible opportunity, usually between sessions and at the beginning and end of the conference days. Our photographs were of the board in order to have some sense of the emergent behavior over time, as well as of conference goers pausing and grouping at the Physiwiki. Our participants thus, were self-selected. This is true not only of the most active participants (those that made posts on the Physiwiki), but also of the low level participants ("lurkers").

None of our group members were PhD social science researchers, so our methodology for data analysis was to rely on our loose observations of the board and the area around the board. We augmented this by performing simple counts of the types of posts and performing simple codings of the post content on the final artifact. We were mostly concerned with the information brought to bear, although in some cases it was also easy to discern some attitudes of posters. Figure 1 also shows the final state of the Physiwiki.

Findings

At the end of the conference, the Physiwiki had 5 major "topic" posts, 3 of which were seed posts, and 41 smaller posts. There were 23 non-anonymous posters and 19 of those made only one post. Ten were anonymous posts. The Physiwiki seemed to be fueled by utility and fun. One important finding was that there was little response to abstract topics but concrete topics attracted attention: the "Things to Do in Madison" had the highest number of respondents and the entry on Kurt Squire (a presenter at the conference) also gathered silly/sweet commentary and inside jokes; in contrast, the Games and Learning Topic had fewer respondents. The longest conversation, however, was under that topic and was one of utility ”" making connections around ecology games. Many more small posts (additions and corrections) were created than new, top-level posts. The two top-level posts needed encouragement from Physiwiki staff.

While we do not have exact numbers, the Physiwiki attracted a great deal of attention, including repeat viewers. Given the number of posts we counted at the end of the conference though, it was apparent that there were many more readers/lurkers than there were contributing posters. Some would meet at the Physiwiki and start conversations about it. The RTR manager, positioned next to the Physiwiki, related a story that indicates how people responded to the Physiwiki:

"On one occasion I overheard someone being pulled over to the board by a friend. 'Have you seen the 'Things to do' post over here?' Both were from out of town and looking for a place to go to eat, but they stopped here to see if any ideas were posted since the one had been here last. Newly initiated, the second commented in awe, "This is simply the best thing I've ever seen at a conference."

The RTR manager also observed that people rarely asked for their RTR stickers (the official incentive that was offered) to participate in the Physiwiki. Instead, participation appeared to be a reward in itself.

Conclusions and Next Steps

The Physiwiki's success certainly has implications for bulletin boards at future conferences. It is difficult to know what made it successful, however. Using humor to lower the stakes may have increased participation and many conference goers found it to be a worthwhile endeavor. Our target audience (the GLS conference-goers) may have been more likely to playfully engage in new things, which probably meant they were more likely than the average audience to alter the traditional bulletin board through the metaphoric use of a technology like a wiki. But it is also possible that wiki-style conference bulletin boards could be successful with other communities if similarly pitched for a given conference.

Our research question was undoubtedly answered ”" conference-goers definitely used the Physiwiki. Several made posts, and many more were drawn to it for information or, at the very least, for the spectacle of it. The seed posts successfully performed their task of shaping the conversation and providing useful models; however, they also limited the conversation. And while we reduced some barriers to entry, more might have been done to scaffold new top-level posts, and to encourage even greater participation.

The conference organizers later commented that they wanted to include the Physiwiki in future GLS conferences, and it would be interesting to experiment with ways to encourage participation. What form might participation take if using a larger space or more blank top-level posts on the board, inviting others to fill them in? While the RTR stickers were generally passed over (or missed) by Physwiki participants, it would be worthwhile to find out what enticements (not undue ones, of course) could encourage lurkers to cross the threshold and become users. To drill down even further, it would be interesting to survey users and/or record in finer detail the sequence of posts and to observe readers and gawkers to determine what effects posts have on users with respect to encouraging others to participate. Further, conference goers could be asked at the end of a given conference whether the Physiwiki was a fair map of the conference. The early promise of the Physiwiki is worth improving with a series of design-based inquiries in order to yield a model information storehouse and mind map for ad hoc communities (such as conferences). Physiwiki research could even inform research into ubiquitous computing and data augmented objects, acting as a prototype for how we might successfully connect the physical, onsite collective intelligence needs of ad hoc communities to more enduring digital presences.

References

Jenkins, H., Purushotma, R., Clinton, K., Weigel, M., Robison, A. (2006). Confronting the Challenges of Participatry Culture: Media Education for the 21st Century. Chicago, IL: The John D. and Catherine T. MacArthur Foundation. Retrieved October, 6 2009, from http://www.newmedialiteracies.org/files/working/NMLWhitePaper.pdf

The New Media Consortium & EDUCAUSE Learning Initiative. (2008). The Horizon Report (2008 edition). Retrieved October 6, 2009, from http://www.nmc.org/pdf/2008-Horizon-Report.pdf

Emotional Values

of Inventory Items

in World of Warcraft

By Vance S. Martin, Lee Sherlock & Dongping Zheng

Is there a difference between lower level and higher level characters (newbies and experts) in terms of whether they held on to seemingly "worthless" items and the reasons they gave for keeping those items?

Is there a difference between lower level and higher level characters (newbies and experts) in terms of whether they held on to

seemingly "worthless" items and the reasons they gave for keeping those items?

As part of the Games+Learning+Society (GLS) Conference 4.0 in July 2008, we took part in a Real Time Research Activity. As a team decision, we chose a set of cards made up of constructivism/situated cognition, ethnography, interview and survey, and World of Warcraft (WOW). Within this general framework, we brainstormed what topics each of us was interested in looking into. Among the five original researchers, four of us had advanced at least one character to level 70 (the maximum character level at the time of our study) in WOW. One of the authors has studied (inter)actions in the virtual worlds Second Life and Quest Atlantis for over 6 years. One topic that stood out in particular was the underlying human characteristics that keep gamers engaged, either combating or working tightly together in guilds.

The ecological psychological concept of "meaning making" and "value-realizing" in human activities appeared to be sufficient and satisfying to dissipate our puzzle (Gibson, 1979; Reed, 1996), specifically, Reed's account of collective appropriation of affordances, Hodges and Baron's (1992) account of values as multiple, heterarchical and dynamical constraints on actions and interactions, Hodges and Lindhiem's (2006) account of carrying as value-realizing activity, and Hodges' (2007) account of caring to go on in conversing. Grounding our thinking in ecological terms, we shared our experiences in WOW and virtual worlds in terms of our emotional engagement, things we carry in our packs, people we have the most interaction with, and so on. One of the members mentioned he carried a worthless item, a cracked bill, in his pack because his character's first name was Bill. So we began to discuss what people might carry in their packs, which held personal value to them but had little functional value in the game (i.e., those items not directly related to the dominant terms of progress in WOW via gameplay, such as combat, active quest items, profession advancement, or in-game profiteering via selling to other players).

Methods

Similar to our switch from Constructivism to Ecological Psychology in the theoretical perspectives, we also modified both ethnographic and survey data

collection techniques to accommodate our real-time data collection in

the GLS 3-day conference. We had one and a half days to grab people on the fly during session breaks, at lunch and breakfast tables, and in the game room (GLS has a game room set up with any game you name for conference attendees to take action in playing). As a result, a qualitative short interview questionnaire in the form of short survey items (see Table 1) seemed to be suitable for the nature of the study and the context where research took place.

The questionnaire asked,

"In WOW, on your main character, name one object you regularly carry in your inventory that has nothing to do with advancing. It must be something useless in combat or combat support. One item only please."

We also asked people to give their main character's level, race, class, last login, when they got the item, and if they had ever passed on a green (uncommon) item or higher to keep it. While the question was emergent, it seemed to be focused on differences in players, their levels, and their reasons for valuing particular items.

In order to gather the data we followed several steps. Each of the five members of our group distributed surveys and asked people at the conference to either fill them out themselves or group members filled them out as they conversed with conference goers. Some surveys were left on clipboards. Participants received stickers for participating. However, gathering the information was not as straightforward as simply asking questions about a character. As Brown and Thomas (2009) have discussed, playing a character is an act of being the character, so we were touching on something personal which had often involved a large time outlay for the participants. This time outlay was evidenced by the time it took to answer the questions. Intense conversations occurred about characters, class, how items were acquired, and why they were kept. These conversations could last for fifteen minutes or more. In some cases it was deemed necessary by the participant to show the interviewer the character and item on computers provided at the conference. Thus, data collection often took the form of listening to and asking further questions about participants' recalled stories surrounding the item's history and context. In this sense, our study elicited what Gee (2007) calls "embodied stories" of video gaming in which experiential, emergent meaning is constructed based on in-game events.

After a day and a half of gathering data, 70 surveys were collected. The group sat down with the surveys and copied the items and reasons for keeping the items. Some items were excluded as they were skinning knives or mining picks, which actually do have a value concerning professions. Once these items were excluded we had 37 items and reasons listed out of 70 surveys. An example of an item and reason would be "I've been carrying this lieutenant's insignia I got in Durotar since level 8 just in case it's useful" or "I got this cool pet earlier in the game you remember at the end of year one."

Findings

We anticipated that a high-level player would have fewer items with emotional ties because bag space (at the time of this writing) is at a premium at higher levels. Stories of players having to clean out and organize their bags to prepare for a raid (large group events) ”" or worse, forgetting to do so and having to run home to a bank ”" are legendary in the WOW community. However, we found that almost all players had a few items that they held onto for various non-utilitarian reasons.

We did not find any correlation in items kept or reasons for keeping them related to level or time playing the game. Overall, the reasons were highly personal; typically, the items related to a personally important event in the game or had been kept so long as to take on personal meaning. Reasons given in the survey included aesthetics ("looks awesome" or "cute"), performances of social capital ("not many people have it" or "needed reputation to get it"), humor and amusement ("funny," "whimsical," or "humorous visual effect"), individual or group identification (relates to character name or guild affiliation), and emotional attachment ("made the game more human" or "gives me a sense of belonging"). Additionally, 35% of survey respondents reported they would pass on a green item or higher to keep the item mentioned.

It can take 240 or more hours for a player to take a character from level 1 to 70. What we did not perhaps fully grasp at the time of the study itself was the truly personal experience we were touching on. At the time of the event itself, we speculated that we could have asked different questions or perhaps shortened the questionnaire, but did not understand what we were getting at in terms of a broader concept of valuation. Were we to have added what items do you keep in your office, bedroom, or house that you have refused to get rid of numerous times, and why do you keep them, we may have come closer to understanding the participants and their rationale for hanging onto what, by all definitions, can only be considered mementos.

Conclusions & Next Steps

Our initial research question was, "Is there a difference between higher and lower level characters in WOW and the non-functional items they keep or their reasons for doing so?" In the end, there was no correlation between level, race, or time played to show why they kept an item. Most people had a worthless item, and they all had a personal story that they loved sharing to explain why they kept it. Along similar lines, Hodges and Lindhiem's (2006) study revealed that participants were rated more careful in carrying invisible children across uneven steps than grocery bags or trash. Discussing this result, Hodges and Lindhiem reflected that there are many variables that affect the carefulness rating. Among them, the motion movement can reveal something of the content of what is carried. Regardless of the observed differences between perceptual and behavioral critical action boundaries, social engagement is crucial. Social engagements, such as trust between a guild leader and guild members and the cooperation between the guild members, together with moral dimensions are important constraints on actions. In other words, the things that gamers carry in their inventories can have social impact and thus can possibly alter gaming behaviors in significant ways. A possible parallel application of their research findings to our current study might be to collect a larger N and replicate the study, potentially yielding findings that bear on the issue of whether or not the mementoes we carry around in-game make a difference in relation to our perception and action boundaries in individual questing or group battles. Another interesting question might be whether items that players carry can boost their avatar's self-efficacy. In other words, will the players feel more confident, comfortable and caring in some uncomfortable situations with these items in their inventory?

Something we did not examine in this small study was the affordances an MMORPG could have for emotional attachment. Such emotional attachment may have great implications for educators as they attempt to integrate digital technologies into their instruction. How can we elicit positive feelings in learning so that it has real import? The stories told about seemingly worthless items held value for the players interviewed just as mementoes do for many in the real world.

We believe that there would be value in repeating this study with small samples; however, there should be some revision. As mentioned above there did not seem to be a correlation with race, character class, or time played in the attachment to an item, so it appears following those hypotheses would yield little knowledge. However, perhaps asking about an item and its importance in the game as well as in the real world may create a clearer understanding of how people view their online versus real life (social) encounters and whether they perceive a difference in value between the two. It might be revealing to report cases of how high-level players perceive their longest carried items as opposed to lower level players. The aim of understanding how novice players become experts in the spirit of legitimate peripheral participation may shed light on how we scaffold novice learners in communities of practice in educational settings (Lave & Wenger, 1991)

Age or generation might be one important factor to consider in such future investigations, however. As Angela Thomas (2007) has touched on, younger people see little delineation between online and real world encounters. Thus, including age as a variable would help further interrogate the possible connections between "real" and virtual systems of value and meaning making.

Acknowledgments

This study is a collaborative effort and the order of authorship is alphabetical. We would also like to thank William Chamberlain and Lisa Nakamura's participation in the initial research, design, and data collection.

References

Brown J. S. & Thomas, D. (2009). Why virtual worlds can matter. International Journal of Media and Learning, 1.

Gee, J. P. (2007). What video games have to teach us about learning and literacy (2nd ed.). New York: Palgrave Macmillan.

Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.

Hodges, B. H. (2007). Good prospects: Ecological and social perspectives on conforming, creating, and caring in conversation. Language Sciences. 29, 584”"604.

Hodges, B.H., & Baron, R.M. (1992).Values as constraints on affordances: Perceiving and acting properly. Journal for the Theory of Social Behaviour, 22, 263”"294.

Hodges, B. H., & Lindhiem, O. (2006). Carrying babies and groceries: The effect of moral and social weight on caring. Ecological Psychology, 16, 93”"111.

Lave, J., & Wenger, E. (1991). Situated learning: Legitimate peripheral participation. Cambridge: Cambridge University Press.

Reed, E. S. (1996). Encountering the world: Toward an ecological psychology. Oxford: Oxford University Press.

Thomas, A. (2007). Youth Online: Identity and Literacy in the Digital Age. New York, Peter Lang.

Main Character Level

#

Class

Short Text

Race

Short Text

When was the last time you logged on?

# in Days [estimate]

In WOW, on your main character, name one object you regularly carry in your inventory that has nothing to do with advancing. It must be something useless in combat or combat support. One item only please.

Text

Why do you like the object?

Any length response, use back if you like.

At what level did you get it?

#

Have you ever passed up a green item [or higher] to keep it?

Yes or No

Table 1

WoW

Intelligence

by Vinod Srinivasan

Can "participatory mapping" serve as a

useful research methodology?

Can 'participatory mapping' serve as a useful research methodology?

The goal of this unique project was to come up with a research question and devise an experiment to address that question. The primary constraint was that the experiment had to be carried out during the remaining one and half days of the conference among conference attendees. Each group was also randomly given several keywords to use as the drivers to formulate the research question and the experiment.

For our experiment, we wanted to incorporate the idea of participatory mapping as a research methodology. Participatory mapping is a practice whereby participants map ideas of concern to them. The goal is to enable ordinary people to have a say in how spaces and resources around them are utilized. As a research method, this is interesting since the underlying social values run counter to the tradition of positivist research. The balance of power is shifted from professionals and experts who have dominated media discourse on various topics to those who have a more direct relationship with those topics (the "participants"). Although in this particular instance, the participants (conference attendees) were already in a position of power, the study had the potential to serve as an example that could be replicated in other contexts.

The subject of our research was World of Warcraft (WoW), the popular massively multiplayer online role-playing game (MMORPG) that has been the subject of several academic studies. Given the primary themes of the conference (games and learning) we decided to poll conference attendees on what they thought World of Warcraft taught. While this was the primary goal of our experiment, the larger goal was to investigate whether the methodology had use as a research tool in this context.

Methods

In order to minimize the barriers to participation and encourage active engagement, we devised a two-phased approach in which participants would come up with individual responses in the first phase and then deliberate on those responses collectively in the second phase. In Phase 1, conference attendees were asked to come up with a word to complete the phase 'I believe WoW teaches [blank]'. Participants wrote their responses on Post-it notes and put them up on a public board. Post-it notes of two colors were used to represent those who had played WoW before and those who hadn't. In Phase 2, the sentence was changed to 'We believe WoW teaches [blank]'. The notes

collected from Phase 1 were randomly laid out on a board along a horizontal axis indicating the level of agreement of the participant with the choice of word to complete the new sentence. Each participant was allowed to move only one note. No distinction was made in Phase 2 between WoW players and non-players.

At the end of Phase 2, the layout of the notes was analyzed to determine if there was consensus among conference participants on the choice of words. The words themselves were also analyzed to determine if there was any difference in perception between those who had played WoW and those who had not.

Findings

Our first finding was that there was an implementation problem in Phase 1. Our original goal was to restrict participant responses to single words rather than long phrases or sentences. We also wanted to keep the responses private until Phase 2. However, our instructions to participants and volunteers were not completely clear. Some participants gave multiple responses on the same note and the responses were also made public from the beginning. We made modifications to clarify our instructions, but allowed all responses to remain public.

The public nature of the responses led some attendees to pass on participating if they saw an existing response that they agreed with. It also encouraged people to gather around the board, acting as an interface encouraging participants to engage in discussion and debate.

Although the response rate in Phase 1 was very good (in terms of number of notes), we saw less participation in Phase 2, making us wonder if there was less interest, less obligation and/or participants did not see Phase 2 as a separate part of the experiment that needed everyone to contribute, even if for the second time. For those that did participate in Phase 2, observers noted that the instruction to 'move one note per person' made them 'serious', causing them to deliberate their choice carefully.

Analyzing the notes themselves, we noticed that WoW players were more opinionated in their responses and chose words like "aggression" and "obsession" to complete the sentence in Phase 1. Non-players chose words like "leadership", indicating that they may be basing their opinions on previous WoW studies that they were aware of. Interestingly, in Phase 2, the two categories of words clustered in entirely different ways. Words chosen by WoW players tended to move towards the "disagree" side of the axis, while those chosen by non-players tended to hover around the neutral zone. This would indicate that WoW players' perception of the educational content in the game is at variance with the perception of the larger community.

Conclusions & Next Steps

The fact that the participant responses were public in Phase 1 appears to have had an impact on the study. It clearly influenced the decision of some attendees to not participate in Phase 1. It could have also led to participants giving a response that was not their first choice, if their first choice was already represented on the board. Thus, keeping the responses private could also have led to a smaller range of words to work with for the second phase. In Phase 2, rather than a random distribution, the notes could have been placed in a "neutral" zone at the start. Apart from providing a 'cleaner' layout for participants to work with, this could have led to more defined clusters in the final distribution. It would also have eliminated the effect of any 'inertia' that may have prevented participants from moving a note that was already in the general area of where they thought it should be.

Overall, the study as conducted did give us interesting results as noted above. We concluded that our research method appears to be a good one to get collective opinion at a venue like a conference. However, effective implementation requires proper monitoring of the data collection and enforcement of constraints imposed on participants. In order to scale this experiment to a larger group, additional data collection stations would probably be needed. To improve the validity of the findings, additional steps would be required to ensure that the same person does not participate more than once in each phase. Further work on the method would be needed but would also be worth the exploration.

Acknowledgments

I'd like to thank to Colleen Macklin, Ben Stokes, Shawna Kelly and Tim Burke who were my partners in this project. Colleen and Ben also made contributions to the initial version of this paper.

Me, You, We:

Tweet Analysis

by Pazit Levitan & Mark Friedman

"The most fascinating thing about Twitter is not what it's doing to us. It's what we're doing to it"

(Johnson in TIME magazine, June 2009)

"The most fascinating thing about Twitter is not what it's doing to us. It's what we're doing to it."

(Johnson in TIME magazine, June 2009).

When we arrived to the Games, Learning and Society (GLS) Conference in June 2009, we expected a dominance of social network interactions. However, none of the group members expected "Twitter" to become the main communication channel throughout the conference, embracing idea exchange, stimulating real-time feedback and discussion, and acting as a game platform, all concurrent with the "traditional" learning opportunities resulting from conference interactions. Essentially, the Twitter-generated communication in GLS became a mini-conference in its own right, inviting a range of potential research and interesting observations.

Considering the three RTR cards that we received ("Behaviorism" as Theory, "Social Networks" as Topic, and "Statistics" as Method), we chose to investigate the nature of the tweet content tagged as #GLS and #GLS09. Specifically, we asked whether Twitter posts during 24 hours (one conference day from 2pm until 2pm the next day) refer to the self ("Me"), to another person's speech or action ("You"), or both ”" a tweet of a community nature ("We"). In investigating the spirit of the social network content, we sought not only to observe the type of messages participants exchanged during a professional conference but also to examine, and possibly challenge, the common perception that Twitter is a platform for excessive ego blasting, manifested in self-display.1

Methods

Utilizing TweetGrid's somewhat unknown feature of IRC (http://tweetgrid.com/irc), we were able to turn on "capture mode" for the two Twitter hashtags being used at the GLS Conference. Hashtags are a way to self-filter tweets by placing a descriptor of along with a # symbol. Users post their comments along with either #GLS and #GLS09 ”" the two hashtags we saw used at the conference. Once we had the two lists of tweets for the time period in date and time sequence, it was a matter of reviewing, analyzing and categorizing the tweets according to their content as "Me", "You", "We" or "Unidentified."

Defining these four categories did not happen automatically. Our group chatted after dinner about how to classify the huge lists of tweets, which had been captured during the conference. After exploring a few ideas we defined the parameters for each category as follows:

Our hypothesis was that most tweets would be of "Me" nature and that many of the tweets, given the free-flow spirit of the Twitter social network, would fall under the "Unidentified" category. We were wrong in both predictions, ultimately leading to our Real Time Research (RTR) Award for "Most Surprising Findings". Indeed, we were happy to be wrong, as our findings signify that Twitter enhanced the community-driven communications in GLS09.

Out of the 235 #GLS and #GLS09 tweets that we analyzed (June 10th at 2pm until June 11th at 2pm), we found that 50% (116) referred to the other ("You"), 29% (69) referred to the writer in a group ("We"), and only 18% (43) were tweets about the self ("Me"). Figure 1 provides a visual overview of these proportions.

Additionally, we identified some trends within the nature of the tweets:

These two trends show that Twitter writers are interested in posting content that is being presented during the conference in real time. On the other hand, since Gee's keynote address had a proportionally high weight in the results it would be interesting to research whether this type of "You"-dominated Twitter activity is typical for every morning session and/or whether a particular keynote speaker could "bias" the results. Additionally, it is possible that Twitter participants are more active in the morning presentations in comparison with evening keynote presentations. Such questions arose as we analyzed our findings. We agreed that would be interesting to explore through additional research.

Other interesting observations:

These two observations demonstrate that there is a difference in community engagement at different times of day as well as a difference in the community involvement process that takes place in (technologically-savvy) professional conferences, such as GLS 2009.

A bonus trend: Self-reflective tweets demonstrating aesthetic caring about the Twitter community:

"oops, sry for spam #gls09"

This type of message reminds us the Twitter mastery is sometimes developed within the context of a broader community. Here, a user who accidentally repeated a tweet is apologizing to the network collaborators for cluttering the network. We found that such an approach was more pronounced during conference sessions where participants used Twitter as a "back-channel" discussion. Overall community comfort level developed in such contexts over time.

In summary, we were intrigued to find multiple layers of patterns in Twitter use at the event. As we got deeper into the analysis, we realized that we would have liked to have extended our research beyond the place and time provided by the RTR limitations, which included: only one day of observation and only one conference as the context for research. The event itself was information-heavy, possibly increasing the proportional use of "You" category tweets. Moreover, we found a repetitive community of writers participated on Twitter, possibly not providing the full picture of the GLS conference communicators.

The most notable limitation, however, was that Twitter content was affected by a back-chatting game, which took place on Twitter simultaneously with our research. Specifically, the game incentivized players to tweet particular content in order to earn a higher score. Clearly this may have also affected the results of the "Me, You, We" research.

Conclusions & Next Steps

The "Me, You, We" research is merely a drop in the sea of possible investigations that could be done related to how social media is being used in conferences. Indeed, additional research is called for in order to examine Twitter communication and its social and cultural meaning deriving from its integration as part of professional conferences. On the other hand, our research suggests that there is emerging acceptance of the Twitter backchannel communication, exploring multi-layered interactions during the Games, Learning and Society conference (June, 2009).

With more time and resources, we would have extended the analysis to draw comparisons between different GLS09 days and between GLS and other conferences that take place around the same time and deal with similar topics (i.e. Games for Health, Games for Change, and DiGRA and Game Education Summit). This type of comparison would allow researchers to test the relationship between the "Me, You, We" categories in various conference settings, in multiple locations, cultures, and in under varying levels of integration of Twitter within the professional event itself. Additionally, breaking up categories into sub-categories (i.e. nature of the "You" content ”" is it a quote? Thought? Reply Tweet?) could clarify cause and effect relationships.

Overall, our research shows that the #GLS and #GLS09 tweets enhanced the depth of discussion around games, learning and society by allowing every writer to present their thoughts and challenge things presented officially on stage. This type of liberation and democratization of professional communication not only provides a platform to every participant (as well as those who could not make it to the conference in the first place, as seen in our "We" example above), but it also reshapes the presenter-participant power hierarchies that exist in traditional conferences.

As one conference-goer tweeted weeks after the event itself,

"One thing we noticed at #gls09 ”" if your presentation couldn't produce Twitter one liners, it did not exist." (@cstubbs, July 29th, 2009, personal communication)

How might this type of social network-driven approach to attending events affect professional conferences in the future?

Acknowledgments

We'd like to thank the other group members that did the initial research at the conference: Hal Scheintaub, Stephanie Richter, and Bonnie Saunders. It was a joy to work with them & their energy was key to the success of the activity.

1This view is common enough that we are assuming it here. For an example, see this view in blogs like Mapping The Web that make the case at www. mappingtheweb.com.

2 RT is a one-click direct reply feature on Twitter,

which is used frequently among Twitter communicators.

Table 1. Examples of each tweet category

Category

(n) 235

%

Definition

Me (self)

49

18%

About the writer

You (other)

63

49%

About someone else or event

We (community)

116

29%

About writer in a group

Unclassified

7

3%

Unclassified

Category

Example

ME

"Last final over...time to wrap up so I'm free to head to #gls09"

You

"Jim Gee: "Women's play is central to the future of gaming."

We

"Keep those #GLS09 updates coming. For those of us who couldn't be there, it's the next best thing!"

Unclassified

"can't believe Javier "hurled" rather than "tossed the candy bars... #gls"

Table 2. Proportion of each category represented in the data corpus.

All I Really Needed

to Know I Learned

by Playing Games

by Brett Bixler, Dona Cady, Maryellen Ohmberger Wendy Huang, Tanya Joosten, & Turkan Karakus

What do people say they learn

from their favorite games?

What do people say

they learn from their

favorite Video games?

As part of the Real Time Research (RTR) project conducted at the Games, Learning, and Society (GLS) conference in Madison, Wisconsin in June of 2009, our research group was given a set of criteria within which to investigate a learning phenomenon related to games. The assigned criteria were: "constructionism" as a supporting theory, "survey" as a research method, and "problem solving" as the topic to investigate.

Constructionism, inspired by constructivist learning theory and connected to notions of experiential learning (see Piaget, 1955), asserts that learning occurs most effectively when individuals are active in making things that they can share (Papert & Harel, 1991). Although our theoretical criterion was constructionism, our research was situated more in a constructivistic paradigm. In explaining the difference between constructivism and constructionism, Papert (1991) explains "[t]he word with the v expresses the theory that knowledge is built by the learner, not supplied by the teacher. The word with the n expresses the further idea that happens especially felicitously when the learner is engaged in the construction of something external or at least sharable" (p. 3). We chose a more constructivist theoretical approach in that we wanted to explore the idea that the playing of games resulted in the building of knowledge by the learner. Therefore, we surveyed our participants, GLS conference attendees, about the most important game which he or she has played and what was learned from that game following a constructivist theoretical approach in order to discover if one of those skills learned through game play would be problem solving.

Specifically, we examined what games and what genre of games influenced practitioners and academics in the field of games and learning and what areas of knowledge playing these games created. As a metaphor, we chose to design a research theme based on Robert Fulghum's best-selling collection of essays, "All I Really Need to Know I Learned from Kindergarten" (1986). We modified this theme to address what games, illustrating learning by doing, led to what types of knowledge creation (e.g., problem solving) by our participants. Being aware of the varying professional affiliations (educators, researchers, game designers) present at the conference, we wanted to explore the differences and similarities between these groups as well. Our research questions were:

1. What games have impacted GLS attendees?

2. What do GLS attendees believe they

have learned from games?

3. Is there a relationship between one's

professional identity and the types of

games played and/or skills learned?

4. What game genres were most prominent

for which professional identities?

Methods

Our methodological approach consisted of surveying participants at the conference. It can be assumed that these participants had a professional affiliation in the field of games and learning based solely on his and her attendance at the conference.

We asked each participant to first classify his or her professional identity as either: Educator, Designer/Developer, Researcher, or Other. Individuals were then directed to choose the color of the Post-it note that best matched his or her professional identity (See Table 1). Post-it notes were used as a mode of data collection due to their ability to be completed efficiently, to limit the length of the response, and to be posted on a flip chart.

Participants were directed to write his or her responses to the proposed survey questions on the selected Post-it note. Specifically, participants were given written instructions to use one word to answer each of the following questions: "All I really needed to know I learned by playing X. What is X?" and "What did you learn?" Participants then placed their colored Post-it note anywhere across the four quadrants on the chart (See Figure 1).

The chart is a Cartesian grid presented on a flip chart with the X-axis ranging from Digital to Non-Digital and the Y-axis ranging from Entertaining to Educational (See Figure 1) resulting in four quadrants: Digital/ Entertaining, Non-digital/Entertaining, Digital/Educational, and Non-digital/Educational.

We strategically requested a one-word response in order to require participants to prioritize the games that they have played, the skills that they learned in their most significant game, and the most significant skill learned in his or her overall game play. Even though participants were instructed to use one-word responses, few were able to do so when describing the skills learned from playing a game.

We created two charts and positioned them in high-traffic areas within the conference space in order to gather the maximum number of responses from participants in the limited amount of time (See Figure 2). We collected responses over a 24-hour period.

Participants were solicited non-systematically based on their proximity to the flip charts. Given the focus of the conference, it was assumed that individuals in the area wearing conference nametags were conference attendees who were also professionals in the field of games and learning.

Once the data collection was complete and due to the time constraints of the RTR project, we transcribed the data from the flip charts into a data sheet based on professional identity, games, and classification of game. Later, we coded the games based on genre (e.g., action adventure, board game).

For our data analysis, we used Wordle word clouds using the Wordle software (Feinberg, 2009) to produce visual representations of frequency data to address research questions 1 and 3. These word clouds are an effective representation of these data because they do not represent simply a collection of responses, but rather, they illustrate how the group working together influences individuals and collectively creates understanding. Wordle was the most appropriate method of analysis due to the breadth of responses and ability to produce a visual representation of the data. Frequency charts could not capture the essence of the data for these research questions or illustrate the findings concisely. To address research questions 2 and 4, we entered the data into SPSS statistical package for further analyses. We produced frequency percentage pie graphs to address research questions 2 and 4.

Results

Our participants were attendees at the Games, Learning, and Society conference (n=82). Their participation was completely voluntary. We did not collect demographic information from the participants beyond their professional affiliation. As seen in Table 2, the majority of participants classified themselves as "Researchers" (n=30), the second highest reported professional affiliation was educator (n=25), with the lowest reported professional affiliation being designer or developer (n=13). We did have a category of "other" (n=10) and we did have a few individuals that reported multiple affiliations (n=4).

When entering all data collected, including games reported and what was learned, into the Wordle software and not in any attempt to address our proposed research questions, games that were predominant in the word clouds were World of Warcraft (WoW) and Dungeons and Dragons (D&D). It also appeared that Civilization, baseball, and Risk were highlighted as slightly predominant (See Figure 3).

Prominent skills learned included "how to relax" and "patience" with other skills, such as "guts," "creativity," "strategy," and "collaboration," appearing as slightly prominent (See Figure 3). We did not find frequent reports that problem solving was a skill learned in playing games. In the following, we selected out data and continued our analyses to have a more clear illustration of the predominant games and skills to address our research questions.

Research Question 1:

What games have impacted GLS attendees?

When addressing our first research question, we entered the data, disregarding professional identity, including the games the participants reported. The word cloud as seen in Figure 4 illustrates the predominance of WoW and D&D as the games most frequently reported. Other games that were reported multiple times included Risk, Civilization and baseball. Also, we see that Sims (multiple versions) was reported frequently as well, but due to the multiple versions of the Sims game, the game did not appear from an initial analysis to be a dominant game reported.

In examining the data more closely, there were a high number of games participants reported (n=66). However, only five games, baseball, Civilization, D&D, Risk, and WoW had more than one response (See Table 3). The other 61 games only had one response each indicating a high diversity amongst games that impacted participants.

Research Question 2:

What do GLS attendees believe they have learned from games?

As Figure 5 shows, participants reported learning affective skills (23%) from games more than any other kind of skill. Management skills were the second most reported (15%) skill learned.

Research Question 3:

Is there a relationship between one's professional identity AND the types of games played and/or skill learned?

In our study, among educators, WoW is the most frequently cited game. Further, we can see in Figure 6 that "strategy," "guts" (bravery), and "collaboration are the most frequently cited skills learned.

Among designers and developers, WoW and D&D are the most frequently cited "games" and "strategy" is the most frequently cited skill learned

(See Figure 7).

Among researchers, WoW and D&D are again the most frequently cited along with "patience" and "how to relax" as the most frequently cited as the skills learned in games (See Figure 8).

Among participants classifying themselves as "other," there were no clearly predominant themes (See Figure 9).

Research Question 4:

What game genres were most prominent for which professional identities?

Fourteen genres of games were coded (See Table 4). When examining the data set as a whole, role-playing (29%) was the most dominant genre of game reported by our participants. Action-adventure (15%), simulation (13%), and board game (12%) genres followed (See Figure 10). Note that the most dominant genres (role-playing and action-adventure) also describe the game titles most frequently reported previously (i.e. WoW and D&D).

As seen in Figure 11, when examining Educators and the most frequent genre reported, both board games (23%) and role-playing games (23%) were the highest reported. Outdoor games (19%) were reported second highest with platform games being reported the least (4%).

In Figure 12, when examining Researchers and the most frequent genre reported, role playing games (13%) was the most frequent and obviously the most prominent.

Finally, (see Figure 13), when examining Designers and Developers, we again see role-playing games (6%) as the most frequent followed closely by action and adventure (5%).

Role playing games were the most prominent across the professional identities. For each professional identity, educator, research, and designer and developer, role-playing was the most frequent reported genre of game.

Conclusions

Though initially surprising, a common pattern emerged among the games most frequently reported. World of Warcraft and Dungeons and Dragons were the most frequently cited games across all categories of profession. In retrospect, however, such results are not all that surprising. World of Warcraft is the most popular MMORPG (Massive Multiplayer Online Role Playing Game) to date, boasting more than 11 million subscribers as of 2008 (Blizzard Entertainment, 2008). Dungeons and Dragons is the most famous non-digital role playing board game ever created (Waters, 2004). It is no wonder, then, that these two game titles would be more frequently reported than any other game.

The findings of these frequencies should not be overstated. There were over 66 games reported with only baseball, Civilization, D&D, Risk, and WoW having more than one response with baseball, Civilization, and Risk only having two responses each. WoW and D&D were overwhelmingly the most frequently reported, but more importantly may be the number of different games that participants uniquely reported, 61 uniquely reported games. It is evident that games have an impact on our participants' development of skills, but there is no clear evidence that any one game is the leader. The categorization of genre was then needed since no clear evidence could be drawn from the name of the game alone.

These data suggest that, while more or less everyone, regardless of profession, reports an array of games highlighting two popular games (i.e. WoW & D&D), what, specifically, individuals report having learned from them does vary somewhat based on professional identity. In fact, we were surprised at the vast number of individualized and highly nuanced "skills" identified by participants. For example, educators reported learning "strategy," "guts," and "collaboration." Designers and developers reported learning "strategy," and researchers reported learning "patience" and "how to relax." This diversity could be attributed to the fact that participants often used more than one word to describe what they learned from playing a game. It is, indeed, interesting that participants had difficulty expressing what they learned in only one word. Perhaps this suggests that what is learned from games deeply resonates on many levels and is hard to precisely define.

It could also be that professional areas of expertise color perceptions of games via the affordances participants perceived in the games. Affordances are features the individual perceives in an environment that can be manipulated towards a desired end (Gibson, 1979) leading players to bring their real selves into a game (Gee 2007). Thus, different people will see the same game in different ways, take different actions, and possible learn very different things. In other words, because games are interactive and individuals perceive them in differently, what is learned from a game is not consistent across all people and all game play experiences. However, more research is needed to clarify these possibilities and the influence of professional identity.

In this project, the most frequently reported types of skills were affective skills followed closely by managerial skills ("leadership," "how to run a business"). On the surface, the finding that many people learned affective skills from games is not surprising in that much of game play taps into strong emotions ("fun," "fear," "excitement," "frustration,"), however, the wide range of affective skills reported suggests that playing games is somewhat of an introspective and personal adventure, regardless of how collaborative or public the game may be. Both WoW and D&D are intrapersonal, communicative and collaborative games, yet the skills participants report having learned from them are first and foremost introspective and personal and only second managerial and social.

It is not surprising that role-playing was the most reported genre of game across professional affiliations. Role-playing has been identified as a strategy for constructivist learning for years due to the experiential nature of role playing and its ability to not only promote cognitive learning, but also promote behavioral and affective learning (See Moradi, 2004; Smith, 2004). The known outcome of role playing as facilitating affective learning can also help us better understand the high reports of affective skills by our participants. Since most of the participants were reporting role playing games, it is only natural that they would also be reporting affective skills learned by playing those games, which can also lead us to better understand why there is sometimes resistance in educational institutions to implement games for learning. If the primary skills learned by playing games are affective and managerial skills, neither of these are tested to determine an educational institution's success or effectiveness. Therefore, these schools have no motivation to implement experiential activities, such as role playing games, since it does not directly impact the measured outcomes of student performance on math, English, and reading in standardized testing, although a conclusion can be drawn that affective and managerial skills can be gained by playing games and are pre-requisite for certain professions (educators, researchers, and more).

These research findings suggest that, when prompted, game players can and do report having learned specific skills from the games they play supporting a constructivist theoretical foundation of learning by doing.

Implications for

Future Research

The topic of problem solving in gaming needs further investigation and could be well served by taking a constructionist perspective. Follow-up studies could investigate the pros and cons of this approach, by devising one study similar in nature and methodology to this one, and another where all participants' contributions were done in the blind. Comparisons between the groups on pattern swarming (where later participants follow along with previous participants in a "me too" pattern), uniqueness of responses, and time for patterns to emerge could be performed.

Personal observation of the activity of the participants indicated that approximately 50% of participants contributed "blindly" to the study, not reading previous participant's responses. The other participants did browse other's responses, sometimes commenting to the researcher on their thoughts about previous participant's responses. Also, some individuals perused other's responses then left to "think about it," returning later to participate. The difference in time delay between those who blindly and immediately participated, those who perused the board then participated, and those who perused the board, left for a period of time ranging from several minutes to a day, and then returned to participate, could have introduced a variable that is not accounted for in this document. Or, making the data collection private could control this variable.

The broadly reported games identified by our participants are interesting phenomena. Although there were some popular games reported with some frequency, there needs to be additional research in understanding why so many different games impacted our participants.

With the identification of role-playing games as the prominent genre of games across professional affiliations, we urge continued research into the impact of role-playing games on learning. Specifically, an investigation of the skills learned from role-playing, such as affective and managerial skills, on student success would be viable research.

We do not know clearly, based on these data alone, however, if the skills learned "transfer" to real-world situations in some way. The research on role playing would confirm this idea, but further research on the transfer of skills learned by role playing in games on all four of our quadrants is still needed.

References

Blizzard Entertainment. (Oct. 28th, 2008). World of Warcraft surpasses 11 million subscribers worldwide. Retrieved September 10, 2009, from http://www.blizzard.com/us/press/081028.html .

Feinberg, J. (2009). Wordle. Retrieved June 12, 2009 from http://wordle.net .

Fulghum, R. (1986). All I really need to know I learned in kindergarten: Uncommon thoughts on common things. New York: Villard Books.

Gee. J. P. (2007). Good video games + good learning. New York: Peter Liang Publishing, Inc.

Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.

Moradi, B. (2004). Teaching about diversities: The shadow/role-play exercise. Teaching of Psychology, 31(3), 188-191.

Papert, S. (1991). Mindstorms. Children, computers and powerful ideas. New York: Basic books.

Papert, S. (1991). Situating constructionism. In I. Harel & S. Papert (Eds.), Constructionism. Norwood, NJ: Ablex Publishing.

Piaget, J. (1955). The child's construction of reality. London: Routledge and Kegan Paul.

Smith, N. S. (2004) Teaching for civic participation with negotiation role plays. Social Education, 68(3), 194-197.

Waters, D. (April 26, 2004). What happened to Dungeons and Dragons? BBC News Online. Retrieved September 10, 2009 from http://news.bbc.co.uk/2/hi/uk_news/magazine/3655627.stm.

Post-It Color

Professional Identity

Blue

Educator

Green

Designer and/or Developer

Pink

Researcher

Yellow

Other

Table 1: Correspondence Between Post-it Note Color and Professional Identity

Professional

Affiliation

Counts

Percentage

Researcher

30

36.6

Educator

25

30.5

Designer or Developer

13

15.9

Other

10

12.2

More Than One Reported

4

4.9

Total

82

100.0

Table 2: Frequency Table: Professional Affiliation

Genre

Count

Percentage

Strategy

2

2.4

Simulation

11

13.4

Role Play

24

29.3

Puzzle

5

6.1

Platform game

1

1.2

Outdoor game

6

7.3

NotCoded

3

3.7

Intelligence games

2

2.4

First person shooter

1

1.2

Fight

1

1.2

Exergame

1

1.2

Card game

2

2.4

Board game

10

12.2

Action - Adventure

12

14.6

3D Virtual Environment

1

1.2

Total

82

100.0

Table 4: Game Genre Frequency

Losing Track of Time in the GLS Arcade

by Anythony Betrus, Janet Beissinger,

Greg Casperson & Yoonsin Oh

How does a player's perceived

game-playing time compare to a

player's actual game-playing time?

How does a player's perceived game-playing time compare to a player's actual game-playing time?

This "Real Time Research" (RTR) study was conceived at the May, 2009 Games, Learning, and Society (GLS) conference at the University of Wisconsin-Madison.  Our inquiry started when one of our group members (Betrus) read the description of the RTR session on the way to the conference. After some thought, he brought the idea of observing GLS attendees as they played a game to our randomly chosen group  at the RTR session.  The group expressed a mutual interest in Csíkszentmihályi's (1990) theory of flow and discussed how we could do some basic measurement and observation to determine whether players had entered a flow state while playing.

Just as we were thinking about which game we would set up and where, one of the conference organizers (Steinkuehler) made the timely suggestion to use the games already set up in the conference arcade. Our UW-Madison hosts had set up a giant dream arcade with games of every different shape, size, and variety (along with free flowing kegs and unlimited ice cream). We all agreed that observing people in the arcade was a good idea.

For our RTR experiment, we decided it would be relatively easy to ask some simple questions about players' perceptions of their experiences and to gather some basic demographic information after they finished playing a game. We found, among other things, that players were miscalculating their time played 95% of the time ”" a significant amount even for a quick test like ours.

Literature Review

& RTR Questions

Csíkszentmihályi's (1991) book, Flow: The Psychology of Optimal Experience, is the seminal work in the area of flow.  In it, Csíkszentmihályi describes "flow activities" as supporting enjoyment, and gave examples of play, art, pageantry, ritual, and sports. He then explained that flow activities "...transformed the self by making it more complex.  In this growth of the self lies the key to flow activities."

The achievement of flow through an appropriate balance between Anxiety and Boredom has since become a commonly accepted goal among researchers and scholars interested in improving the teaching/learning environment through the use of games. Csíkszentmihályi stated that:

"Although the operationalizations of flow diverge from one another, almost all flow measuring instruments include the challenge”"skill dimension that has been argued to be the most important flow antecedent " (Csíkszentmihályi, 1991, p.191).

Kiili (2006) divided the conditions described by Csíkszentmihályi (1991) into antecedents and the experience itself. Kiili (2006) outlined the antecedents as, "...challenges matched to the skill level of a player, clear goals, unambiguous feedback, a sense of control, playability, gamefulness, focused attention, and a frame story." He sought to correlate these antecedents with the indicators of flow experience: "...concentration, time distortion, autotelic experience, and loss of self-consciousness" He concluded that that: 

"The flow antecedents studied—challenges matched to a player's skill level, clear goals, unambiguous feedback, a sense of control, and playability—should be considered in game design in order to produce engaging and enjoyable experiences for players" (Kiili, 2006).

In other words, he concluded that the basic descriptive characteristics of flow outlined by Csíkszentmihályi (1991) could be used as prescriptions for creating learning games that support flow experiences. In his conclusions he went on to state that:

"The results of the study supported the assumption that the concentration, time distortion, autotelic experience, and loss of self-consciousness dimensions can be considered indicators of the flow experience. The interplay of these dimensions facilitates the flow level experienced by players. Furthermore, the results indicated that the flow experience was independent of gender, age, and prior gaming experience" (Kiili, 2006).

Csíkszentmihályi (1991) described losing track of time as a common description of flow experience. He explained that most people mentioned time went faster than it actually did, but there is also the opposite case, and used an example of a ballet dancer who thought time went slower while performing a difficult turn. He concluded from his observation that, "The safest generalization to make about this phenomenon is to say that during the flow experience the sense of time bears little relation to the passage of time as measured by the absolute convention of the clock."

Based on our understanding of flow, we suspected that some people would enter a flow state in which time perception becomes distorted. For our study, we focused somewhat narrowly on this particular aspect of flow, which is about losing track of time. Our primary research question was "How does a player's perceived game-playing time compare to a player's actual game-playing time?" We further compared that to basic information about the person playing and the game he or she was playing. We looked at whether they lost track of time, the game was challenging, and whether players had fun while playing. 

We hypothesized that as players played for longer periods of time, they would be more likely to enter a flow state and would therefore evidence a correspondingly greater discrepancy between perceived time and actual time played. We also wondered if the aggregated data would reveal some sort of statistical "break point," where before that point there would be less time distortion and after it there would be greater time distortion.

Research Methodology

We conducted our research in the GLS arcade throughout the conference (2 days, 2 evenings). Participants were a convenience sample chosen from those playing in the GLS arcade. They were observed without their knowledge and clocked from the time they started playing a game until they quit the game. Immediately afterward the players were asked to estimate the amount of time they had just played the game. Then they were interviewed based on a short seven-item protocol (Appendix A) that included questions about whether they enjoyed the game and whether they found it challenging, as well as their age, gender, and prior game experience. We also asked whether they had checked the time during game play to know whether their estimate was a true guess or based on a clock.

Results

Given the limits of the study and the difficulty in controlling variables, over-analyzing the data would not be appropriate or useful. We looked mainly for general patterns.

Here is some basic data about the participants:

”  25 males & 15 females participated

”  Age range was 8 to 54 with a mean of 33.2.

”  Games played: Rock Band, Dance Dance Revolution, Conspiracy Code, Flower, Samba De Amigo, Guitar

Hero, Team Fortress 2, & Left 4 Dead.

”  50% were playing a game they had never played before.

”  On average, players rated them- selves a 3.0 on a 5-point Likert scale, with 1 being a non-gamer and 5 being a hardcore gamer.

We found that most players (84%) estimated time played by guessing, while 16% used some other reference to help estimate time such as counting how many songs played in Rock Band and multiplying it by average pop song length of three minutes and thirty seconds, glancing at time during or just after play, or estimating based on when a previous conference session ended and the next started. So for most players, the time reported reflects their own perception of time.

In regards to our main research question of "How does a player's perceived game-playing time compare to a player's actual game-playing time?" 47.5% of participants underestimated the time they played, 47.5% of them overestimated, and 5% of them answered exact playing time (Figure 1). The range of difference in perceived time went from an underestimate of 15 minutes to overestimates of 70 minutes, and the average player was off in their estimation by 39%. We found the average absolute time difference between perceived and actual time was 9 minutes 4 seconds. However, we did not see any pattern between actual played time and this time distortion (longer play did not seem to correspond with greater or less distortion).

Of the three most commonly played games, Rock Band players underestimated time played by 17%; Conspiracy Code players underestimated 23%; on the other hand Dance Dance Revolution players overestimated by 22%. Players who did not think the game was challenging underestimated time played by 2%. On the other hand, those who found the game challenging overestimated time played by 11%.

Other findings: 75% of participants were playing in a group (2 or more), 75% percent of participants rated their game as fun, and 75% rated their game as challenging.  57.5% found their game to be both challenging and fun.

Conclusions &

Future Research Questions

Although our initial goal was to investigate whether players experience a flow state while playing games, we are limited in what we can conclude. There were so many uncontrolled variables in our study that it is hard to attribute errors in time reporting necessarily to flow. While some players seemed to engage in the games, the testing environment hampered this possibility for others. Players had constraints of upcoming sessions or social distractions from colleagues or others in the gaming environment. Additionally, half were playing the game for the first time, which may affect ability to reach a flow state. Finally, we do not know how well participants would be able to estimate time passage during other activities. Correctly estimating time might just be a difficult thing to do regardless of the activity.

We were hoping to observe whether players entered into a state of flow, primarily comparing their perception of time played with their actual time played. We found in general that players did not accurately report their time played (95% of participants), and they did have a distorted sense of time. We found it surprising that only 5% accurately estimated time played and those players' time estimates were an average of 39% off. These findings may have been inflated somewhat due to some shorter game play times for which estimates were often rounded to the nearest 5-minute increment. However, even in longer playing situations, there were similar differences between times played and estimated time played. Either people entered a state of flow rather quickly in game play and lost track of time or else people have a poor sense of time in general.

It is interesting that the game that required moderate physical activity (e.g., Dance, Dance, Revolution) was also the game that had the highest overestimation of time played (by 22%), and that in general as players rated games more challenging they overestimated their time played. We wonder if this is similar to the case Csíkszentmihályi (1991) described in which a ballet dancer's perception of time was slower while performing a difficult turn.

Although all players who played Dance, Dance, Revolution reported that the game was challenging, there were mixed reports from players on overestimation and underestimation compared to their actual play time. It would be interesting to see if there is a relationship between overestimation of time and increasing challenge level of an activity. In future studies one might start with the assumption that flow is not an absolute, but a relative concept.  In other words, a player could be at the very limits of flow, just before the challenge of the game increases to the point where it pushes the player from a flow state to a state of anxiety.  Alternatively a player could be in a flow state on the verge of boredom. In any case, the intersection and relationships among skill level of the player, the challenge of the activity, and time distortion is certainly an interesting area to examine in future research.

We are also interested in finding if there are differences in the people we talked to, such as background, immediate contextual variables, or personality that, if measured, could predict whether someone would overestimate or underestimate their time played.

"Do people engaged in video games lose track of time?"  "Does the time distortion change (increase or decrease) if they enter a flow state?" 

"Does a person's perception of time while playing video games differ any more or less than their perception while doing other more mundane activities?"

We would also observe players in more natural settings. We would control for variables in our sample, such as game genre, actual amount of time played, and prior experience with game. We would also need to determine how accurately participants keep track of time doing other activities.

To finish this study, our RTR team met every day during conference, informally in the morning and afternoon, and formally in the evening. We spent one particularly long night analyzing data and preparing our presentation. While this was not what any of us had in mind when we went to the conference, somehow, the sense of accomplishment we got from working together made it worth the time and effort.

We focused on generating research questions and producing results that could lead to future research. For you, the reader, we hope the process worked. Through our reflections of the deficiencies in our research process, we are in turn identifying potential areas of inquiry to be explored. Ultimately, in our inquiry we were seeking to determine the circumstances and factors responsible for getting people into a flow state and similarly to look at what keeps it from emerging. In the end we hope that our study helped to accomplish the muse-like goals of the RTR project itself ”"that is, to foster dialogue and conversation about research in the domain of learning games and to propose next-step research questions and areas of inquiry.

Acknowledgments

We thank everyone who participated in our research at the GLS 5.0 conference. We also thank to GLS committes who had basically set up a giant dream arcade with games of every different shape, size, and variety (along with free flowing kegs and unlimited ice cream). It was simply amazing. We appreciate organizers for the real-time research, Eric Zimmerman, Constance Steinkuehler, Kurt Squire, and Seann Dikkers for making this research happen. They were warm, friendly, responsive, and really do have genuine interest in our collective studies. Their feedback and support for the research helped us to finish this study within such a short period time. And finally, thanks again to Seann Dikkers for his very gentle proddings to get the chapter drafted.

Contact Information

Anthony Betrus (State University of New York at Potsdam, betrusak@potsdam.edu), Janet Beissinger (University of Illinois at Chicago, beissing@uic.edu), Greg Casperson (Michigan State University, gcaspers@msu.edu), Francois Emery (Ubisoft, Francois.emery@yahoo.fr), and Yoonsin Oh (University of Wisconsin-Madison, yoonsinoh@gmail.com).

References

Csíkszentmihályi, M. (1991). Flow: The psychology of optimal experience. New York, NY : Harper Perennial.

Kiili, K. (2006). "Evaluations of an Experiential Gaming Model". in Human Technology,Volume 2 (2), 187-201.

Wii Observe

by Carol Rees, Laurie Hartjes,

Yoonsin Oh, Amy Adcock & Kae Zenovka

What roles do players take in offline

social interactions?

What roles do players take in offline social

interactions?

At the first Real Time Research (RTR) session at the Games, Learning and Society (GLS) conference in July 2008, we were invited to design a research project that would be conducted, analyzed, and presented by the conclusion of the two-day conference.  We accepted this challenge as a group of strangers with diverse disciplinary backgrounds, which included a grade school science teacher, a college professor, an instructional designer and researchers from the fields of nursing and education.  In order to begin the process, index cards representing theoretical perspectives, research methodologies and data analysis techniques were selected randomly by each RTR group. Using the selected cards as a starting point, we began our quest for a research topic by seeking common interests in gaming research. Not surprisingly, we all shared a passion for games and learning.  This collective interest led us to a conversation about the games that we had observed and/or played in the GLS arcade the night before. We were curious about the apprenticeship (Lave & Wenger, 1991) that seemed to be developing around some of the games and how the GLS arcade offered an easily accessible venue for observing the sociocultural nature of learning (Vygotsky, 1978), including the conversations that go on around games (Squire & Jenkins, 2004).

Much of the research done on the topic of learning through social interactions during game play examines the learning that happens when individual learners interact with other people in an online gaming space (Nardi, Ly, & Harris, 2007; Steinkuehler, 2004; Thomas, 2009). Fewer studies have described the apprenticeship around games in offline environments (Reed, S., Satwicz, T., & McCarthy, L., 2008). We decided to take advantage of the opportunity offered by the GLS arcade to conduct an RTR study of offline learning through social interactions around games.

Our initial research question was:  How do conference participants experience offline social interactions in the GLS arcade and learn through them? We further refined this to two more specific questions:

Methods

Choice of Methodology

For our RTR project, we were asked to select a research methodology from one of several possibilities offered to our group. We selected "phenomenology" as the best fit because in a phenomenological study the researchers begin with a question that is important in their own lives (van Manen, 2002a).  It was our shared learning experiences interacting with some of the new games in the GLS arcade that motivated our inquiry. Phenomenology is by its nature is not a methodology to be rushed, but our time constraints dictated a compressed reflection and validation process. We collected data for the purpose of reflecting on the meaning of these experiences for participants.

We chose two methods for gathering information for our study that are consistent with a phenomenological approach (van Manen, 2002b).

--

Choice of Games

We limited our study to a subset of the games available at the GLS arcade. We selected the games, Wii Sports and Wii Fit (released 2 months before the conference), since they were new, drawing larger numbers of participants, and we had all enjoyed them personally.

Selection of Participants

Study participants were individuals playing Wii Sports or Wii Fit or individuals observing or waiting their turn to play the game. Participation was largely limited to breaks between conference sessions with a few running over into the start of a new session. We selected all willing participants who were available for conversation during the limited time frame.

Definition of Player, Lurker, and Mentor

Players were actively involved in the game play; lurkers were observing others play, but were not actively involved in the game. Mentors were guiding other players either through conversation as lurkers or through joint game play with other players. 

Data Collection

In order to structure our observations and to document our findings, we constructed a data collection sheet (See Appendix A). We collected data between 10:30 am on Day One of the Conference and 3pm on Day Two of the Conference, primarily using session breaks, when the population in the GLS arcade was the most active, to make observations and conduct interviews.  During session breaks, one or two researchers from our group approached conference attendees who were either playing or observing at the Wii Sports or Wii Fit area. We explained our RTR project and then invited the conference attendees to become participants.

To address our first research question we engaged in close observation of the game play, this sometimes led researchers to involve themselves in the game play (van Manen, 2002b). We closely observed participants and classified them as players, lurkers, or mentors. We also noted the conversations that were occurring between participants as they involved themselves in the game.

To address our second research question, we conducted brief interviews with participants (see questions in Appendix A), asking players and mentors to indicate their comfort level with the game in the following simple terms: level 1 = very comfortable, level 2 = okay, and level 3 = frustrated.

We compiled the data from our data sheets by first counting the number of participants who were players, lurkers and mentors. Next we counted the number of participants who had assigned themselves to each comfort level. Finally, we examined the interview data for themes, which were established by consensus after reviewing our interview notes  (Appendix A).

FINDINGS

Of the 400 GLS conference attendees, 35 (8.8%) participated in this study. The age range was 19-64 years.  When examining behavior in this public social context, our data showed that 20 participants were players only, 13 were lurkers, 2 were mentors (see Figure 1). Interestingly, both mentors were also playing the games we observed, although for ease of category assignment, we did not include them in the player category.

In our player category, eighteen out of twenty players identified their level of comfort with the games (see Figure 2).  Of these 18 players, eight were very comfortable with this game experience, seven felt okay, and three felt frustrated. Both mentors classified themselves as very comfortable.

One of the questions that lurkers were asked is whether they planned to play. Of the 13 lurkers, three planned to play, seven preferred to watch, two said maybe, and one did not answer.

To elucidate themes from our interview data, we focused on the player and lurker roles because we collected more data from individuals in these two roles.  Players described their experience predominantly in the terms shown in Table 1; it was fun, cool, engaging, and learning was intuitive in most cases (Table 1).

In addition to these general comments, specific comments on Wii Sport - Golf and Wii Fit were collected. Responses to Wii Sports - Golf were mixed with some players describing it as fun while others described it as frustrating, remarking that it was not like the real sport or otherwise expressing dislike for the title. Responses concerning Wii Fit were more consistently positive although one player found the balance board in Wii Fit - Ski Slalom uncomfortable. Examples of comments from lurkers on the reason they watched but did not play included: no time, liked watching, preferred to watch, just observing, heard about it, want to see what was new, play it all the time, not interested, interested in other games, sometimes watch, sometimes play.  Players were observed encouraging lurkers to play with mixed results, while mentors offered tips and comments to the players that facilitated learning new skills and reinforced successes.

CONCLUSIONS & NEXT STEPS

Conclusions can be drawn from both the micro-level about what can be learned from the findings of this one example of an RTR study and the macro-level concerning the feasibility of doing RTR at a gaming conference more generally. Although the sample size in our RTR study was small (n=35) and our data were preliminary, we had three findings that lead to ideas for further work. First, we found that all mentors were players, but not all players were mentors. This raises interesting questions about the characteristics and motivations of those players who were also mentors. Second, another interesting finding concerns the different experiences of players with the games. For example Wii Sport-Golf elicited contrasting reactions (intuitive versus frustrating) from different players. It would be informative to extend the study and investigate explanations for these different reactions to this game. Third, an unexpected finding from our study was the high proportion of lurkers (13 out of 35). This finding suggested another interesting research avenue for further investigation on lurking.

We began this study with the objective of observing how participants responded to a recently released game in the GLS arcade, however we did not collect data focused on what study participants learned.  Next steps for research would be to conduct a study of the teaching/learning process in a gaming arcade such as this one. Additionally, we noticed that a fairly consistent population of conference attendees entered the arcade during the break periods over this 2-day time frame. We would like to explore the reasons reported by conference attendees for entering or not entering this on-site play space.

On the macro-level, our process demonstrated that five professionals, previously unknown to one another, are capable of pulling study design characteristics "out of a hat" and then doing rapid prototyping to arrive at specific methodology and population of interest.  We see value in doing RTR rapid prototyping in providing a forum for informal knowledge generation, in collecting preliminary data that can be used to generate new research questions, and in enabling feasibility testing for various methodologies.  For example, we discovered that individuals in the GLS arcade were willing participants in a study of game play, easily accessible, and generally open in their behaviors while engaged in game-play.  This made the experience of collecting data less cumbersome and more meaningful for the investigators and provided an added benefit to their conference attendance. The nature of the RTR process is highly creative, collaborative, and it offers opportunities to ask questions that might otherwise not arise in game and simulation research.

References

Lave, J., & Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation (Learning in Doing: Social, Cognitive and Computational Perspectives). New York, NY: Cambridge University Press.

Nardi, B. A., Ly, S., & Harris, J. (2007). Learning conversations in World of Warcraft. Paper presented at the System Sciences, 2007 HICSS 2007. 40th Annual Hawaii International Conference on Education.Waikoloa, HI. 

Reed, S., Satwicz, T., and McCarthy, L. (2008). In-Game, In-Room, In-World: Reconnecting Video Game Play to the Rest of Kids' Lives. In K.Salen (Ed.) The Ecology of Games: Connecting Youth, Games, and Learning. The John D. and Catherine T. MacArthur Foundation Series on Digital Media and Learning. Cambridge, MA: The MIT Press, pp. 41”"66.

Steinkuehler, C. (2004). Learning in massively multiplayer online games. Paper presented at the Sixth International Conference on Learning Sciences, Santa Monica, CA.

Squire, K., & Jenkins, H. (2004). Harnessing the power of games in education. Insight, 3 (5), 7-33.

Thomas, D. (2009). Scalable learning: from simple to complex in World of Warcraft. On the Horizon, 17(1), 35-46.

van Manen, M. (2002a). Writing in the Dark: Phenomenological Studies in Interpretive Inquiry. Althouse Press, University of Western Ontario.

van Manen, M. (2002b). Inquiry: Observing experience Retrieved September 11, 2009 from: http://www.phenomenologyonline.com/inquiry/29.html.

Vygotsky, L. S. (1978). Mind in Society: Development of Higher Psychological Processes. Cambridge, MA: Harvard University Press.

Table 1. Terms used to describe theirWii Sport & Wii Fit experience

General Comment

Wii Sports (Golf)

Wii Fit

Easy

Fun

Tiring workout

Familiar

Enjoyable

Good. Accurate.

(Body Test)

Comfortable

Intuitive

Easy, Cool, Neat (Yoga)

Relaxed

Challenging

Uncomfortable on the balance board (Ski Slalom)

Fun

Frustration

Love it, like it

Don't Like That

Entertaining

Different from real game.

Cool

Exciting

Interesting

Motivating

Challenging

Intriguing

Appendix:

"Wii Observe"

Data Collection Sheet

Gender: Male / Female

Age:

Sport/Game Played:

Comfort Level:

1 (very comfortable)  2 (okay)       3 (frustrated)

Player / Lurker / Mentor:

Observation notes:

(Space allowed for notes on sheet)

Interview Questions:

Players

1) Have you played Wii before?

2) Played this sport in real life before?

3) 1-3 words on their experience

Lurkers

1) Played Wii before?

2) Are you planning on playing?

3) Why or why not?

Viral Notebooks

by Michelle Aubrecht, Yoonhee Lee,

& Monica Martinez-Gallagher

What happens when viral notebooks

are used as a research collection

method at a conference?

WHAT HAPPENS WHEN VIRAL NOTEBOOKS ARE USED AS A RESEARCH COLLECTION METHOD AT A CONFERENCE?

This was the question that we formed after we much discussion. Determining our question was difficult for our group. We had three cards to help guide our formulation of a research question: Activity Theory, Social Interaction, and Ethnography. Michelle wrote ideas on a large piece of paper as our team threw around ideas. We began to focus on data collection methods and lit upon the idea to use small notebooks, each with a question written on it, and ask people to answer a question on the notebook and then pass it along to someone else. This method of collecting data used the idea of social interaction and was a not to general ethnography (participant journals). Theory was not a strong component of our research question.

METHODS

We wrote two questions on five notebooks each for conference participants to answer. The topic of the question, albeit interesting, was not relevant to our research question per se. Rather, we wanted to focus primarily on the data collection method itself. Since we were attending a conference about games, we assumed that many of the attendees would be game players. The Questions:

1. What character class do you usually choose and why?

2. What is your favorite game and why?

There were ten notebooks divided into two piles of five, with question one on one set and question two on the other set of five. We wrote the question on the top half of the cover of the notebook. The directions were written on the lower half of the front cover and read: "Please return to the GLS info desk by 9 pm Thursday." A box was placed on the counter at the conference's information desk for notebook return.

It was the first night of the conference (Wednesday) during a poster session. Our notebooks were ready. Monica and Michelle each answered a question in order to provide an example. Monica and Michelle then went to the poster session area and asked people to take the notebook and answer the question, giving no further instructions.

The following morning (Thursday), we noticed that there were no notebooks being passed around. We found some in the game room. By the following evening (Thursday evening) before the plenary session, Michelle gathered as many as she could find because none had been returned to the designated box.

Rather than being passed around, the notebooks appeared to have been abandoned and left in various places or lost. Because so little data had been collected, (the notebooks were nearly empty at that time), we decided to deviate from our original protocol and directly approached approximately 15 people to take a notebook with Question #2 on it and answer the question. We waited while the person answered the question and then Yoonhee took the notebook back.

That evening, we gathered to tabulate and analyze the data we were able to obtain. For each of the two questions posed to participants, we counted the number of notebooks returned and noted how many responses were in each notebook. We also considered the number of responses relative to the number of conference participants.

FINDINGS

Estimating the total number of GLS attendees to be around 300, we received responses from approximately 16% of the attendees (see Figure 1). If we subtract the 15 solicited responses, then that participation drops to approximately 11%.

Generally we found that when we asked people to answer the question in the notebook, they were very willing to participate. We did, however, ask people who appeared to be relaxing or standing around. Table 1 shows the overall response rate. Actual responses can be found in the Appendix.

Question

# Books

Returned

# Responses

#1 Character Class & Why?

2

10

#2 Favorite Game & Why?

3

37

Total

5

47

The fact that, when asked directly, individuals were consistently willing to respond to the questions indicates that there was some degree of willingness to participate in this research project, yet overall we found that people were not willing to pass the notebook along to someone else or to return them to their drop box.

CONCLUSIONS & NEXT STEPS

In conclusion, we found that asking people to take responsibility for a notebook during a conference was unsuccessful, perhaps because it was perceived as interfering with the conference attendee's reasons for being there. While willing to participate if asked, they were not willing to pass the notebook or return it. It was interesting however, that they were willing to write in it and leave it on a table. There could be other explanations for why the data collection method did not work. Perhaps participants needed better directions. Maybe the notebooks were too plain looking or people needed an incentive to participate.

These speculations led us to ask: How do viral notebooks become a game? What is fun about anonymous interaction? Why would someone participate, or not? Would more playful & intriguing questions make a difference? Perhaps, making the covers more colorful or decorating in them in someway to generate interest would have made the project more successful.

Would this make a good research study on a large scale or for follow up? After looking over the data collected, we concluded that although there maybe ways to make this a more viable project, the reason for collecting the data should be a more integral part of the project. The lesson learned is that personal requests for information are effective in gathering data. Asking people to take responsibility at a conference for passing a notebook is not. We concluded that we made too many assumptions about how our questions would be interpreted and about people's willingness to participate.

Appendix:

"Viral Notebook"

Raw Data (for the lulz)

Q1: Which Character class do you

usually choose and why?

Q2: What is your favorite game & why?

Tweeting GDC

by Jesper Juul, Mia Consalvo,

John Sharp & Ulrika Bennerstedt

What do people talk about on the

backchannels of the 2009 GDC conference?

What do people talk about on the backchannels of the 2009 GDC conference?

Based on a shared interest in the public discourse around conferences, our Real Time Research group decided to investigate the "backchannel" of the 2009 Game Developers Conference (GDC). We chose to focus on the micro-blogging service Twitter and the stream of commentary generated using the #GDC and #GDC09 hashtags. Because we anticipated a large amount of data and had little time to review and analyze, we recognized that the traditional data analysis tools we planned to use, including manual analysis and bar chart visualizations would not be sufficient given the large quantity of data captured and the short time three-day time frame for the research project. We therefore targeted Wordle as a powerful means of presenting the data drawn from the stream of tweets captured over the five-day period of our research project.

Methods

As the data that we gathered are based on short (up to 140 characters) text-based postings on the micro-blogging service Twitter (http://twitter.com/), we needed to manage these to answer our research question. In order to do this we started to define which posted messages, also known as tweets, should be included in our research. A common practice by conference participants is to include in tweets related to a conference in order to make clear that their tweet was about the given conference. In the case of GDC 2009, there were two commonly used hashtags: #GDC and #GDC09. Those interested in following the Twitter backchannel of the event knew to search Twitter using these hashtags in order to follow the tweetstream for the conference.

We focused on tweets including the #GDC and #GDC09 hashtags in our data mining. In this way we used Twitter's own search capabilities to pull out tweets that were tied to #GDC and #GDC09 during the week of the conference (Monday-Friday). We sorted the tweets by day to better account for initiation and development of topics, and to see if events that took place at the conference were picked up by other conference participants. The amount of tweets made each day shifted, but an average of 825 of tweets was found per 24-hour period.

With that data, we performed two analyses. Firstly, an analysis of tweets specifically referencing the GDC keynote by Satoru Iwata was performed, with results presented using traditional methods (bar charts/graphs). Secondly, a parallel analysis was made by means of Wordle, a visual tool (http://www.wordle.net/) that counts frequencies of words and then represents them in aesthetically appealing "word clouds". In this process the tool makes more recurrent words appear larger in the cloud, allowing size to correspond with frequency. The end result can be manipulated by changing colors and the form of the cloud. By means of these word clouds, we then made some preliminary analyses based on what we found to be relevant ways of explaining the data.

Findings: Analysis of

Iwata Keynote Tweets

Instead of hunting down conference attendees and making them answer a survey or asking participants directly about what topics they initiated and discussed most frequently at GDC, we studied the participants' conference related practices on the social networking service Twitter.

The first analysis was concerned with exploring how participants twittered about the first keynote of the conference, given by Nintendo President Satoru Iwata. First, we wanted to find if attendees twittered at all about Iwata's talk, and also how they did so. We looked for tweets that included either the tag #GDC or #GDC09, and also made some mention of "Iwata," "Nintendo" or the first keynote talk. We explored how they twittered about the content of this talk (including journalistic accounts of what he said, positive and negative reactions to it, jokes made about it, and the like); and we also investigated how participants employed various functions of Twitter such as re-tweeting, posting links, and responding to others' tweets. The results are presented below.

As seen in Figure 1, the greatest amount of tweets concerned anticipation before the actual keynote began. Many tweets mentioned waiting in the (long) line that was forming well before the event began. Some mentioned curiosity about what would be discussed, while others merely noted the wait time they were enduring. The next greatest amount of tweets focused on summarizing or vpresenting information gleaned from Iwata's keynote. These tweets were not evaluative, but instead merely were repeating information that Iwata was giving out. To a far smaller degree, tweets offered positive or negative reactions to the keynote address. It seems that more twitterers were interested in passing along information, rather than giving responses or evaluations of the keynote.

Twitterers also highlighted a few notable areas from Iwata's talk (see Figure 2)—particularly his mention of a "death spiral" that could occur in the game development business, a giveaway of the DS game "Rhythm Heaven" to promote its impending release, the next Zelda game, and discussion of game designer Shigeru Miyamoto's particular design process. The death spiral in particular was well received, as developers quickly posted pictures of Iwata's slide of the spiral, with one creative individual Photoshopping a game CD box to promote the Nintendo game 'Death Spiral'.

Finally, as seen in Figure 3, we looked at how twitterers employed the particular mechanics of Twitter to direct their tweets in particular ways. Somewhat surprisingly to us, most tweets were basic, with only a small number employing retweeting or replies to other tweets. Several more tweets included links to websites, but again, most tweets referred to themselves, and did not appear to be part of a larger, specific conversation.

Findings:

Wordle Representations

In our second analysis, we were inspired by the way the tool Wordle can be used to visually represent word frequencies. We developed a small Java program that queried Twitter.com for tweets about the Game Developer Conference. We then removed sender names and times from the tweets before processing them in Wordle. By analyzing discrete sets of tweets from individual conference days, we found the word clouds generated by Wordle to be a fruitful presentation tool. Wordle uses basic word counting techniques to generate word cloud visualizations that use size to indicate the relative frequency of word usage within the given data set. Below we present the five different days based on the restrictions on tweets signed with the tag '#GDC'.

The first day of GDC, which is comprised of the niche summits (i.e. Mobile, IGDA Education SIG, Indie Games, etc.) saw a general focus of tweets on topics related to the conference theme of games but also including a more diffuse range of topics such as iPhones as well as on the fact only summit sessions were taking place.

On the second day of the summits, the words used mostly by the twitterers were again the words games and game, but another word emerged as just as common; party. One explanation about the word party is that it was used in the context of social events outside the conference. The conference attendees are either recurrently twittering about a certain party or about different parties; based on this data set, there is no way to shed light on which. Words as tomorrow (seen above the word game) indicate some prospective topics about what's to come, highlighting future events of importance.

The tweets about the summits began to be outnumbered by the tweets about the conference proper, which began on Wednesday. This stands to reason, as the summits are much more lightly attended.

Figure 6. The third day (Wednesday).

Substantially different content of tweets on the third day of the conference is seen in the graphical model of the third day's word cloud. Here words such as Nintendo, Iwata and keynote are the most common. This can be understood as a situation in which participants took up for discussion what was happening from the conference website and the subject matter of that day's keynote. In relation to the day before, the word party has receded in importance (seen above the word iwata).

Figure 7. The fourth day (Thursday).

In Figure 7, another keynote presenter is brought up on Twitter by the word Kojima (Hideo Kojima, designer of the Metal Gear series). Here the word party is seen in a smaller text beside the word Kojima, indicating that participants were planning social events or commenting on earlier ones. Here the twitterers make the 'last day' a topic, besides game designers such as Wright (Will Wright).

Conclusions & Next Steps

The use of two data analysis and visualization techniques proved to be instructive. The use of traditional analysis methods for the tweets relating to Iwata's Wednesday keynote—where we as researchers structured the data, labeled emergent themes, and made inferences about what various frequencies might suggest— and the more open-ended use of Wordle to visualize the complete data set suggested different uses. The presentation of the Iwata data as interpreted by our project team lead to a focus on the part of our audience on our methodologies rather than the data.

On the other hand, the word clouds, which presented the full data set, led to the audience joining us in the analysis of the data.. Using Wordle to provide an initial structure to the Twitter data was inspiring for us as researchers. As a visual data mining tool, word clouds provided an excellent method for drawing to the surface trends that otherwise might be overlooked. These visually translated statistics (i.e. word frequencies) seems to inspire people to become more engaged in active interpretation themselves than compared to traditional graphs in which the researcher controls the interpretation.

In order to tune in to what people at the conference were talking about, we went online and studied conference attendees' text-based postings on the social networking service Twitter. As this tweeting practice is only done among certain members of the conference attendees, our analysis is limited to a certain population. It can be interpreted that what goes on there might be read by more than the messages being posted. The postings in themselves can also be seen as a way to talk about how people make themselves and their ideas/thoughts/experiences/opinions 'heard' and 'seen' online by a specific community (i.e. GDC).

More generally, this work makes visible methods for utilizing social interaction in already established social media and ways to work with such forms of computer-mediated communication using both traditional research methods and visually inspiring word-counting tools. We see the social network services such as Twitter not only as a new and exciting way of gathering data but also as a way to follow what participators on conferences actually say and do. For some, such communication is everyday practice; for others, it might be viewed as an exotic data-mining excursion.

As for the way we chose to present and analyze the participants doing there are some positive as well as negative aspects. Wordle functions as both a data-mining tool, by letting us process chosen amounts of words, and as a visually-appealing presentation tool. As has been brought up earlier, the traditional use of graphs when presenting our findings for an audience give rise to certain expectations in the audience. This can be caused by the strong tradition within the research community of using particular forms of graphs, making such standard representations a core part of the researchers' toolbox, often undisputable. The word clouds might be interpreted as representations that are more open for interpretation, where the researchers do not have the final say on the interpretation. The word clouds of the conference attendees most recurrent topics opened up for meaning-making practices when presenting out result for the GDC audience that were not restricted to the stories that we, the researchers, in the group presented. The clouds aesthetically appealing appearances seems to engage people in a way that they overlook the fact that it is, at root, statistics. That is, the word clouds seem to engage people who might not usually attempt to unpacking statistical data on their own.

Using Twitter together with Wordle becomes a first step into statistics. The word cloud acts as an illustration of something, a not yet analyzed phenomena. Presenting our word clouds for the audience at the conference gave rise to other explanations of the data, other stories being told. This has the consequence that these meta-level interpretations of the words could miss out on the function of the word in its original context. In other words, the same word could be posted in very different situations, making the word have different meanings. This can of course be overcome by going back and studying the details of how the word is used, for example how the word party is used on a specific day in their various postings. However, the word clouds open up for many storytelling events that makes it a tool for quickly getting a survey of the topics in circulation among a group or community of people.

One consequence of using Wordle, then, is that the interpretations, or stories, that the word clouds represent are dragged out from the context they were made in. Thus, interpreting them will always be an imagined way of putting them back in some imagined context, particularly in relation to the other recurrent words in the clouds. From our perspective, we can present a story about what we see in the clouds that, if we don't consider the context the word was originally used in, might be strange from the Tweeters' point of view.

Postscript

Our research project has lead to a spin-off project by Local No. 12, a game design collective made up of Mike Edwards, Colleen Macklin (RTR alumni), John Sharp (member of the original project team) and Eric Zimmerman (one of the organizers of the RTR project). Working with the idea of mining conference related-tweets, this group designed and developed the conference game Backchatter. Sharp and Zimmerman saw the potential to develop a game around the tradition of conference reporting through Twitter in order to more fully realize the value of the backchannel reporting.

To conclude the game, the game's creators hold a conference session in which they use Wordle to present the data set. With Backchatter, two sets of data are presented in the word clouds: the words Backchatter players anticipated would be tweeted, and the actual words tweeted during the conference. As happened at the RTR presentation, audience members join in the interpretation of the visualized data set.

Backchatter was playtested at the 2009 Games for Change Festival in New York, and then premiered at the 2009 Games, Learning and Society conference in Madison, Wisconsin and ran at the Digital Games Research Association conference in September 2009 and the Indiecade Festival conference in October 2009.

Play Style Survey

by Tone Vold, Richard Marzo & Annika Waern

Is there any coherence in how different

professions place themselves as players on the Bartle's graph of different play styles?

Is there any coherence in how different professions place themselves as players on the Bartle's graph of play styles?

At the beginning of the RTR workshop, we were given some choices for theory, topic, and method to work with and, after some swapping and discussion during both the first session and later meetings, we decided to use the topic card on Play Styles which depicted Bartle's (1996) Interest Graph. Our goal was to see where the participants at GDC09 would place themselves as gamers and whether there were any differences among participants based on occupation. Would, for example, programmers always place themselves as "interveners" or "achievers"? Would managers be "Socializers"?

Theoretical Framework

The Interest Graph was developed and presented by Richard Allan Bartle, a British writer, professor and game researcher. He has also co-authored the first Multi-User Dungeon (or MUD) ("Richard Bartle," 2009). Bartle found that there were four things that gamers enjoyed about MUDs: (1) achievement within the game context, meaning that they gave themselves goals within and related to the explicit goals of the game; (2) exploration of the game, meaning that they wanted to explore the virtual world that this MUD provided; (3) socializing with others, meaning that they used the game to get in touch with and communicate with other players; and (4) imposition upon others, meaning that they wanted to compete or otherwise interact with others either in combat or otherwise.

Thus, based on Bartle's (1996) framework, one can categorize gamers as achievers, explorers, socializers, or interveners. Whereas the achievers are interested in acting on the world and mastering the game, the explorers want to be surprised by the game and interact with the world, the socializers want to interact with other players, and the killers/interveners want to act on other players. This results in the graph where the X-axis goes from interest in players towards the right to the environment. The Y-axis represents the differences in "acting with" at the bottom to "acting on" on the top (Figure 1).

Method

We decided on making a board (Figure 2) and have conference participants place post-it notes as to where they see themselves as gamers. Participants were asked to choose a color of post-it note that would best represent their occupation using the following categories developed for this study (Table 1).

We started out with the blank board with only the interest graph drawn on to it and walked around in the convention area and stopped participants and asked them to pick a post-it note that would best represent their occupation, write their job title on it, and place it on the board in the quadrant corresponding to how they would characterize their game-play style. We carried the huge cardboard around from table to table, asking participants to take part in our little survey. Surprisingly, very few turned us down and most people were very positive and took time to respond properly (see Figure 2).

Results

The results from our research were quite interesting. We had a total of 66 respondents. The distribution of profession category is shown in Table 1.

Across all responses, there were few self-reported "interveners." Programmers placed themselves "all over the place" with most tending towards the "focused on world" end of the horizontal axis (in contrast to "focused on people" end). Audio and visual professionals classified themselves as 'explorers', that is, placed themselves more toward the "focused on world" end of the horizontal axis and with more emphasis on "interacting with" (bottom of vertical axis) rather than" acting on" (top of vertical axis). Business and management professionals gravitated towards both the "socializers" quadrant (interacting with players) and the "achieving" quadrant (acting on world). Participant who chose the category "other" were relatively evenly distributed between the "socializers" quadrant and the "explorers" quadrant (interacting with world) with only one exception.

Discussion

Despite our relatively small sample size (only 66 out of all GDC-participants), we did see some trends regarding profession and play style. The general trend towards an interest in 'worlds' rather than 'people' is perhaps the most interesting observation. It makes sense that developers and artists would have a high interest in worlds, since so much of the effort in creating a game must go into the world simulation; ranging from physics engines to visuals. The trend was also particularly pronounced for audio and visual artists, who tended to classify themselves as 'explorers'. On the other hand, business and management professionals had a tendency towards classifying themselves as socializers and achievers, again roles that rhyme well with their chosen profession.

The fact that so few participants chose to classify themselves as 'interveners' might be less significant. In Bartle's original classification schema this group was named 'killers', and although we did not use that term we can suspect that many participants knew about it and hesitated to classify themselves as such. It is worth noting that since the players classified themselves, the graph does not reflect their actual play styles: it reflects how they perceive themselves as players, or perhaps even how they wish to be perceived.

Bartle's (1996) model of play styles is, of course, a simplification of what motivates players; Bartle constructed it as an aggregate model of the responses that players gave to a host of questions. It is likely that most players do not fit into a single category, at least not all of the time. One of the audience members, the famed ARG designer, futurist and academic, Jane McGonigal suggested adding an axis to the plane to see how much deeper a three dimensional version of the Bartle theory could be. Although this is an interesting idea, it is equally compelling to see that the study participants had very little problem in classifying themselves according to the Bartle simple typology. During the experiment, we only used a single board to aggregate the results, so as participants answered, the board filled up. It was suggested that each participant should have had their own sheet, to have a clean view of the two axes of play-style. But with the one board method we used, the participants themselves were able to immediately see the results up to that point and the result when they added themselves to the board. Just like a game, there was an immediate interaction between the player/participant and the system/experiment (with a short tutorial/marketing phase by us).

Our results show that there indeed is something interesting to find out about preferred play styles of people in the game industry. It would be interesting to do the same study but involving all GDC conference-goers to see if these trends endure. Another interesting option is to investigate if there are differences between how players choose to classify themselves and their actual play styles. It could, for example, be interesting to investigate the difference between how players classify themselves and how their friends or colleagues classify them. Another approach might be to investigate how participants might redesign the play style graph based on their own preferred play styles. Although the graph worked in our study, it is not optimal; it is now quite dated and it was developed with one particular game genre in mind. This could be combined with the aforementioned "three-dimensionalization" of play styles.

Reflections

This was a fun experiment and indeed we got to know a lot of people and also the group members and made it very social to be a participant at the GDC. For that reason only one could promote doing real time research, but maybe even more important what could "real" benefits of such research bring us? What data would be interesting to publish from a conference such as the GDC?

For the research area that we drew on (specifically, examination of play styles among varying professions represented at GDC), the study could be viewed as a pilot study of sorts. During the presentation of these preliminary results on the second day of RTR, we had many interesting comments and questions from those in attendance. Though we were under some pressure to enjoy the conference for ourselves while also doing research, the results were indeed interesting and drew eager participation from our audience, which consisted of a great many academics. This was encouraging for both researchers who are thinking about attending future conferences as well as developers with intentions of linking up with the world of academia.

One of the challenges we and other groups most definitely faced was how to approach our subjects, how to let them know what we were doing as fast as possible without taking up too much of their time. Many things are happening at a conference of this stature (ie. GDC), but people were generous enough to give us some of their time and help with our project.

More broadly, RTR could almost be thought of as given Salen and Zimmerman's (2004) definition of a game as artificial systems in which players engage in conflict, defined by rules and resulting in quantifiable outcomes. We were "players" ”" our group on the same team, but in competition with other groups (the "conflict"). Our defined rules were our cards. In doing our "research" in "competition" with other groups, the rules and constraints were common. We could choose to follow all or only some of our cards, we could ask assistance from RTR workshop leaders, we could choose our research materials from those made available (pens, stickers, post-it notes, etc.), and we had shared time constraints. All groups had an outcome. Ours was quantifiable and to the best of our memory, so were a few of the others.

RTR could even be said to be a ludic activity - we had a lot of fun during the experiment! One difference between the two is that, in a game, you are in an alternate reality while RTR was "really" real. And while we did not win "a prize" per se, the opportunity to present research results at "THE" GDC could very well be considered a prize in and of itself.

Acknowledgements

We would like to thank Kauthar Tung, Jim Diamond and Jauver Elizondo who were also members of our RTR team and participated in the early phases of this project.

References

Bartle, R. (1996). Hearts, clubs, diamonds, spades: Players who suit MUDs. Journal of MUD Research, 1(1).

Richard Bartle. (2009, August 21). In Wikipedia, the free encyclopedia. Retrieved August 21, 2009, from http://en.wikipedia.org/wiki/Richard_Bartle

Salen, K., & Zimmermann, E. (2004). Rules of Play: Game Design Fundamentals. Cambridge MA: MIT Press

Profession

Post-It Color

# of Respondents.

Business/Management

Green

13 [14]

Audio/Visual

Pink

5

Design

Blue

6 [8]

Production

Orange

25 [23]

Programming

Light Yellow

6 [7]

Other

White

3

Multiple*

* Some participants had several profession titles.

Game Developers'

Descriptions of

their Player

by Carla C. E. Fischer

How do video game developers

describe their ideal players?

How do Video game

developers describe their ideal players?

When faced with the challenge to design a research study, collect and analyze the data, and present it before the end of the 2009 Game Developers Conference (GDC), our group chose to focus on the game developers' ideal player. Identifying the intended audience is a key element in the game design process. As such, we wondered how game developers would describe the player they have in mind for their current (or most recently completed) game.

Method

During the 2009 GDC, members of our research group approached conference attendees before and after sessions as well as when they were sitting at tables during lunch and asked if they would be willing to complete a small survey, consisting of the following items.

We did not count the number of attendees who declined to complete the survey. Anecdotally, several team members mentioned that they were turned down frequently, reminding them why asking random people to fill out surveys is so difficult.

The other method used to capture the audience characteristics was to ask respondents to draw a quick sketch of their ideal player (four examples of which are illustrated here).

Results

Respondents (n=51) named a total of 247 characteristics. After reviewing the list of characteristics, similar words were edited for consistency (i.e. females became female). Because of the small sample size, the characteristics (discussed below) are not further analyzed by the respondents' role or the type of game.

Figure 1 is a word cloud of the characteristics, which was generated using wordle.net. The font size of the word provides a visual indication of the frequency by which the characteristic was named. (Color was merely for visual presentation purposes and does not represent any factor in the analysis.) The most frequently named characteristics were social, casual, young, explorer, boy-or-girl, likes, creative, and curious. Social and casual likely represent genres of games. Young and boy-or-girl likely represent audience types. Likes was commonly used in conjunction with another word (e.g. likes puzzles or likes building), and the algo rithms used in wordle.net to generate word clouds breaks the words apart into separate elements. Explorer, creative, and curious are assumed to represent player traits.

The characteristics could also be informally grouped into categories, such as age (generally specified as ranges), gender specifications, personality and lifestyle qualities (e.g. academic, competitive, curious, environmentally-conscious, explorer, social, low technology user, musical, playful, problem-solver, web-savvy), game genre or gamer type (e.g. hardcore, casual, nongamer, gamer, MMO), or cultural descriptions (e.g. English language learner, Caucasian, third-world).

Conclusions

By compiling the characteristics game developers use to describe their target audience, this survey, albeit informal, provides a snapshot of current trends. Rather than focus on the hot genres or technologies, this snapshot illustrates the ideal audience. However, the picture is limited ”" many of the provided characteristics could fall into several categories, but without contextual information from the respondent, it's impossible to interpret their exact meaning. Despite this, the concept of quickly capturing five characteristics was relatively easy to implement. Additionally, if performed repeatedly over time, it would likely provide a changing picture of the times.

The other method used to capture the audience characteristics was to ask respondents to draw a quick sketch of their ideal player. While the pictures drawn by respondents were not analyzed, the use of a drawing prompt is worth noting as a method for future researchers to consider. Most respondents' chose to draw a picture, and they ranged from stick figures to 3D drawings to abstract representations. Respondents' reactions ranged when asked to draw a picture. Many were hesitant but agreed to "do their best" with a bit of encouragement from the researchers. Others appeared to be perfectly happy to only draw a picture and not fill in the remaining questions. It appeared to be an unexpected and enjoyable part of the survey. It added to the informal feeling of the research and is one that researchers should consider as a method to both break the ice and gather infor mation quickly, assuming the researchers are prepared to do the analysis and coding of the drawings during the conference setting.

Tool 1

RTR CARDS

The cards of Real-Time Research have become one of my favorite tools. They really don't do anything more than start the conversations that build toward ideas, yet they provide a structured game element to the process that sets a tone for play in work alongside giving the facilitator the ability to frame the conversation. Each time we run RTR some groups have used these as a challenging constraint, while others simply let their conversations run into completely different topics from where they started. This flexibility is part of the value of the cards.

For you, and your use, the cards below are the latest iteration. They have been changed each time we have done RTR and you'll want to alter them for your use also. Play with the types of cards, contents, and topics and you can really use RTR as a tool in any setting, classroom, or project. Have fun.

Theory Cards

Method Cards

Topic Cards

Tool 2

STRUCTURAL OUTLINE

Because there were four of us, we used a shared planning document to organize. This tool was key to making sure everything was set. In addition to the list of items needed, we had a rough breakdown of the time and roles played during the session. At the end are ideas for coming RTR sessions you may want to employ.

MATERIALS NEEDED

Session

” Printer for session room/RTR Booth (w/ associated cords)

” Notepads/pens/markers for each group.

” Power strips (for many active laptops) or check for outlets in room

” Create an e-mail address for RTR (so folks can send files to have printed)

” Clipboards (10)

” Paper (1-2 reams)

” Stapler, markers, pens, notepads, string, tape, portable file/organizer, file folders, easel w/ sheets of paper

Presentation Materials

” Set of Cards - Color glossy printing on card stock

” Keynote presentation (if wanted) and template for group presentations

” Consent and ethical research guidelines printed out

” Templates of data collection tools printed and ready

SESSION 1: THURSDAY (90 MINUTES)

Introduction (10 minutes)

C/E/K introduce the impetus for the project, talk about how everything is going to work, and ”" most importantly ”" set expectations. For example, the goal is to do real research here, but it will be fast and loose, in a prototyping and collaborative spirit. This introduction is also the chance for us to see who is in the room and strategize about how to divide into groups.

Divide into groups (5 minutes)

This is perhaps the trickiest step. We need to be sure that we have right-sized groups, and that the groups are balanced in some way. Groups could be randomly determined, or we could have a system, such as giving each person a color-coded index card corresponding to their identity (researcher, educator, designer, etc) and make sure that each group has at least 1 of each color. Typically in design exercises it is good to have people working with strangers, but since this experiment will last across the entire conference, there are advantages to taking advantage of existing social ties ”" so we might let people self-select groups.

Hand out cards (5 minutes)

Each group will have a set of cards. These are the things that we're going to discuss in the call tomorrow. We'll probably have at least two types of cards (such as methodology and topic) and each group should get a limited number, such as two of each type.

Trade cards (10 minutes)

Once groups are set with their cards, we'll have a quick trading frenzy. We'll set up a dozen additional cards on a table, and groups can put one of their cards down on the table and take another. They always have to put one down to take another, so that other groups all always have the same number of choices. This kind of thing helps groups feel like they are authoring their own parameters, and it is a nice bonding experience where they get to see who in the group is interested in what.

Brainstorm experiments (20 minutes)

OK, now the groups are set, they have their card-based parameters, they know what this session is about, and here's the time when they actually begin to design experiments, using their parameters to guide them. The goal here is to think in guerilla terms ”" to figure out what they actually can do over the course of the conference. While this is going on in the room, we should have up on the screen a list of the resources and contexts available to them, such as the number and type of attendees in the conference, a list of breaks and meals where people can be corralled into doing a survey, etc. We might also create a handout that has this information. The goal of this first brainstorm is for the groups to come up with a concrete plan for their experiment. Twenty minutes goes quickly, but less time is better because it forces them to be decisive. The three of us can listen in on these discussions, but it's better to not have too much input so each group can own what they are doing.

Quick critiques (15 minutes)

Here we pair up groups, so that each group can pitch another group their experiment, and can get feedback on their idea. This is an important moment in the design process, when you get outside criticism and support. Also, all the participants will realize that other groups are struggling just like they are, and this builds solidarity within a group and among all the attendees. It also helps set the stage for the second session, because attendees will want to see how the experiments of their matched-up group fared. The three of us will also sit in on these critiques and offer feedback.

Final planning (25 minutes)

The last section of the session is for groups to finish their planning, put together surveys, etc. They should be dividing the labor at this point among the people in the group, and we should have printers ready to print out what they need. Also, the three of us should be helping groups strategize the logistics of what they want to accomplish over the conference, since we won't all meet as a group until the final session again. Groups should plan to present whatever it is they managed to do (or not do ”" there is no such thing as "failure" on the Real-Time Research planet) at the final session the next day.

IN-BETWEEN THE SESSIONS

This is harder for me to plan, because you two know the structure of the conference better than me. Ideally, we would have a table in the main concourse with a big sign that says "Real-Time Research." At the plenary session, whenever we have a chance to make an announcement to the whole conference, we should tell people to stop by the table.

This table will be the central gathering place for attendees to come and pitch in as research subjects. Students or volunteers should be at the table the entire conference and be able to conduct experiments without the session attendees being there. One thing we might do is think of swag that a GLS attendee would get for being a subject, like a t-shirt or some candy, or be entered into a drawing for a prize (but I think a small prize for each subject is probably best).

I think we should expect that these experiments will probably not come together until lunchtime on Thursday, but even so, they still have more than a full day to gather data. We should expect that some groups may do much wackier things, such as design a social game for the conference itself, that won't require the table and setup we're going to provide.

SESSION 2: FRIDAY (90 MINUTES)

At this session our main goal will be to have groups share their experiences, for better or worse. The exact time each group has will depend on the number of groups, but we should aim for very short presentations and then spend most of the time on discussion.

Each group should appoint one responsible person to gather all of the materials and results from the group and send them to K/C/E. And perhaps we can promise to think about writing an article ”" or even some kind of short blog entry ”" about it.

ALSO...

SUNDRY STUFF

(ideas that came up as part of our article):

” add a session between (midway through as a kind of "check in" for participants) to make sure everyone's on track?

” add a panel of judges to the final presentation & give out awards for various categories?

” have some example studies that we present in our intro?

” get some safari type "field hats" with cards stuck in them (like old "press" hats) so those who are running studies are clearly designated?

” get some "lab coats" for visual distinction when in field?

Tool 3

GOODIES

Setting up goodies also meant organizing a booth, or drop-in location for our RTR groups that served as a central station to pick up supplies, make copies, or come for help with anything they may need. The goodies list became set up document we used for the booth in addition to listing ideas for future conferences that you may make use of too. Short and sweet, the goodies list helped get ready for the non-session parts of RTR.

BOOTH

” Table SMD

” Table Cloth

” Big sign ("RTR: Real Time Research w/ GLS logo) Lamenated 2 - 2' x 3' Poster

” Printer for booth Sharing with new GLS printer.

” Internet connection here Wireless at MT

” Create an e-mail address for RTR (so folks can send files to have printed) Will cre ate on G-mail if no preferences

” Clipboards (10)

” Paper (1-2 reams)

” Other supplies: Stapler, markers, pens, notepads, string, tape, portable file/organizer, file folders, easel w/ sheets of paper

GOODIES

” t-shirts?

” decks of RTR cards

” CANDY (preferrably chocolates for CS)

” "I Subjected" stickers

” Cute participation awards

POTENTIAL MATERIALS

(have a plan for access if needed):

” Full Time RTR Grad student w/ parking passes (to make supplies runs)

” Access to copy machine will be needed.

” Separate table, close to registration.

” Set up RTR forum/wiki/type thing for folks to stay in touch if wanted.

” Video cameras

” Digital cameras

” Tag boards/White boards w/ easels/public wall space

Tool 4

SLIDE TEMPLATES

Slides

Tool 5

GROUP SUMMARY FORM

GROUP CONTACT INFORMATION

Members E-mail

WRITING FOLLOW UP & CONSENT

With your permission, we would like to include your RTR Project Report in an edited book we are publishing online through ETC press <http://www.etc.cmu.edu/etcpress/>.

There are two levels of participation ”" "extended abstracts" versus "book chapters." For those of us who aren't able to do any writing or editing outside of the conference time already given to the RTR project, we would publish your write up "as is" (with only minimal copy edits where necessary" as an "extended abstract" (2 pages or less) in the book. For those of us with a special interest in the RTR project and its publication, we would work with you to revise and expand your RTR Project Report as a peer-edited book chapter for the volume. Turn around time would again be relatively short but here you can take credit for a full peer reviewed publication with promise to make genuine contribution to the field. Either way, you get authorship on the piece included in the edited volume to reflect your intellectual ownership of the work.

Please indicate below if you are willing to publish your RTR Project Report as either an Extended Abstract or Full Chapter in our edited, online volume entitled "Real Time Research." Also indicate who can serve as the main contact on your team (we will assume it is the first name listed if no other name is indicated as main contact). If you are unable to participate in either way, we still love you and will scrap your report.

Tool 6

REPORT TEMPLATE

RTR: Report Name

Author 1, Author 2, Author 3, Author 4, Author 5

Review & Research Question

Replace this text: In a paragraph or two summarize how your group came to your targeted research question and why you felt the question was important or interesting to study. If there is specific body of literature that brought your group to the question or helped you frame this work in some way, reference it here.

Clearly state your group's research question.

Methods

Replace this text: Explain the method by which your group gathered data to answer your stated research question above. The methods described here should how you selected participants and the procedure that you used to gather data from those participants. If the data consists of a pre-existing set of information (such as anonymous forums), detail the nature of the data here . This section varies broadly across projected given the wide range of various project pursued as part of the RTR workshop; word your description carefully so the reader could replicate the process if they wanted.

In a second paragraph, explain the process your group chose for data analysis. What methods are you using to interpret the data you collected in order to answer the research question stated above Again, this section will be highly dependent on the kind of question your group decided to pursue.

Findings

Replace this text: After your group collects and then analyzes your data, record here any general conclusions, indications, or trends you see based on your investigation. Include not only the general characteristics of the data that you happen to notice but also findings that you didn't expect to see. Given the iterative and quick turnaround of RTR, unexpected findings are common and part of what makes the experience fun and interesting.

Include any mini-charts, graphs, telling photographs or other visuals that help represent your project. Please do not include any images of participants themselves.

Conclusions & Next Steps

Replace this text: Explain the importance of your findings on a broad, more generalized level. What are the implications of your findings? What takeaways, if any, do you have from the project ? What did you learn?

In a second paragraph talk about if and how this project would be expanded or followed up on formally. Would you recommend a large scale version of this study or a modified version? Is it worth the effort of further work? Why? Why not? Are there follow-up questions you believe are worth pursuing?

Tool 7

GROUP ETHICS GUIDE

& CONSENT FORM

Any contact with people outside of the session with the intent to collect data becomes subject to common research guidelines. We ask you to adhere to these guidelines in any interactions for your group project at the conference.

Any and all data you collect as part of the project may be used in the final session along with your analysis and conclusions from the experience, however it cannot be used for outside publications beyond the RTR online book through ETC press. If you want to conduct further research based on this RTR project beyond the purview of this conference workshop, you must contact your own institution's IRB board (if applicable) and obtain the necessary consent and approvals as deemed necessary. RTR is intended to function as an 'open source' venue for idea generation and dialogue between professionals in disparate domains here at the conference. Subjects explored as part of the RTR workshop are open for exploration elsewhere but not the data collected as part of this learning experience is not for use anywhere else, under any circumstances.

DO'S

DON'TS

CONSENT SCRIPT

Please use the following script to obtain oral consentfrom individuals before gathering any data:

"Hello, my name is .

I'm part of the "Real Time Research" session here at the conference and I am looking to gather information as part of the session. You were selected because .

There is no expected risk or benefit for being part of this study, other than this nifty sticker that says "I Subjected" which, we feel, is a clear benefit.

Any participation on your part is voluntary and you can exit the process at any time for any reason. We will neither collect nor keep no personally identifiable information.

Would you be willing to spare a bit of time to be part of RTR this year and get a cool sticker?"

Tool 8

FOLLOW-UP FORM

GROUP CONTACT INFORMATION

Name E-mail or IM

WRITING FOLLOW UP AND CONSENT

With your permission, we would like to include your RTR Project Report in an edited book we are publishing online thru ETC press <http://www.etc.cmu.edu/etcpress/>.

There are two levels of participation ”" "extended abstracts" versus "book chapters." For those of us who aren't able to do any writing or editing outside of the conference time already given to the RTR project, we would publish your write up "as is" (with only minimal copy edits where necessary" as an "extended abstract" (2 pages or less) in the book. For those of us with a special interest in the RTR project and its publication, we would work with you to revise and expand your RTR Project Report as a peer-edited book chapter for the volume. Turn around time would again be relatively short but here you can take credit for a full peer reviewed publication with promise to make genuine contribution to the field. Either way, you get authorship on the piece included in the edited volume to reflect your intellectual ownership of the work.

Please indicate below if you are willing to publish your RTR Project Report as either an Extended Abstract or Full Chapter in our edited, online volume entitled "Real Time Research." Also indicate who can serve as the main contact on your team (we will assume it is the first name listed if no other name is indicated as main contact). If you are unable to participate in either way, we still love you and will scrap your report and not make use of it in any published materials generated as a result of this workshop.

CONSENT FOR PARTICIPATION

I am interested in publication of our report as (sign and check):

Name

Extended Abstract Book Chapter Neither

Extended Abstract Book Chapter Neither

Extended Abstract Book Chapter Neither

Extended Abstract Book Chapter Neither

Extended Abstract Book Chapter Neither

Replace this text: Explain the importance of your findings on a broad, more generalized level. What are the implications of your findings? What takeaways, if any, do you have from the project ? What did you learn?

In a second paragraph talk about if and how this project would be expanded or followed up on formally. Would you recommend a large scale version of this study or a modified version? Is it worth the effort of further work? Why? Why not? Are there follow-up questions you believe are worth pursuing?