Participation by Invitation Source: In-game vs. Open Call

When I began this research my plan was to invite players from within the game. It seemed to be a more personal way to send out invitations, through in-game mail or whispers, and it allowed for random selection of guilds based on census data. I dutifully enlisted my assistant to help collect census information, and over three months we logged information on thousands of characters and hundreds of guilds across five randomly selected realms. Sounds good, right? Wrong.

This is a tale of failure. I failed to recognize the impact immense amounts of in-game spam and the aggressiveness of hackers have on players. I thought that by including my personal information and offering to answer questions I would be able to overcome any similarities between my messages and those of a hacker. Once I realized this was not the case, I opened the survey to all World of Warcraft players by sending out a call for participation through email, World of Warcraft forums, Facebook, Twitter, this site, and on Reddit.

The response from Reddit was fantastic. More Reddit members answered the call for participation than all other sources combined. Over 100 people followed the link on the first day, and activity continued throughout the open call.

An interesting aspect of using social media is that calls for participation end up overlapping outlets. Based on differences in responses across media —phone, mail, email— shown in past research (Dillman, 2007), I created each outreach message with a different link. In just the waves I can track, I’ve seen the Facebook link Tweeted, and the email and Reddit links posted to Facebook. It looks like Facebook overlaps everything. The overlapping social media sharing could also explain why Twitter doesn’t show up in the final representation of participants.

Another thing I was able to learn by using social media, and Reddit specifically, is that my survey is too long. Feedback during the survey open call was very helpful. Although I did a pilot study, and asked friends to take the survey before that, to test the survey length and questions no one told me this during those tests. Thanks to the helpful folks at Reddit, I have already cut the survey in half for its next application.

Resources

Dillman, Don A. (2007). Mail and internet surveys: The tailored design method. Hoboken, New Jersey: John Wiley & Sons, Inc.