Profiling Academic Research on Massively Multiplayer On-line Role-Play Gaming (MMORPG) 2000-2009: Horizons for Educational Research

Perfil de la investigación académica sobre Juegos Masivos en Línea para Múltiples Jugadores (JMLMJ) 20002009: Horizontes para la investigación educativa

Harold Castañeda Peña¹
Adriana Salazar Sierra²
Nadya González Romero³
Luis Ignacio Sierra Gutiérrez⁴
Alfredo Menéndez Echavarría⁵

¹Universidad Distrital Francisco José de Caldas y Pontificia Universidad Javeriana, Bogotá, Colombia. Correo electrónico: harold.castaneda71@gmail.com
²Pontificia Universidad Javeriana, Bogotá, Colombia. Correo electrónico: adriana.salazar@javeriana.edu.co
³Pontificia Universidad Javeriana, Bogotá, Colombia. Correo electrónico: ngonzale@javeriana.edu.co
⁴Pontificia Universidad Javeriana, Bogotá, Colombia. Correo electrónico: lisierra@javeriana.edu.co
⁵Pontificia Universidad Javeriana, Bogotá, Colombia. Correo electrónico: menendez@javeriana.edu.co

Abstract

Whilst there exists a large body of publications around Massively Multiplayer On-line Role-Play Gaming (MMORPG), there is little profiling academic research on this type of game. This study aims at unveiling what, when, where and who constitute scholarly work in research about MMORPG. A 777-register dataset was configured with primary documents taken from 16 databases and two web-portals. The dataset was drilled down using specialized text-mining software. Findings revealed four main research interests that comprise the games themselves, gaming experiences, systems architecture and educational MMORPG. It was also found that research on this topic started out in 2002 and some milestones of emerging research were charted out. The most prolific organizations and authors were also identified in which the USA, Canada and Italy occupy outstanding places. It is recommended that research profiling studies be carried out to extend more informed literature reviews and support further research questions.

Key words Massively multiplayer on-line role-playing gaming, MMORPG, research profiling, text mining.

Resumen

La investigación sobre Juegos Masivos en Línea para Múltiples Jugadores (JMLMJ) es amplia; sin embargo, no hay mucha literatura especializada que perfile la investigación sobre este tipo específico de juegos. El presente estudio persigue describir el qué, el cuándo, el dónde y el quién que constituyen trabajo investigativo y académico sobre los JMLMJ. Se configuró una base de datos de 777 registros con documentos de investigación provenientes de 16 bases de datos académicas y dos portales web. La base de datos que se organizó fue explorada utilizando un software especializado en minería de textos. Los resultados revelan cuatro tendencias principales en la investigación sobre los JMLMJ: los juegos en sí mismos, las experiencias de juego, los sistemas de arquitectura de estos juegos y los JMLMJ relacionados con el fenómeno educativo. Se encontró que la investigación sobre estos juegos se origina en 2002 y se encontraron rutas de investigación relacionadas como desarrollo del campo. Se identificaron los autores más prolíficos quienes son provenientes de organizaciones en USA, Canadá e Italia. Se recomienda la realización de estudios de perfil para ampliar las revisiones de literatura que sustente la formulación de preguntas de investigación.

Palabras clave Juegos Masivos en Línea para Múltiples Jugadores, JMLMJ, estudios de perfil, minería de textos.

Introduction

This paper stems from a research project⁶ seeking to investigate the state of the art about research conducted between 2000 and 2009 on Massively Multiplayer On-line Role-Play Gaming (MMORPG). Bragge and Stordgårds (2007) have claimed that profiling academic research 'helps in seeing the forest from the trees' (p. 714). As implied by them, this is a way to enhance the traditional literature review conducted in most research processes and this by extension applies to educational research. Thus profiling studies come in handy to go beyond old-fashioned literature reviews (Porter, Kongthon & Lu, 2002). According to these authors ''a key advantage of searching/profiling large bibliographic databases is the possibility of uncovering relevant research not being taken into account by the 'invisible college' of researchers working a given domain'' (Porter et al, 2002, p. 365). In particular, this is beneficial for education as profiling would allow speeding up unveiling new related topics that go unseen in the field.

Research profiles have also been recently used as a source to outline how publications about specific fields of knowledge are distributed and how much is invested in such type of research. For example, Guerin, Flynn, Brady & O'Brien (2009) compared the relationship between population demographics and research outlays. Within the same spirit and in order to assess the impact of research, Thomas (2009) is able to suggest many topics that crop up together shaping inter-disciplinary research agendas for business schools by understanding the research profile of management as a discipline. Gupta & Dhawan (2009) draw on a comparative research study where the Indian research profile is measured against other research-productive countries. This has also been done on a smaller topic-driven scale where strategies for research collaboration have been found at a micro-continental level (Kay & Shapira, 2009). As it might be noticed this is much needed in educational research.

In that line of thought, research profiling studies on MMORPG become very useful as these games could be considered one of the most innovative phenomena in contemporary life and their impact on education should not be ignored. These games have also been well-thought-out as plausible laboratories to describe and experiment with social systems. This experimentation could be done purposefully to systematically study human, linguistic cultural and educational processes with regard to childhood, adolescence, youth and gender issues, for example. The fact also seems to be undeniable that these games mediate all sorts of interactions and ways to construct the self and otherness, aspects which are key in contemporary education. We would like to argue that the present research profiling study could be assumed at a foundational level for myriad research interests that will perhaps include topics related to game designing, game experiences, identity construction, interculturality, symbolic constructions, and mediated interactions in educational contexts. We would also like to highlight that this study could be useful to generate further research processes that could shed light on understanding deeply new forms of interaction through and in virtual and educational environments where multiple languages are involved.

Despite the outburst of studies dealing with research profiles, with different goals and scopes, there is little profiling academic research on MMORPG. Whilst digital gaming has got some attention from this perspective (Bragge & Stordgårds, 2007), there is a gap in the literature about understanding how MMORPG constitutes a newly-born research domain and how it is shaping emerging research topics and horizons for a range of academic disciplines where education is included. Evidently, academic research on massively multiplayer on-line role-play games started out systematically around a decade ago. However, it is also in Bragge's & Stordgårds' (2007) line of thought that there is a felt need to initiate profiling research on MMORPG. This paper, being part of a major research process, then seeks to profile academic research on MMORPG providing a holistic panorama about what has been researched, when it has been done and who has researched on massively multiplayer on-line role-play games. In the first section of this paper, the research profiling process will be briefly described. Then the paper moves on into extensively profiling the research on MMORPG. Finally, the last section addresses some issues and options around profiling academic research on MMORPG.

Understanding profiling academic research

Porter et al (2002) have wondered about the initial research phases as researchable phases on their own. This is why these authors have claimed that there is not that much scholarly attention to literature review processes and this is apparently forgotten in educational research in general. There is however a number of studies whose concern have addressed categorizations in the literature review (e.g. Stiles & Mick, 1994) and have reflected upon the implications of doing systematic reviews in large bodies of literature (Bannigan, Droogan & Entwistle, 1997). Ottenbacher (1983), for example, thinks of the literature review process as scientific inquiry whilst Glass, McGaw & Smith (1981) size it up as a meta-analytical practice in social research. All in all, it seems that these preliminary research phases related to the literature review have been named differently (e.g. literature review, background to the study, state of the art, etc.) to refer to the same research process that inquires about conceptual and methodological knowledge about a research topic or field. Painstaking examination of academic research already conducted is undertaken to find out about plausible gaps and to formulate research questions. We would like to contend however that social and educational researchers have to keep in mind the very different nature and scope of such processes. Among these processes, research profiling studies appear to provide researchers with quality input based on quantitative evidence to undertake potential research projects. Fundamentally, a research profiling study comprises a scanning activity of academic research papers in relation to their titles, abstracts and subject key words. This could be conducive to a panoramic view of a field of knowledge categorized by prolific authors, subject areas involved, partnerships and research groups, for example. This is worthwhile for identifying trends within the field and to pinpoint key authors whose work truly constitutes a background for the specific studies.

Porter et al (2002) place research profiling at an underlying determinant level when it comes to undertaking research processes. Basically, the idea is to broaden and complement the scope of the traditional literature review. Whilst a well-executed literature review undoubtedly can provide in-depth observation of the issue at stake, the research profiling process can add scope and breadth. Both depth and breadth are required to account for a better interpretation of a researchable problem in a specific and widened field of knowledge which has been profiled. They (depth and breadth) can also tally up trustworthiness to the proposal of achievable research goals. Figure 1 illustrates the place of research profiling:

It is within and in this level that research profiling becomes a powerful research process in its own right as research communities can extensively benefit from it. According to Porter et al (2002) there are at least three main requirements for carrying out profiling studies. Firstly, the researcher needs to have access to databases; this also implies the availability of time and resources as a research profiling study is not about a ‘novice doing a ''one-nighter'' contextual description' (Porter et al, 2002, p. 366). Secondly, text mining tools and text mining expertise are considered necessary because the work is done within large sets of databases. Text mining has also been recently applied to open-ended questions and its use is suggested in the study of interviews and classic texts (Bécue-Bertaut, 2010). Thirdly, subject expertise within an inter-disciplinary group is important as the end goal is knowledge discovery worthy of being researched further.

This is certainly a new approach that harmonizes research processes embodying a richer literature review practice. In the words of Porter et al (2002) ''research profiling should augment, not replace, the traditional literature review'' (p. 352). There are, however, strong differences in both approaches: the literature review and the research profiling study. For example, the traditional literature review is focused at the micro-level in the sense that research studies are reviewed one by one on an individual basis. With that research logic the range of a literature review is potentially narrower. A traditional literature review zooms in on a specific topic; this could also be assumed as disadvantageous because by ''zooming in tightly on a singular topic we fail to take advantage of clues that may suggest vital topical linkages to related findings and issues'' (Porter et al, 2002, p. 353). At the end of such practice, texts are discussed and the overview appears to be reduced. In contrast, research profiling has a macro-focus approach because the literature is understood as a body (yet unstructured or inarticulate) whose payoffs include knowing with a good degree of assurance what, when and who constitute scholarly work in a patterned way (see also Paisley, 1990). This appears to be good research practice because by ''mapping related interests, we may uncover helpful techniques, secondary variables, and unusual applications to provide fresh insights'' (Porter et al, 2002, p. 353). This practice also presupposes a wider range and scope in the profiling task as figures go up from twenty to perhaps more than twenty thousand references. Two more key features characterize research profiling. On the one hand, the search is by no means closely limited to a singular topic but the profile embraces related areas. On the other hand, the results imply not only text discussion but also graphical and statistical interpretation. Table 1 summarizes this comparison between the two complementary approaches.

As already pointed out, text mining is essential to research profiling studies. Text mining procedures are frequently applied to what could be considered a text mining problem. In their own research about pattern discovery in emails, Manco, Masciari & Tagarelli (2008) claim that there is a text mining problem when there is a set of textual documents (i.e. in the case of this research profiling study this would be the set of databases containing academic research papers on MMORPG) that could be assigned to a suitable cluster according to specific contents and research goals. For the purposes of this research profiling study these initial clusters respond to the what, when and who constituting scholarly work (Paisley, 1990) on MMORPG as stated earlier. Losiewicz, Oard, & Kostoff, (2000, p. 100) understand text mining essentially as a pattern-based retrieval practice ''in which a pattern in the data is first discovered, and then that pattern is used to present information (the pattern itself or outlier data perhaps) to the user''. This resonates with Nahm's (2004) idea of text mining being a ''burgeoning new technology for discovering knowledge from text data'' (p. 1). Basically what research profiling studies get from this process is knowledge discovery or -as known by the English acronym- KDD (Knowledge Discovery in Databases⁷). In the same trend of thought, what Losiewicz et al (2000) consider KDD responds to is to the configuration of articulate clusters, let us say answers, to the following set of questions:

• What science and technology is being done globally;
• Who is doing it;
• What is the level of effort;
• Where is it being done;
• What is not being done;
• What are the major thrust areas;
• What are the relationships among major thrust areas;
• What are the relationships between major thrust areas and supporting areas;
• What are the promising directions for new research;
• What are the innovations and discoveries.

(Losiewicz et al, 2000, p.100)

Coming to grips with part or all of these questions is what defines- in a principled way- textual data mining where mining refers to the retrieval process itself; data refer to numeric values which make sense according to schemas (e.g. clusters) and texts refer to the substance of words expressed in natural language and contained in research articles. It could also be said that 'data mining seeks to discover previously unknown regularities or anomalies in large data sets' (from http://www.nasa.gov/centers/ames/research/technology-onepagers/data-miningcomplex-sys.html).

Following Fayyad, Piatetsky-Shapiro & Smyth (1996) we concur with Losiewicz et al (2000) arguing that text data mining is not a trivial process as some ''one-nighters'' (Porter et al, 2002) might have thought; we are dealing with a nontrivial process ''of acquiring valid, potentially useful and ultimately understandable knowledge from large text collections'' (Losiewicz et al, 2000, p. 101). Fayyad et al (1996) document how KDD has served the purposes of science and business in the areas of marketing, investment, fraud detection, manufacturing, telecommunications and data cleaning. In the eyes of Fayyad et al (1996), the KDD process consists of eight steps. These steps include processes of data selection, data processing, data transformation, data mining, interpretation and evaluation. Alternatively and borrowing from Porter & Cunningham (2005), Bragge & Stordgårds (2007) suggest a set of three iterative phases containing nine steps. The first phase is known as intelligence. In this phase the goal of the research profile study is identified, the data sources are selected, data are refined and retrieved and they are finally cleaned. In the second phase, known as analysis and design, data are processed through basic and advanced analyses. The final phase called choice implies three steps: representation, interpretation, and utilization. Porter et al (2002) also suggest at least twelve steps to improve literature reviews ranging from depicting the research domain to identifying information sources and examining trends. In this research profiling study of MMORPG several of these steps have been adopted and adapted. For brevity, we will now concentrate on data retrieval and on analytical mining steps (zooming in and zooming out) in the next section.

Profiling MMORPG academic research

We examined abstract records retrieved from 16 databases⁸ and two web-portals using a reasonable search algorithm configured with various Boolean search operators⁹ or research phrases (Bragge & Stordgårds, 2007). An outcome of 1701 registers was cleaned and refined. The resulting dataset contained 777 records and this dataset was converted into a RefWork-RIS file. We imported the resulting dataset into VantagePoint text mining software.

VantagePoint is a powerful text-mining tool for discovering knowledge in search results from patent and literature databases. VantagePoint helps [ to ] rapidly understand and navigate through large search results, giving … a better perspective—a better vantage point—on … information. The perspective provided by VantagePoint enables … to quickly find WHO, WHAT, WHEN and WHERE … to clarify relationships and find critical patterns— turning … information into knowledge. (http://theVantagePoint.com)
Table 2 characterizes the dataset.

These registers or PDs (Primary Documents henceforth) were also cleaned up within VantagePoint avoiding repetition plus spelling was normalized and unified. Keeping in mind the idea of getting to know the what, when and who constitute scholarly work (Pailey, 1990) around MMORPG and the idea of KDD posed by Losiewicz et al (2000), we followed and adapted Bragge's & Stordgårds' (2007) questions when it comes to research profiling studies using as a main source the top ten criteria. We then designed a path of correlational clusters to answer the following set of questions:

• What is being sought when researching MMORPG?
• When has MMORPG research been conducted?
• Who has researched MMORPG?
• Where has MMORPG research been conducted?

The path of correlational clusters is illustrated in Figure 2. We decided first on exploring the top ten key words used by authors in the resulting data set. Bearing this in mind, we used the two key words more related to the domain we were profiling (MMORPG and Massive Multiplayer Online Game) and made more in-depth and broader analytical and interpretive searches (Porter et al, 2002). After that we designed four different correlational clusters combining analytical options to give a representational account of who has studied MMORPG and when and where this has been done.

We now present the research profiling study on MMORPG answering directly the set of questions posed above. We drilled down information from the title, abstract and key words in the dataset.

What is sought when researching MMORPG?

In Figure 3 we display the 10 most used key words by authors. It is worth noticing that out of 3121 (100%) key words used in the 777 PDs, 792 key words (25.4 %) are distributed along this most-preferred key word selection. Interestingly, the seventh position is occupied by the acronym MMORPG, which is what one might suspect is the most directly related key word to this research profiling study. This is followed by massive multiplayer online game which corresponds to the short form MMOG.

As the data in relation to the key words behaved in this particular way, we followed the spirit of the third iterative phase by Porter et al (2002) and chose to drill down the two more profile-related key words (MMORPG and Massive Multiplayer Online Game) with the goal of finding out about how they correlated to the other eight most-used key words.

The software VantagePoint offers the possibility of elaborating on a visualization of such correlations through Aduna cluster maps. ''The Aduna Cluster Maps serve as a detailed, interactive 'Venn Diagram' co-occurrence visualization'' (http://www.thevantagepoint.com/resources/training-materials/631working-with-aduna-cluster-maps.html). This was very useful as we started zooming in into each correlation (e.g. MMORPG and computer game; MMORPG and online system, etc., and Massively Multiplayer Online Game and computer game; Massively Multiplayer Online Game and online system, etc.) finding out topical co-occurrences which led us to determine qualitatively at least four constant research interests related to the key words MMORPG and Massive Multiplayer Online Game. These interests appear to include i) the game itself as a researchable object, ii) game experiences, iii) systems architecture and iv) educational MMORPG. For brevity, we have chosen not to incorporate in this report statistical representations of the clusters of each individual correlation. We would rather present a detailed view of the topics seemingly researched around MMORPG as we considered this more fruitful for the readers interested in this research profiling study giving special attention to the fourth area related to education. PDs' IDs will be used thoroughly to substantiate the findings that correspond to a correlational cluster we used to answer our first question.

The game itself and the key word MMORPG

To start with, a focus on the games as narrative can be identified as the self-contained stories the games have that are repeated with different functions. For example, the study of narrative structure from categories such as medieval and magic (PD 551¹⁰) and narrative analysis to understand the concept of time are recurrent (PD 137). There is also a research interest on the usability of the games (PD 1175) and on the analysis of specific well-known games (e.g. World of Warcraft) from the perspective of their financial gain (PD 980). Ontological models are also proposed to investigate these games (PD 2177).

In addition, concepts about the significance of the games (PD 2183) and of their system and metrics are discussed (PD 816). Finally, the issue of security within the game itself is addressed (PD 972). This opens room for educational research as well. It seems interesting to start exploring how MMORPG are closely related to education in different school subjects and educational levels. In such case narrative inquiry could be studied in the educational field from the perspective of gaming.

The game itself and the key word Massive Multiplayer Online Game

A small number of studies focus on this trend drawing on different perspectives. For example, description of these games can be found according to type and range (PD 642). Interestingly, one study discusses the management policies that virtual environments have (PD 1962) and another study unveils the economic models underlying these games (PD 558). These games also appear to be addressed from a critical perspective of technology (PD 845) and from a methodological perspective where the relationship among artifacts, technology, gaming experiences and sociability is considered (PD 2536). This is also important to be addressed in educational research as e-learning and e-teaching need to be explored from the perspective of role-play gaming within the critical framework.

Game experiences and the key word MMORPG

There seem to be three trends with regard to gaming experiences: players, experiencing specific games and community building. Initially, the focus on the player is clear-cut. Players are studied from different perspectives and interests: players' performance levels in a specific game (PDs 1536, 226, 989), the ratio of player characters (PD 674), motivations (PDs 2202, 58, 2224), understanding of narrative and justification of actions (PDs 2023, 1123), quality of play (PD 840), customization of game experiences (PD 183), the construction of feelings (PD 2350), addiction and crime (PDs 1003, 881), cognitive problems resulting from the games (PD 896), behavior and impact on traffic in the game (PD 172), social interaction (PDs 103, 120) and characterization of the self and envy (PD 759). Studies are also concentrated on how specific games are experienced, where World of Warcraft is the most studied one (PDs 2222, 64, 2555). Other games studied include, for example, EverQuest (PDs 787, 2269), Final Fantasy XI (PD 2264), The World of William Shakespeare (PD 1071), Shenzhou Online and Dragon Kill Points System (PD 1036), among others. Issues related to community building are also being investigated. Some of the most outstanding topics include online criminal activity (PD 75), the practice of marriage (PD 2571), the formation of youth cultures (PD 2255), the experience of conflict (PD 2269), and the formation of clans and their cultures (PD 539). There are other related interests that highlight the relationship of contextual elements (PD 791), statistical properties of the avatars (PD 793) and the conceptualization of fraud as a practice (PD 2017). Within such scope of topics there seems to be potentially a place for educational research. Research trends in this field might be directed towards investigating types and modes of interaction that foster myriad ways of learning. A game such as World of Warcraft appears to be a great arena where educational research could be conducted. This could be the basis for creating and/or replicating related MMORPG that focus on educational goals.

Game experiences and the key word Massive Multiplayer Online Game

Interestingly, there is a tendency to investigate specific games from different perspectives that include the cognitive processes at play (PDs 1553, 857), the culture of the game (PD 1017), the player itself (PD 1159, 1066) and collaborative game (PD 1550). This tendency on the collaborative aspect is also explored within the constitution of gaming communities (PD 1615, 988) and interaction among players (PD 550, 1098). There is a tendency to investigate the player's experience from its narrative and thematic understandings of what is proposed by the games (PD 2222). At the educational level this could be profitable to investigate collaborative pedagogy and communities of learning.

Systems architecture and the key word MMORPG

This key word is linked to a lesser extent with elements of the systems architecture; in this particular relationship 8 records were found. The most common themes are security features (PD 1666) (PDs 964, 1018), P2P architecture (PDs 883, 1666), the use of technological elements to improve the game (PD 556), game traffic (PD 906) and theory of architecture for these games (PD 2259).

Systems architecture and the key word Massive Multiplayer Online Game

This relationship with 30 records is the most common. This appears to be understandable if one takes into account the entire technological infrastructure that these devices require and the economic profit that they represent. Research interests can be organized into several themes: design models, traffic models and synchronization (PDs 1635, 2000), parameterization (PD 959) and security (PD 1544). The proposals on model designing are varied. Some opt for the client-server alternative (PD 1565); others propose not to use this alternative (PD 1570). The most studied designs seem to be the P2P ones (PDs 1613, 1592, 976, 1153, 1049, 1650, 1649). Other models highly discussed are cluster computing technology (PD 1576), multiple servers (PD 1031), microcells (PD 1120), hybrid schemes (PD 1141), migration of client server to P2P (PD 991), virtual collaboration architecture (PD 1058) and algorithmic models (PD 1598). There are several strategies or proposals that take into account technological elements to improve the games. For example, they include improving the sound (PD 1578), improving the possibility of a greater number of participants (PD 1645) and improving the implementation of mobile hardware (PD 1087). There is also a tendency to investigate traffic patterns (PD 1591) and the reduction of the bandwidth (PD 1052). This appears to set up the conditions to facilitate educational proposals as these technological improvements allow better player experiences where research could address their relationships with learning in any educational context.

Education and the key word MMORPG

An interest in research on the use of games in the classroom was identified (PDs 1547, 15, 163). There are also studies keen on the exploration of learning communities (PD 2272) and research on students learning to create educational games (PD 997).

Education and the key word Massive Multiplayer Online Game

The educational interest reveals the tendency to investigate mobile learning or m-learning (PD 1634) and tends to consider the game itself as a place where learning is generated (PD 857). Some trends appear to be indicative of research in the field of language learning around the development of skills such as listening (PD 1541) and vocabulary learning (PD 1516).

In our experience, we would like to argue that language teaching has been done mostly in a decontextualized way. What these games could provide is precisely a real context for communication that involves more than transferring information. By interacting through these games, it seems possible to negotiate meaning in the target language, to make decisions right at the moment of talking, to develop motor and thinking skills among other advantages. Skills development could probably overcome the technical phase and become more integrated in the contexts of the games. All these thoughts could potentially constitute new research ideas to enrich the applied linguistics to language learning/teaching field. MMORPG could constitute language learning laboratories that appear to be underexplored. In relation to other school subjects, MMORPG such as Whyville which is free and designed for learning and communication is a reference about how to undertake game research in educational contexts. This of course has been explored in first language environments but it appears necessary to unveil what happens when games like this one are played by speakers of other languages (PD 2139, 2334). There are more research possibilities in relation to learning when this is understood from other variables such as gender, race, social class, and origin among others.

When has MMORPG research been conducted?

In order to answer this question, which corresponds to the first correlational cluster illustrated in Figure 2, we chose to track how the two central key words (MMORPG and Massively multiplayer online game) have evolved over time. Figures 4 and 5 illustrate this evolution.

The two key words MMORPG and Massively Multiplayer Online Game are used in our screening for the first time in 2002. Drilling down it was found that both key words were framed within research concerned with their relationship regarding networked virtual environments (Net-VE) (PD 1645) and multimedia systems and their own applications (PD 1183). 2002 seems to be the year when the genesis of research related to architectural-game systems started. This also appears to be the case for research on the narrative structure of the games (PD 551). In the following year, 2003, concern for the improvement of client-server systems continued (PD 1565). 2003 seems to be an important date because the first investigations into gaming experiences from the perspective of applied social activities generated by the players themselves were begun (PDs 550, 549). 2003 also reveals for the first time an interest to research clan cultures (PD 539). Such interest is extended to research regarding the characteristics of the players in psychological terms (PD 2126).

It appears to be that academic research on MMORPG did not spread until 2004 when there seems to be a boost in the field. In that year, research is focused on creating devices (audio, for example) to optimize the experience of immersion in the game (PD 1578) and the research on gaming experiences continued, this time related to usability (PD 1172). In 2005 negative images in games are described (PD 2275), and the reflection on systems architectures continues (specifically, the P2P theme seems to come up for the first time this year) (PD 1153). A first attempt to characterize players' models (PDs 1159, 2202) which relate to structures of both games and narratives (PDs 2183, 2259), located geographically, (PD 2258) is made. In that direction, identities are investigated in specific games (PD 2264) along with peer culture (PD 2255) and conflict (PD 2269). Also in 2005, the interest to do research on devices continues, this time including mobility features in the environments of games (PD 556). Additionally, there is a new trend in which scholars study the games themselves from ontological and theoretical models (PD 2177). 2005 is also a milestone as it seems that for the first time these games are directly linked with an educational context from the perspective of the spontaneous emergence of learning communities (PD 2272). As it could be appreciated, these types of research studies are newly-born.

It was in 2006 when a number of investigations were devoted to studying software architecture (PD 1120), simple architecture schemes for virtual worlds (PD 1141), traffic levels (PD 906) and systems for servers (PDs 1144, 1136, 976) that make the games more effective. The research interest in documenting gaming experiences continues regarding practices of collaborative game (PD 1150), case studies (PDs 2555, 1580, 1123, 183) and interactions (PDs 103, 120). From the educational perspective the relationship of these games with learning processes is explored (PD 1547) and studies about the narrative structure of these games are carried out (PD 137).

The years 2007 to 2009 indicate a great growth in publications around MMORPG. In 2007 there is an outburst of research studies dealing with systems architecture. The trend revolves around game systems from the perspective of architecture, integration of technologies and devices for the improvement of the games (PDs 2000, 1581, 1092, 1649, 1613, 1663, 1655, 1107, 1094, 1087, 1576, 883, 161, 964). Specific gaming experiences and virtual-reality relationships are explored (PD 2222, 1103). More gaming experiences are also studied in relation to specific topics such as cognitive overload problems on players (PD 896), their cognitive processes (PD 1080) and case studies of specific games (PDs 64, 2222). Other topics studied this year are those related to cyber-crime (PD 75), marriage (PD 2571), equity (PD 157), the virtual property protection (PD 881), and selfishness (PD 759); all of them as forms of gaming experiences. There is also the tendency to investigate the business models around these games (PDs 558, 224). Finally, it is during this year when there is a concern about the role of these games in the field of language teaching (PD 1541). This seems to be a whole research area to be explored. Topics relating not only skills development but also identity formation, language learners' social structures, literacy development appear to be fruitful. Language teachers' roles when instructing via MMORPG could also be investigated along with language teacher education.

The year 2008 is a milestone when it comes to research regarding the development of systems architecture with topics such as P2P (PDs 1049, 1635, 1031, 1045, 1592, 1666), traffic patterns (PD 1591), virtual architecture (PD 1058), anti-fraud designs (PD 1544), technology applied to the bandwidth (PD 1052) and the optimization of servers (PD 1029). Additionally, research exploring the experiences of play that include use of the avatar (PD 858) and case studies (PDs 1071, 1036, 791, 980) were also conducted along with research on the formation of clans (PD 1066) and behavioral profiles of the players (PDs 1553, 674). At the educational level, there are studies regarding mobile learning (m-learning) (PD 1634) and new literacies (PDs 857, 15). Finally, game policies are the focus of a new trend of research around these games (PD 1652).

Last but not least, in 2009 the major research trend was gaming systems architecture in relation to the generation of content for games (PD 959) aspect which is very relevant for the educational field, client-server relations (PDs 1650, 991, 1598), the management of fraud (PD 1570 ), P2P (PD 835), and software and security features (PDs 972, 1018). There is also research on gaming experiences (PDs 2536, 1017), which include the network structure or work of players (PDs 995, 988), the motivations of the players, their relationships and understanding of game narratives (PDs 58, 2023), their emotional ties with games (PD 2350), their pathological addiction to games (PD 1003), their cheating practices (PD 2017), their levels of activity within games (PDs 989, 226), and the impact on traffic levels that players' behavior has (PD 172 ). Games in particular are investigated as case studies (PD 787). The use of these games in the context of second language learning emerges again (PD 1516) as the games provide opportunities to improve language proficiency at least at the vocabulary level. Motivational factors related to education (163) and the creations of learning environments by students (PD 997) are also searched. Finally, the trend of studying the games themselves continues (PDs 642, 845, 1590, 840, 816).

These findings reveal when and what has been researched on MMORPG. We decided on presenting these results from a zooming out and a zooming in perspective. We zoomed out with a panoramic quantitative view showing how the interest to do research on these games started back in 2002 and how publications were issued yearly up to 2009. We then decided to be more qualitative by zooming in and pulling out from the data what has been researched on these games within that period. This appears to be very informative as we were able to trace out when different trends of research started.

Who has researched MMORPG?

Various processes were taken to answer this question corresponding to clusters three and four in Figure 2. The more prolific authors were correlated to the two subject key words (MMORPG and Massive Multiplayer Online Game). After this, we studied these authors using their h-index. At the same time, the subject areas corresponding to each one of them in SCOPUS and in ISI were examined. Finally, we correlated the whole dataset's authors with the two subject keywords.

Top 10 authors

The most productive authors were zoomed out of the dataset correlating them to the two subject key words. Figures 6 and 7 display these results. Evidently, these authors use the key word Massive Multiplayer Online Game preferentially. It appears clear in Figure 7 that Shervin Shirmohammadi is the most prolific author.

Authors' h-Index

It was found that Stefano Ferretti's h-index is 16 and 19 in ISI and SCOPUS respectively. From this data it is not clear if this h-index has been calculated as co-author or main author as the author appears to have different affiliations. It is noticeable that Shirmohammadi, who appears to be the most productive author, does not have a higher h-index. One could probably speculate that his production did not have a strong impact at the time of writing this paper. Gorlatch has measured 5 in his ISI h-index getting 95 citations¹¹. This author gets 8 in his SCOPUS h-index. Interestingly, Kafai has 10 in her SCOPUS h-index compared to her ISI h-index that amounts to only 1 where one could probably assume her publications have not been referenced. These results are illustrated in Table 3.

Authors' Subject Areas

In order to keep on research-profiling these authors, we decided to examine the subject areas attributed to them as displayed by SCOPUS. This is illustrated in Table 4.

It could be argued that these authors appear to be the more prolific in relation to MMORPG research in the Computer Science subject area mainly. It also seems that the most researched topic is systems architecture as this is shared by eight of the authors (Shirmohammadi, Ferretti, Safaei, Ahmed, Roccetti, Stern, Chen and Gorlatch). Chen and Stern however do research on the games themselves. Game experiences are also a topic shared by Kafay and Chen. These findings are interesting and perhaps very informative for those keen on knowing about systems architecture in relation to the research profiling study we are presenting here. However, there remains a question in relation to the other research topics or interests we identified earlier. What do less prolific authors research? We would like to argue that the less prolific authors also add to this field of knowledge we are profiling in this study and in order to substantiate this argument we drilled-down the dataset again but this time correlating the two subject key words with the total universe of authors. It is in this way how educational research related to MMORPG emerges, otherwise it would not be that visible. This truly constitutes an opportunity to shape the field.

Correlation between the two subject key words and the dataset's authors

It was found that some authors (not necessarily the top 10) did have an interesting correlational value to the two subject key words (MMORPG and Massive Multiplayer Online Game). This is easily identifiable in VantagePoint as data is graphically marked with one, two or three arrows, three being the highest correlational value. Table 5 displays the highest correlational values. A panoramic and quantitative view of how the dataset behaved was chosen to display this data as registered in the first column; authors are presented in the second column. A brief comment is made regarding these authors' research interests in relation to MMORPG in the last column.

With the exception of three authors (Chen, C.H., Du and Liu), who have also investigated around systems architecture, it was found that the group of authors, as shown in the table above, is highly correlated with the subject key words from a particular research interest around the experiences of play and this, we believe, charts out an important trend. The interest in the educational field is also revealed from the examination made of the correlation mentioned, although to a lesser degree, in five authors (Pathak, Lee, Huang, Shen and Contractor). A similar trend is evident for research on the games themselves with authors like Eladhari, Lee, Huang, Lei and Chen. The coincidence of authors' names in some of the research interests identified (e.g. Lee and Huang) is conducive to wondering who works with whom (co-authoring). Finding out an answer to this query would also make it easier to answer the last question of the profile which is related as regards the where.

Who works with whom on MMORPG?

We opted to text-mine the data creating a sub-dataset made up from the top 10 authors and the dataset's authors highly correlated with the two subject key words. Then we run an auto-correlation in the software which is displayed in the aduna map (Figure 8).

The aduna map in Figure 8 clearly shows how collaborative work has been carried out between authors such as Srivastava, Ahmad, Shim, Pathak, Contractor, Shen, Huang and Keegan. It is noteworthy that Shim, Ahmad, Pathak and Srivastava are co-authors working on player's performance levels as experienced in a specific game (PD 989). It is also important to notice how Pathak, Contractor and Keegan investigate practices in group task-oriented work (PD 1615). Ahmad and Contractor, along with several other co-authors, made up another case of collaboration working on the social behavior of expert players (PD 1561). It is clear that the trend here is to investigate game experiences. Enclaves, clans and cultures are investigated by Sun and Lin (PDs 539, 2258). Additionally, Lee and Song investigate usability as a game experience and as an approximation to research the games themselves (PD 64). The high level of cooperative production between Shirmohammadi and Ahmed with nine common publications around systems architecture (PDs 1039, 1009, 1057, 1058, 1635, 1624, 1016, 1598, 1663) is also noteworthy. Lei, Chen and Huang also shared four publications related to systems architecture (PDs 171, 885, 906, 840). Along the same line of thought, Roccetti and Ferretti share seven publications that deal with solutions to problems of synchronization in the design of the games (PDs 1020, 1422, 1131, 1128, 1136, 1172, 1063).

As can be inferred perhaps, the research trend mostly recognized by the software is the network of research work on topics related to game experiences and systems architectures. This does not mean that the other authors cited above have not done collaborative work. Indeed they have, but they are not read in such way by the application. It should be remembered that all these authors are included in this part because they share the fact of being highly correlated with the two subject key words most related to the research profiling study we are reporting on. This also signals that networked research in education is much needed at this time. After zooming in and out in relation to who has researched on MMORPG, we will finish the second part of this report by providing an answer to the last question we posed.

Where has MMORPG research been conducted?

Table 6 shows the places where there is a greater concentration of the work described by the authors listed in the previous section.

Figure 9 shows how the institutions conducting research on MMORPG are US universities. There is however a research interest spread around the world with diverse countries investigating the topic such as Canada, Italy, Taiwan, Australia, Iran and Germany. The lack of research on this topic in Latin American countries and in some Eastern European and Asian countries in the selected time period for this research profiling study (2000-2009) is noteworthy. Some key questions might involve inquiring as to what has been done in the field in Latin America and other countries and whether or not this topic is relevant for specific regional academic communities.

Issues and options around profiling research on MMORPG

This final section is divided into two parts. Firstly, we will describe the results found in this research profiling study as defined in the second part of this paper. Secondly, we will provide a few conclusions and implications for future research.

Findings

Keeping in mind the iterative spirit for research profiling studies claimed by Bragge & Stordgårds (2007), we used text-mining techniques in which we zoomed in and out on the academic research on MMORPG that we had collected in a single dataset configured from an important number of databases. In order to dig out the dataset we designed a ''path'' (see Figure 2) where we planned on answering a specific set of questions to profile the what, when, who and where that constituted scholarly work in a patterned way (Paisley, 1990) on research about MMORPG. Regarding the ''what'', we drilled-down the two most related keywords (MMORPG and Massive Multiplayer Online Game) and inferred four research trends that take account of i) the game itself as a researchable object, ii) game experiences, iii) systems architecture and iv) educational MMORPG and all of them could be in one or another direction be related to educational research. We then zoomed in specifying what has been researched on each one of these trends. Based on our findings we would like to argue that these research trends constitute emerging research interests not yet documented in specialized lists (e.g. UNESCO thesaurus) as we also explored subject areas later on and found that most of the time the key words MMORPG and Massive Multiplayer Online Game are related to computer science. Our findings however suggest a wider spectrum of sub-topics as described in the second section of this paper which are highly related to education. With regard to the ''when'', it was found that the research topic at stake started being explored in 2002 and that every year up to 2009 there was an initial emergence of diverse research interests. We were able to trace specific moments in which research about explicit topics started to be published (e.g. gaming experiences, language learning, etc.). It is, however, undeniable that through the selected period the most researched topic is apparently related to systems architecture. Moving on to the ''whom'', it was easy to extract from the dataset the most prolific authors providing a panoramic view. We decided to dig out the data however and found out what less productive authors (at the time of writing) were researching. This was beneficial for this research profiling study as we found an important number of authors who highly correlated to MMORPG and Massive Multiplayer Online Game without being placed at the top ranking. We found that these authors mostly researched, with different levels and scopes, game experiences, the games themselves and educational MMORPG. This was also important as it was conducive to the observation of networks. Whilst some collaboration networks were described, it seems important to strengthen collaborative work at least in areas such as education in relation to MMORPG as it seems that the collaborative work around game experiences and systems architecture is more solid from this perspective. As to the ''where'', it was easy to get a panoramic view of organizations working on the type of research we profiled. It seems that some countries still need to make their research projects more visible as they do not emerge in the dataset. In a very concise way, Table 7 summarizes the KDD (Losiewicz et al, 2000) we generated from this research profiling study on MMORPG:

The findings fully described in the second part of this paper show a profile that could be potentially helpful to ''augment'', in Porter's et al's words (2002), the specialized literature reviews. This of course greatly depends on the scholars' research interests that could profit from this study. As stated before, it was our intention to contribute to seeing clearly the forest from the trees (Bragge & Stordgårds, 2007). We would also like to highlight once again that there are differences and complementarities between academic research profiles and literature reviews.

Conclusions and implications for future research

This paper sought to profile academic research on MMORPG. As stated earlier, we would like to contend that the findings described could add to the configuration of more informed literature reviews and to the formulation of potential research questions in relation to education and language learning. There are of course limitations to the study. We opted to answer four questions generalizable to most research profiling studies. Nevertheless, it would be important to also consider in detail for future research Losiewicz's et al's (2000) set of questions to stimulate knowledge generation from databases. As pointed out, using multiple databases was ambitious and as they were not standardized this caused not all fields to be taken in by VantagePoint. We did however recur to other strategies to drill-down the data (e.g. zooming in to identify patterns regarding research topics, consulting subject areas in well-known databases, etc.). In relation to education it seems clear that this is an emergent field where there is a few studies. However, it is necessary to highlight the topics that are weaving the field. Among them it could be found collaborative learning, m-learning, language learning, teaching methodologies, and there is a special concern for taking into account non-traditional features such as race, gender, origin, etc., as related to educational contexts within the MMORPG frame.

Pie de página

⁶Project ''State of the Art of Research on MMRPOG, 2000-2009'' ID 3623
⁷The term Knowledge Discovery in Databases (KDD) was first ''coined'' at the first KDD workshop in 1898 '[ ... ] to emphasize that knowledge is the end product of a data-driven discovery' (Fayyad et al, 1996, p. 39).
⁸The databases explored were: Digra, EBSCOhost, Emerald, IEEE/IET Electronic Library (IEL), ISI Web of Knowledge, JSTOR, Philosopher's Index, ProQuest, PsyscARTICLES (APA), Redalyc, Sage Premier, SciELO, ScienceDirect, Scopus, SpringerLink, Study Games, Wiley InterScience, WilsonWeb
⁹The algorithm (research phrase) used was particular to each database. For the informative purposes of this paper we provide a generic one: ''((COMPUTER GAM* OR VIRTUAL GAM* OR DIGITAL GAM* OR VIDEO GAM* OR GAME OR GAMES OR GAMING) AND ONLINE AND (MULTI-PLAYER* OR MULTI-USER* OR MULTIPLAYER* OR MULTIUSER*)) OR (MASSIV* MULTI-PLAYER* ONLINE GAM* OR MASSIV* MULTIPLAYER* ONLINE GAM* OR MASSIV* MULTIUSER* ONLINE GAM* OR MASSIV* MULTIUSER* ONLINE GAM* OR MULTI-PLAYER* ONLINE GAM* OR MULTIPLAYER* ONLINE GAM* OR MULTI-USER* ONLINE GAM* OR MULTIUSER* ONLINE GAM*)
¹⁰PD refers to the primary document or register in the dataset; this is followed by a number indicating the register ID. PDs bibliographic information could be requested to adriana.salazar@javeriana.edu.co and hcastan@javeriana.edu.co
¹¹ISI and SCOPUS were accessed on February 2011.

References

Bécue-Bertaut, M. (2010). Minería de textos: Aplicación a preguntas abiertas en encuestas. Madrid: La Muralla. Colección Cuadernos de Estadística.         [ Links ]

Bannigan, K., Droogan, J., & Entwistle, V. (1997). Systematic reviews: What do they involve? Nursing Times, 18, 52-53.         [ Links ]

Bragge, J., Stordgårds, J. (2007). Profiling academic research on digital games using text mining tools. Situated Play, Proceedings of DiGRA 2007 Conference, pp. 714-729.         [ Links ]

Castañeda-Peña, H. (2009). A game-specific approach to boyhood. Book Review: Burrill, D. (2008). Die tryin': videogames, masculinity, culture. Signo y Pensamiento, 28(54), 384-385.         [ Links ]

Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI Magazine, 17(3), 37-54.         [ Links ]

Glass, G., McGaw, B., & Smith, M. (1981). Metaanalysis in social research. Beverly Hills, CA: Sage Publications.         [ Links ]

Guerin, M. B., Flynn, T. H., Brady, J., & O'Brien, C. J. (2009). Worldwide geographical distribution of ophthalmology publications. International Ophthalmology, 29(6), 511-516.         [ Links ]

González-Romero, N., Salazar-Sierra, A., & Velásquez, A. (2009). Juego y cultura digital... ¿Qué se traen los juegos en línea? Signo y Pensamiento, 28(54), 369-376.         [ Links ]

Gupta, B. M., & Dhawan, S. M. (2009). Status of India in science and technology as reflected in its publication output in the Scopus international database, 19962006. Scientometrics, 80(2), 473-490.         [ Links ]

Kay, L., & Shapira, P. (2009). Developing nanotechnology in Latin America. Journal of Nanoparticle Research, 11(2), 259-278.         [ Links ]

Losiewicz, P., Oard, D. W., & Kostoff, R. N. (2000). Textual data mining to support science and technology management. Journal of Intelligent Information Systems, 15, 99-119.         [ Links ]

Manco, G., Masciari, E., & Tagarelli, A. (2008). Mining categories for emails via clustering and pattern discovery. Journal of Intelligent Information Systems, 30, 153-181.         [ Links ]

Menéndez-Echavarría, A., Castañeda-Peña, H., Salazar-Sierra, A., González-Romero, N., & Sierra-Gutiérrez, L. I. (2011). Proceso de búsqueda en fuentes de información digital. Investigación: ''Estado del arte de las investigaciones sobre juegos masivos en línea para múltiples jugadores (JMLMJ)''. Paper presented in the Tercer Encuentro de Bibliotecas en Tecnologías de la Información y la Comunicación - Bibliotic, June, Pontificia Universidad Javeriana, Bogotá, Colombia.         [ Links ]

Nahm, U. Y. (2004). Text mining with information extraction. Doctoral Dissertation. University of Texas, Austin.         [ Links ]

Ottenbacher, K. (1983). Quantitative reviewing: The literature review as scientific inquiry. American Journal of Occupational Therapy, 37, 313-319.         [ Links ]

Paisley, W. (1990). The future of bibliometrics. In Borgman, C. L. (Ed.), Scholarly Communication and Bibliometrics (pp. 281-299). Newbury Park, CA: Sage.         [ Links ]

Porter, A., Kongthon, A., & Lu, J. C. (2002). Research profiling: Improving the literature review. Scientometrics, 53(3), 351-370.         [ Links ]

Porter A.L., Cunningham S.W., (2005), Tech mining. Exploiting new technologies forcompetitive advantage, Wiley Interscience, New Jersey        [ Links ]

Stiles, R., & Mick, S. (1994). Classifying quality initiatives: A conceptual paradigm for literature review and policy analysis. Hospital & Health Services Administration, 39, 309-326.         [ Links ]

Thomas, H. (2009). Business schools and management research: A UK perspective. Journal of Management Development, 28(8), 660-667.         [ Links ]